# Web Scraping Extract API (https://www.socialcrawl.dev/platforms/web/extract)

> Fetches one web page and returns structured extraction output under the WebPage extraction field. Every extracted value is checked against the page's own text: a placeholder such as 'Not specified', or a value the page does not state, comes back as null, and `page.extraction_check.removed` lists each such field with the value it had. A value stated in another language than the one you asked for is kept. Fields that could not be checked are kept as extracted and named in `page.extraction_check.unverified` and `_warnings`. Write the schema or prompt for what the page shows: a field the page never mentions will be null.

TL;DR: `GET /v1/web/extract` costs A flat 5 credits. It runs through the metered pricer for consistency with the rest of the web surface, but nothing in your query changes the charge and returns SocialCrawl's unified JSON schema. Single x-api-key auth, 100 free credits on signup.

## Parameters

| Parameter | Required | Description |
| --- | --- | --- |
| `url` | yes | Public URL to extract from. |
| `schema` | no | JSON object schema describing the data to return. |
| `prompt` | no | Plain-language extraction instruction. |
| `only_main_content` | no | Strip nav, headers, footers and sidebars and keep the article body. Default true; send false when you need the whole page, including the chrome. |
| `timeout` | no | Hard cap on each page load in milliseconds, 1 to 30000. A page that exceeds it fails and is refunded rather than returning a partial capture. |
| `max_age` | no | Accept an upstream-cached copy up to this many milliseconds old, which is faster and costs the same. Default 172800000 (48 hours); send 0 to force a live fetch. |
| `proxy` | no | Proxy tier: basic, auto, or enhanced. Raise it only when a site blocks the default tier - enhanced is slower. (basic \| auto \| enhanced) |

## Code example

```bash
curl -G "https://www.socialcrawl.dev/v1/web/extract" \
  --data-urlencode "url=https://example.com/pricing" \
  --data-urlencode "prompt=Extract plan names and prices." \
  -H "x-api-key: sc_YOUR_API_KEY"
```

## FAQ

### What does the Web Extract API return?

Returns structured fields pulled from one web page, shaped by the schema or plain-language prompt you supply. The response uses the same JSON envelope as every SocialCrawl endpoint, with the data, credits_used and request_id fields.

### When should I use the Web Extract endpoint?

Use it when you want specific values such as price or author rather than the page's full text.

### How much does the Web Extract API cost?

The price depends on the request, from 5 to 5 credits per call. Every new account gets 100 free credits. Credits for failed or empty calls are refunded automatically, and there is no subscription.

### What parameters does the Web Extract API need?

The one required parameter is url (public URL to extract from). Identify the target with one of schema, prompt. It also accepts 6 optional parameters, all listed on this page.

### How do I call the Web Extract API?

Send a GET request to https://www.socialcrawl.dev/v1/web/extract?url=https%3A%2F%2Fexample.com%2Fpricing&prompt=Extract%20plan%20names%20and%20prices. with your API key in the x-api-key header. An Authorization: Bearer header works too.

See the full Web Scraping API: https://www.socialcrawl.dev/platforms/web

## Pricing

- Standard endpoints: 1 credit per call
- Advanced endpoints: 5 credits per call
- Premium endpoints: 10 credits per call
- 100 free credits on signup, no credit card required. Cached responses cost 0 credits. Credit packs never expire.
- Full pricing: https://www.socialcrawl.dev/pricing

## Explore with AI

Questions this API answers, phrased for an AI assistant:

- What does the SocialCrawl Web Extract API return?
- How do I get Web Extract data with the SocialCrawl API?