# Web Scraping Search API (https://www.socialcrawl.dev/platforms/web/search)

> Searches web, news, and image sources and returns a single normalized list of web page results.

TL;DR: `GET /v1/web/search` costs 2 credits per 10 results, so `limit` up to 10 costs 2, 11-20 costs 4, and the 100 maximum costs 20. `include_content=true` adds 1 credit per result, because it scrapes each result page as well as reading the SERP row - that is what takes a 100-result content search to the 120-credit ceiling. The hold settles down to the work actually done and returns SocialCrawl's unified JSON schema. Single x-api-key auth, 100 free credits on signup.

## Parameters

| Parameter | Required | Description |
| --- | --- | --- |
| `query` | yes | Search query. |
| `sources` | no | Comma-separated sources: web,news,images. |
| `categories` | no | CSV of source categories to restrict the search to, e.g. 'github,research'. Omit for an unrestricted web search. |
| `limit` | no | Results per source, from 1 to 100. |
| `country` | no | ISO 3166-1 alpha-2 country code the search is run from, e.g. 'US' (default) or 'GB'. Changes which results rank. |
| `location` | no | Free-text locality to bias results towards, e.g. 'London, United Kingdom'. Finer-grained than country. |
| `time_range` | no | Recency window as a Google `tbs` value: 'qdr:d' past day, 'qdr:w' past week, 'qdr:m' past month, 'qdr:y' past year. |
| `sort_by_date` | no | Sort newest-first instead of by relevance. Combines with time_range rather than replacing it. |
| `include_domains` | no | CSV of domains to restrict results to, e.g. 'nytimes.com,ft.com'. Cannot be combined with exclude_domains - sending both is a 400 before billing. |
| `exclude_domains` | no | CSV of domains to drop from the results. Cannot be combined with include_domains. |
| `include_content` | no | Also scrape each result and return its page content as markdown, not just the SERP row. This is what makes the call expensive: it scrapes one page per result, which is why the range runs to 120 credits. |

## Code example

```bash
curl -G "https://www.socialcrawl.dev/v1/web/search" \
  --data-urlencode "query=social media data api" \
  -H "x-api-key: sc_YOUR_API_KEY"
```

## FAQ

### What does the Web Search API return?

Returns ranked web, news, and image results for a search query as one normalized list, with a title, URL, and snippet for each. The response uses the same JSON envelope as every SocialCrawl endpoint, with the data, credits_used and request_id fields.

### When should I use the Web Search endpoint?

Use it when you do not know yet which URLs you need; once you have a URL, scrape it.

### How much does the Web Search API cost?

The price depends on the request, from 2 to 120 credits per call. Every new account gets 100 free credits. Credits for failed or empty calls are refunded automatically, and there is no subscription.

### What parameters does the Web Search API need?

The one required parameter is query (search query). It also accepts 10 optional parameters, all listed on this page.

### How do I call the Web Search API?

Send a GET request to https://www.socialcrawl.dev/v1/web/search?query=social%20media%20data%20api with your API key in the x-api-key header. An Authorization: Bearer header works too.

See the full Web Scraping API: https://www.socialcrawl.dev/platforms/web

## Pricing

- Standard endpoints: 1 credit per call
- Advanced endpoints: 5 credits per call
- Premium endpoints: 10 credits per call
- 100 free credits on signup, no credit card required. Cached responses cost 0 credits. Credit packs never expire.
- Full pricing: https://www.socialcrawl.dev/pricing

## Explore with AI

Questions this API answers, phrased for an AI assistant:

- What does the SocialCrawl Web Search API return?
- How do I get Web Search data with the SocialCrawl API?