# Web Scraping Start Batch Scrape API (https://www.socialcrawl.dev/platforms/web/batch-scrape)

> Starts an async batch scrape job for multiple URLs and returns a SocialCrawl job id (job_...). Poll or cancel it via GET/DELETE /v1/web/jobs/{job_id}.

TL;DR: `POST /v1/web/batch-scrape` costs 1 credit per URL submitted, held up front and refunded down to the URLs actually scraped when the job settles and returns SocialCrawl's unified JSON schema. Single x-api-key auth, 100 free credits on signup.

## Parameters

| Parameter | Required | Description |
| --- | --- | --- |
| `urls` | yes | Array or comma-separated list of public URLs to scrape. |
| `ignore_invalid_urls` | no | Skip invalid URLs instead of failing the job. |
| `formats` | no | CSV of output formats for each URL, e.g. 'markdown' (default) or 'markdown,html'. |
| `only_main_content` | no | Strip nav, headers, footers and sidebars on every URL and keep the article body. Default true. |
| `proxy` | no | Proxy tier applied to every URL: basic, auto, or enhanced. (basic \| auto \| enhanced) |
| `webhook_url` | no | Accepted but not used: a batch-scrape job sends no webhook. Poll GET /v1/web/jobs/{job_id} for its status. |

## Code example

```bash
curl -X POST "https://www.socialcrawl.dev/v1/web/batch-scrape" \
  -H "x-api-key: sc_YOUR_API_KEY" \
  -H "content-type: application/json" \
  -d '{"urls":["https://example.com/a","https://example.com/b"]}'
```

## FAQ

### What does the Web Batch Scrape API return?

Starts a job that scrapes a list of URLs you supply, returning a job id to poll for the results. The response uses the same JSON envelope as every SocialCrawl endpoint, with the data, credits_used and request_id fields.

### When should I use the Web Batch Scrape endpoint?

Use it when you already know every URL you want; use crawl instead when the site should be explored for you.

### How much does the Web Batch Scrape API cost?

Each call costs 1 credit, so the 100 free credits every new account gets cover 100 calls. Credits for failed or empty calls are refunded automatically, and there is no subscription.

### What parameters does the Web Batch Scrape API need?

The one required parameter is urls (array or comma-separated list of public URLs to scrape). It also accepts 5 optional parameters, all listed on this page.

### How do I call the Web Batch Scrape API?

Send a POST request to https://www.socialcrawl.dev/v1/web/batch-scrape with a JSON body and your API key in the x-api-key header. An Authorization: Bearer header works too.

See the full Web Scraping API: https://www.socialcrawl.dev/platforms/web

## Pricing

- Standard endpoints: 1 credit per call
- Advanced endpoints: 5 credits per call
- Premium endpoints: 10 credits per call
- 100 free credits on signup, no credit card required. Cached responses cost 0 credits. Credit packs never expire.
- Full pricing: https://www.socialcrawl.dev/pricing

## Explore with AI

Questions this API answers, phrased for an AI assistant:

- What does the SocialCrawl Web Batch Scrape API return?
- How do I get Web Batch Scrape data with the SocialCrawl API?