# Web Scraping Scrape API (https://www.socialcrawl.dev/platforms/web/scrape) > Fetches a public web page and returns clean content, metadata, and optional media in the unified WebPage schema. When the page scrapers cannot open a social platform profile or post (a LinkedIn profile, company page or post; an Instagram, Facebook, TikTok, X or Threads profile or post; a Reddit post, subreddit or user page; a YouTube video or channel), the page is read through that platform's own endpoint, for example /v1/linkedin/profile, at that endpoint's price: 5 credits for LinkedIn, 1 for the others. `page.native` then carries the endpoint (`endpoint`), what it charged (`credits_used`) and its full structured record (`data`). `page.content.markdown`, `html` and `raw_html` are written from that record rather than taken from the site's HTML, and `_warnings` says so. Only markdown, html and rawHtml requests are read this way. A page the scrapers can read keeps the scrape and its price. TL;DR: `GET /v1/web/scrape` costs 1 credit for a standard scrape. `proxy=auto`, `proxy=enhanced` and `pdf_parse=true` hold 5 credits and settle to what the fetch actually used, so a page a plain request could read still costs 1. Nothing else moves the price: screenshots, tag filters, `wait_for` and a mobile viewport are included. A page that comes back as the site's own block or error page (HTTP 400 or more), a CAPTCHA, a sign-in wall or an empty shell is returned with `page_state` saying so and a `_warnings` line, and is not charged. When the page scrapers cannot open a LinkedIn, Instagram, Facebook, Reddit, TikTok, YouTube, X or Threads profile or post, the page is read through that platform's own endpoint and costs that endpoint's price instead: 5 credits for a LinkedIn profile, company page or post, 1 credit for the others. The scrape's own charge is refunded to 0, so you pay the platform price once, and Activity Logs show the platform call as its own row and returns SocialCrawl's unified JSON schema. Single x-api-key auth, 100 free credits on signup. ## Parameters | Parameter | Required | Description | | --- | --- | --- | | `url` | yes | Public URL to fetch. | | `formats` | no | Comma-separated output formats such as markdown,screenshot. A social platform URL is read through its own endpoint only for markdown, html and rawHtml. | | `only_main_content` | no | Strip nav, headers, footers and sidebars and keep the article body. Default true; send false when you need the whole page, including the chrome. A markdown-only scrape of a site the primary scraper cannot fetch is served by a second extractor, which returns the whole page. | | `wait_for` | no | Milliseconds to wait after load before capturing, for pages that render their content in JavaScript. Leave unset unless the page comes back empty or half-built. | | `mobile` | no | Render in a mobile viewport with a mobile user agent. Use it when the site serves a different layout to phones. | | `timeout` | no | Hard cap on the page load in milliseconds, 1 to 30000. A page that exceeds it fails and is refunded rather than returning a partial capture. | | `max_age` | no | Accept an upstream-cached copy up to this many milliseconds old, which is faster and still one credit. Default 172800000 (48 hours), or 0 (a live fetch) with `proxy=auto` or `proxy=enhanced`; send 0 to force a live fetch. A cached copy of a page that failed (HTTP 400 or more) is never served: it is fetched again, live. | | `location_country` | no | ISO 3166-1 alpha-2 country code to fetch from, e.g. 'us' or 'de'. Use it for geo-varying pages such as pricing or availability. | | `screenshot_full_page` | no | When a screenshot format is requested, capture the entire scrollable page instead of the visible viewport. | | `include_tags` | no | CSV of CSS selectors or HTML tags to keep, e.g. 'article,main'. Everything outside them is dropped. Use it to pin extraction to a known container. | | `exclude_tags` | no | CSV of CSS selectors or HTML tags to drop, e.g. 'nav,footer,.cookie-banner'. Applied after include_tags. | | `proxy` | no | Proxy tier: basic (the default), auto (a plain request first, then a stronger one if the site blocks it), or enhanced. Use auto when a plain scrape returns a block or CAPTCHA page. auto and enhanced read the page live unless you set max_age. (basic \| auto \| enhanced) | | `pdf_parse` | no | Parse the target as a PDF and return its extracted text. Send it when the URL points at a PDF rather than an HTML page. | | `block_ads` | no | Block ad and tracker requests, which is faster and quieter. Default true. | | `remove_base64_images` | no | Drop inline base64-encoded images from the returned content instead of carrying them in the payload. Default true. | ## Code example ```bash curl -G "https://www.socialcrawl.dev/v1/web/scrape" \ --data-urlencode "url=https://example.com" \ -H "x-api-key: sc_YOUR_API_KEY" ``` ## FAQ ### What does the Web Scrape API return? Returns one web page's content as clean markdown or HTML, with the resolved URL, status code, fetch metadata, and an optional screenshot. The response uses the same JSON envelope as every SocialCrawl endpoint, with the data, credits_used and request_id fields. ### When should I use the Web Scrape endpoint? Use it for a single known URL; for whole sites start a crawl job instead. ### How much does the Web Scrape API cost? The price depends on the request, from 1 to 5 credits per call. Every new account gets 100 free credits. Credits for failed or empty calls are refunded automatically, and there is no subscription. ### What parameters does the Web Scrape API need? The one required parameter is url (public URL to fetch). It also accepts 14 optional parameters, all listed on this page. ### How do I call the Web Scrape API? Send a GET request to https://www.socialcrawl.dev/v1/web/scrape?url=https%3A%2F%2Fexample.com with your API key in the x-api-key header. An Authorization: Bearer header works too. See the full Web Scraping API: https://www.socialcrawl.dev/platforms/web ## Pricing - Standard endpoints: 1 credit per call - Advanced endpoints: 5 credits per call - Premium endpoints: 10 credits per call - 100 free credits on signup, no credit card required. Cached responses cost 0 credits. Credit packs never expire. - Full pricing: https://www.socialcrawl.dev/pricing ## Explore with AI Questions this API answers, phrased for an AI assistant: - What does the SocialCrawl Web Scrape API return? - How do I get Web Scrape data with the SocialCrawl API?