100 free credits. No credit card required
Universal Search Everywhere API
Returns one ranked, clustered result set for a query run in parallel across Reddit, X, YouTube, TikTok, Instagram and more, with top comments and each post's stance and relevance attached.
GET /v1/search/everywhereLast updated November 2026Maintained by the SocialCrawl team
When to use the Universal Search Everywhere API
Use it when you want a topic's coverage everywhere at once instead of calling each platform's search endpoint yourself.
Not for a platform you already know, whose own search endpoint is direct, nor for forums only (search/forums), news only (search/news) or creator discovery (search/creators).
From the API reference
One query, searched on every platform at once, returns one ranked list with clusters and top comments. It searches up to 20 sources across 15 platforms: Reddit (reddit), X posts (twitter), a written summary of X posts (twitter-ai-search), YouTube (youtube), TikTok (tiktok), Instagram (instagram), Hacker News (hackernews), Polymarket (polymarket), GitHub (github), Threads (threads), Pinterest (pinterest), a cited web answer (perplexity), web search results (web), LinkedIn (linkedin), Rumble (rumble), Naver (naver) and Facebook (facebook), plus TikTok hashtags (tiktok-hashtag), Instagram hashtags (instagram-hashtag) and YouTube hashtags (youtube-hashtag) in hashtag mode. naver (Naver blog, cafe and KnowledgeiN) joins when the query contains Hangul, and on any query when you name it in sources; Naver rows carry the corpus in metadata.corpus. facebook runs only when you ask for it: include=facebook adds it to the default sources, and sources=facebook runs only the sources you name; it returns about 5 public posts, newest first inside your date window. The hashtag lanes run only in hashtag mode: an explicit #tag, a single word, or a topic term the planner names. A source you name in sources is always searched. What ranks and what filters: lookback_days is a ranking signal that sets the freshness curve, so an older row can still rank when nothing recent matches; since, until, from_date and to_date are hard bounds that drop rows outside the window (rows with no date are kept and counted in data.since); relevance=filter drops the rows judged off topic; limit (1 to 40) trims the answer. Every result carries computed.relevance and computed.stance, and data.judgments says whether each judgment ran. Top comments ride the results on the six lanes that have them (Reddit, Hacker News, YouTube, TikTok, Instagram and GitHub) at items[i].source_items[0].metadata.top_comments[]: up to 15 on each of the top 10 results and 5 after, each { score, excerpt, truncated, author, url, date }. X posts (twitter, also x) are ranked rows. The X summary (twitter-ai-search, which takes 15 to 45 seconds) runs only with include=synthesis and arrives in data.synthesis.x beside the cited web answer (data.synthesis.web), never as ranked rows. On the JSON path the X summary is included only when it is ready with the rest of the answer, and otherwise sources_failed says so; the stream waits for it. Every source is a call to its own public endpoint, with that endpoint's fallbacks, inside the flat price: a source whose endpoint keeps failing is skipped (sources_failed says skipped: breaker_open), and data.platforms reports each platform's status, rows, latency and outcome code. Each source has 12 seconds (25 seconds when you search one platform, 20 seconds for facebook, 6 seconds for twitter beside other sources), and a full search usually takes 20 to 30 seconds: allow a 120-second client timeout. Answers are cached for 120 seconds, and a cache hit is free. Each row's date_confidence and date_source say how far to trust published_at. Streams over SSE with Accept: text/event-stream (the terminal done chunk carries the page blocks of the JSON answer except platforms and synthesis; on the stream the web answer and the X summary arrive as items chunks), or answers JSON with Accept: application/json. Flat 20 credits per call: an answer with no row anywhere costs 0 credits, and when fewer than half the planned sources answer, half is refunded. A parameter this endpoint does not take is ignored with a data._warnings line, never refused.
Searching 69 platforms in parallel
·
TikTok·
Instagram·
YouTube·
Facebook·
X·
LinkedIn·
Reddit·
Threads·
Pinterest·
Twitch·
Truth Social·
Snapchat·
Kick·
Bluesky·
Kwai·
Rumble·
Spotify·
Apple Music·TikTok Shop·
Amazon Shop·
Google Shopping·
Trustpilot·
TripAdvisor·
Yelp·
Linktree·
Komi·
Pillar·
lnk.bio·Facebook Ads·Google Ads·LinkedIn Ads·Google Search·Google News·
Finance·
Polymarket·
Tavily·
Hacker News·
GitHub·
Perplexity·
Naver·
Utility·Universal Search
TikTok·Instagram·YouTube·Facebook·X·LinkedIn·Reddit·Threads·Pinterest·Twitch·Truth Social·Snapchat·Kick·Bluesky·Kwai·Rumble·Spotify·Apple Music·TikTok Shop·Amazon Shop·Google Shopping·Trustpilot·TripAdvisor·Yelp·Linktree·Komi·Pillar·lnk.bio·Facebook Ads·Google Ads·LinkedIn Ads·Google Search·Google News·Finance·Polymarket·Tavily·Hacker News·GitHub·Perplexity·Naver·Utility·Universal Search·TikTok·Instagram·YouTube·Facebook·X·LinkedIn·Reddit·Threads·Pinterest·Twitch·Truth Social·Snapchat·Kick·Bluesky·Kwai·Rumble·Spotify·Apple Music·TikTok Shop·Amazon Shop·Google Shopping·Trustpilot·TripAdvisor·Yelp·Linktree·Komi·Pillar·lnk.bio·Facebook Ads·Google Ads·LinkedIn Ads·Google Search·Google News·Finance·Polymarket·Tavily·Hacker News·GitHub·Perplexity·Naver·Utility·Universal Search
TikTok·Instagram·YouTube·Facebook·X·LinkedIn·Reddit·Threads·Pinterest·Twitch·Truth Social·Snapchat·Kick·Bluesky·Kwai·Rumble·Spotify·Apple Music·TikTok Shop·Amazon Shop·Google Shopping·Trustpilot·TripAdvisor·Yelp·Linktree·Komi·Pillar·lnk.bio·Facebook Ads·Google Ads·LinkedIn Ads·Google Search·Google News·Finance·Polymarket·Tavily·Hacker News·GitHub·Perplexity·Naver·Utility·Universal Search·TikTok·Instagram·YouTube·Facebook·X·LinkedIn·Reddit·Threads·Pinterest·Twitch·Truth Social·Snapchat·Kick·Bluesky·Kwai·Rumble·Spotify·Apple Music·TikTok Shop·Amazon Shop·Google Shopping·Trustpilot·TripAdvisor·Yelp·Linktree·Komi·Pillar·lnk.bio·Facebook Ads·Google Ads·LinkedIn Ads·Google Search·Google News·Finance·Polymarket·Tavily·Hacker News·GitHub·Perplexity·Naver·Utility·Universal Search
TikTok·Instagram·YouTube·Facebook·X·LinkedIn·Reddit·Threads·Pinterest·Twitch·Truth Social·Snapchat·Kick·Bluesky·Kwai·Rumble·Spotify·Apple Music·TikTok Shop·Amazon Shop·Google Shopping·Trustpilot·TripAdvisor·Yelp·Linktree·Komi·Pillar·lnk.bio·Facebook Ads·Google Ads·LinkedIn Ads·Google Search·Google News·Finance·Polymarket·Tavily·Hacker News·GitHub·Perplexity·Naver·Utility·Universal Search·TikTok·Instagram·YouTube·Facebook·X·LinkedIn·Reddit·Threads·Pinterest·Twitch·Truth Social·Snapchat·Kick·Bluesky·Kwai·Rumble·Spotify·Apple Music·TikTok Shop·Amazon Shop·Google Shopping·Trustpilot·TripAdvisor·Yelp·Linktree·Komi·Pillar·lnk.bio·Facebook Ads·Google Ads·LinkedIn Ads·Google Search·Google News·Finance·Polymarket·Tavily·Hacker News·GitHub·Perplexity·Naver·Utility·Universal Search
TikTok·Instagram·YouTube·Facebook·X·LinkedIn·Reddit·Threads·Pinterest·Twitch·Truth Social·Snapchat·Kick·Bluesky·Kwai·Rumble·Spotify·Apple Music·TikTok Shop·Amazon Shop·Google Shopping·Trustpilot·TripAdvisor·Yelp·Linktree·Komi·Pillar·lnk.bio·Facebook Ads·Google Ads·LinkedIn Ads·Google Search·Google News·Finance·Polymarket·Tavily·Hacker News·GitHub·Perplexity·Naver·Utility·Universal SearchUniversal Search API
What can you do with the Everywhere API?
The Everywhere endpoint gives you structured Universal Search data with computed fields in a single request. No scraping infrastructure to build or maintain.
Example Request
# GET /v1/search/everywhere: 20 credits per call. Cache hits are free and failed calls are refunded.
# Check the price first for 0 credits: add dry_run=1 to the query. Nothing is fetched or charged.
# --max-time 38: this endpoint's measured p99 latency x 1.5.
# --retry 3: retries timeouts, 429 and 5xx, honoring Retry-After.
# -o response.json keeps only the final attempt's body.
# One Idempotency-Key per request, reused on every retry, so a retried call is never charged twice.
IDEMPOTENCY_KEY=$(od -An -N16 -tx1 /dev/urandom | tr -d ' \n')
curl -sS --max-time 38 --retry 3 -o response.json \
-H "x-api-key: YOUR_API_KEY" \
-H "Idempotency-Key: $IDEMPOTENCY_KEY" \
"https://www.socialcrawl.dev/v1/search/everywhere?query=kanye+west&lookback_days=30&sources=reddit%2Cyoutube%2Cgithub&include_transcripts=false" && cat response.json# GET /v1/search/everywhere: 20 credits per call. Cache hits are free and failed calls are refunded.
# Check the price first for 0 credits: add dry_run=1 to the query. Nothing is fetched or charged.
import os
import random
import time
import uuid
import requests
API_KEY = os.environ.get("SOCIALCRAWL_API_KEY", "YOUR_API_KEY")
TIMEOUT_S = 38 # this endpoint's measured p99 latency x 1.5.
MAX_RETRIES = 3
URL = "https://www.socialcrawl.dev/v1/search/everywhere"
def backoff(attempt):
# Exponential backoff with jitter: about 1 s, 2 s, then 4 s.
return 2 ** attempt * (0.5 + random.random() / 2)
def socialcrawl(params):
# One Idempotency-Key per request, reused on every retry, so a retried call is never charged twice.
idempotency_key = str(uuid.uuid4())
for attempt in range(MAX_RETRIES + 1):
try:
res = requests.get(
URL,
params=params,
headers={"x-api-key": API_KEY, "Idempotency-Key": idempotency_key},
timeout=TIMEOUT_S,
)
body = res.json()
except (requests.RequestException, ValueError):
# A timeout, a dropped connection or a non-JSON gateway page: retry with the same key.
if attempt == MAX_RETRIES:
raise
time.sleep(backoff(attempt))
continue
if body["success"]:
return body
error = body["error"]
# Branch on error["type"], never on the message text.
if error["type"] == "RESOURCE_NOT_FOUND":
return None # refunded: nothing to read
retry_after = res.headers.get("Retry-After")
# SERVICE_UNAVAILABLE without Retry-After is a withdrawn endpoint: permanent.
permanent = error["type"] == "SERVICE_UNAVAILABLE" and retry_after is None
if not error.get("retryable") or permanent or attempt == MAX_RETRIES:
raise RuntimeError(f"{error['type']}: {error['message']} (request_id {body['request_id']})")
# Retry-After (seconds) wins; otherwise back off.
time.sleep(float(retry_after) if retry_after else backoff(attempt))
params = {
"query": "kanye west",
"lookback_days": "30",
"sources": "reddit,youtube,github",
"include_transcripts": "false",
}
body = socialcrawl(params)
if body is not None:
print(body["data"])// GET /v1/search/everywhere: 20 credits per call. Cache hits are free and failed calls are refunded.
// Check the price first for 0 credits: add dry_run=1 to the query. Nothing is fetched or charged.
const API_KEY = process.env.SOCIALCRAWL_API_KEY ?? "YOUR_API_KEY";
const TIMEOUT_MS = 38_000; // this endpoint's measured p99 latency x 1.5.
const MAX_RETRIES = 3;
type Page = { next_cursor: string | null; has_more: boolean };
type SocialCrawlResponse =
| { success: true; data: any; pagination?: Page; credits_used: number; request_id: string }
| { success: false; error: { type: string; message: string; retryable: boolean }; request_id: string };
const sleep = (ms: number) => new Promise((resolve) => setTimeout(resolve, ms));
// Exponential backoff with jitter: about 1 s, 2 s, then 4 s.
const backoffMs = (attempt: number) => 2 ** attempt * 1000 * (0.5 + Math.random() / 2);
async function socialcrawl(url: URL) {
// One Idempotency-Key per request, reused on every retry, so a retried call is never charged twice.
const idempotencyKey = crypto.randomUUID();
for (let attempt = 0; ; attempt++) {
let res: Response;
let body: SocialCrawlResponse;
try {
res = await fetch(url, {
headers: {
"x-api-key": API_KEY,
"Idempotency-Key": idempotencyKey,
},
signal: AbortSignal.timeout(TIMEOUT_MS),
});
body = (await res.json()) as SocialCrawlResponse;
} catch (error) {
// A timeout, a dropped connection or a non-JSON gateway page: retry with the same key.
if (attempt >= MAX_RETRIES) throw error;
await sleep(backoffMs(attempt));
continue;
}
if (body.success) return body;
// Branch on error.type, never on the message text.
if (body.error.type === "RESOURCE_NOT_FOUND") return null; // refunded: nothing to read
const retryAfter = res.headers.get("Retry-After");
// SERVICE_UNAVAILABLE without Retry-After is a withdrawn endpoint: permanent.
const permanent = body.error.type === "SERVICE_UNAVAILABLE" && retryAfter === null;
if (!body.error.retryable || permanent || attempt >= MAX_RETRIES) {
throw new Error(`${body.error.type}: ${body.error.message} (request_id ${body.request_id})`);
}
// Retry-After (seconds) wins; otherwise back off.
await sleep(retryAfter ? Number(retryAfter) * 1000 : backoffMs(attempt));
}
}
const url = new URL("https://www.socialcrawl.dev/v1/search/everywhere");
url.searchParams.set("query", "kanye west");
url.searchParams.set("lookback_days", "30");
url.searchParams.set("sources", "reddit,youtube,github");
url.searchParams.set("include_transcripts", "false");
const body = await socialcrawl(url);
if (body) console.log(body.data);Parameters
| Parameter | Required | Description |
|---|---|---|
| query | Yes | Search query (1-512 chars) |
| lookback_days | No | Recency window in days (minimum 1, default 30). This is a ranking signal, not a hard filter: it sets the freshness curve, so a much older item can still rank when nothing recent matches, and it sets the date window sent to `facebook`. When `from_date` is sent too, `from_date` applies. On the stream, windows over 90 days add a `warning` chunk. For hard bounds use `since`, `until`, `from_date` and `to_date`. Check each row's `date_confidence` and `date_source` before filtering on its date: `low` means the date is an estimate (a rounded YouTube label, a KnowledgeiN estimate). |
| from_date | No | Lower date bound, YYYY-MM-DD (from the start of that UTC day) or an ISO 8601 timestamp; when `lookback_days` is sent too, `from_date` applies. A hard bound on `items`: rows dated before it are dropped, rows with no date are kept, and `data.since` reports `{ since, until, dropped, kept_undated }`. Sent with `since`, the later of the two applies and `data._warnings` says so. An invalid date is a free 400. |
| to_date | No | Upper date bound, YYYY-MM-DD (to the end of that UTC day, inclusive) or an ISO 8601 timestamp; defaults to today when from_date is set alone. A hard bound on `items`: rows dated after it are dropped, rows with no date are kept. Sent with `until`, the earlier of the two applies and `data._warnings` says so. An invalid date is a free 400. |
| since | No | Only rows published at or after this time (ISO 8601 timestamp or date), for polling: pass the previous run's time. A hard filter on `items`, unlike lookback_days; rows with no date are kept and `data.since` reports `{ since, until, dropped, kept_undated }`. Billed as usual. |
| until | No | Only rows published at or before this time: YYYY-MM-DD (that whole UTC day is included) or an ISO 8601 timestamp. A hard filter on `items`, like since; rows with no date are kept and counted in `data.since`. Billed as usual. An invalid date is a free 400. |
| period | No | A window that holds, by name: `Nd` (1 to 3650 days, ending now), `yesterday`, `this_week`, `last_week`, `this_month`, `last_month`, `last_quarter`, `this_year` or `ytd`, with calendar periods read in `tz`. Each platform is sent its own date filter, and every row is checked against the window again, so rows outside it are dropped whatever the platform did; `data.request.window` echoes the resolved window and `data.platforms.<p>.window` counts the rows dropped. `since`, `until`, `from_date`, `to_date` and `lookback_days` win over `period` when sent with it, and `data._warnings` says so. Billed as usual. |
| tz | No | IANA time zone name (default UTC, for example Asia/Seoul) that `period` and bare `since`, `until`, `from_date` and `to_date` dates are read in. An unknown zone is a free 400. |
| undated | No | What a row with no date does inside a window: `keep` (default) keeps it and counts it in `data.request.window.undated_kept`, `drop` removes it and counts it in `undated_dropped`. Applies only when a window is set. (keep | drop) |
| limit | No | How many ranked results `items` returns, 1 to 40 (default 40). Above 40 is clamped to 40 with a `data._warnings` line. It trims the answer only: the search, its price and its cache entry are the same whatever the limit, so `limit=10` then `limit=20` on the same query is one billed search. |
| sources | No | Optional CSV allowlist of sources (mutually exclusive with exclude). Valid names: reddit, twitter, twitter-ai-search, youtube, tiktok, instagram, hackernews, polymarket, github, threads, pinterest, perplexity, web, linkedin, rumble, tiktok-hashtag, instagram-hashtag, youtube-hashtag, naver, facebook. `naver` (Naver blog, cafe and KnowledgeiN) is in the default set only when the query contains Hangul. `facebook` (public post search) runs only when you name it here. `x` is another name for `twitter` (X posts) and `tavily` for `web`; `twitter-ai-search` here means `twitter` plus include=synthesis. Platform shorthands expand to their full group: youtube, instagram, tiktok also include their -hashtag lane. Unknown names return a 400. |
| platforms | No | Another name for sources, with the same values. When both are sent and differ, sources applies and `data._warnings` says so. |
| exclude | No | Optional CSV blocklist of sources (mutually exclusive with sources). Same valid names and platform shorthands as sources: excluding youtube/instagram/tiktok also excludes the platform's -hashtag lane. Unknown names return a 400. |
| include_transcripts | No | Fetch spoken-word transcripts for the top 3 video results (default false), so a video whose title never mentions your query but whose narration does can still be found and quoted. Applies to the YouTube, Rumble, TikTok and Instagram lanes; YouTube and Rumble transcripts carry timestamped segments, TikTok and Instagram are plain text. Transcripts land on `data.items[i].source_items[0].metadata.transcript` and stream as `transcript_enriched` chunks. Included in the flat 20 credits. |
| relevance | No | Optional. filter drops the ranked results our relevance judgment already scored as off-topic for your query (unrelated, or clearly about a different thing that shares its words) and lists their ids in data.relevance.dropped_ids, with counts. A result that could not be judged is never dropped; when none could be judged nothing is dropped and data._warnings says relevance_unavailable. items_by_source is not filtered. Omit it to keep every ranked result, ordered as before; each result still carries computed.relevance by default (see judgments). Included in the flat 20 credits. (filter) |
| include | No | Optional CSV of extra blocks and opt-in sources. synthesis adds the X summary (twitter-ai-search) and returns it in data.synthesis.x with its citations. facebook adds Facebook public post search to the default sources (it is not in the default fan-out, and `sources=` would replace the defaults instead). stance is now on by default and accepted for compatibility: data.items[i].computed.stance ({ label: positive, negative, neutral, no_opinion or null when unsure, probabilities, confidence }) says what each post, with any replies shown, says about your query, and data.stance_split counts it per platform and overall with the ids behind every count. Posts judged under the confidence floor count as uncertain, never as a stance. If the judgment is unavailable the search still returns, stance is null and data._warnings says stance_unavailable (or stance_skipped when judging was skipped). Included in the flat 20 credits. |
| judgments | No | Optional. on (default) or off. On, every result also carries data.items[i].computed.stance (see include) and data.items[i].computed.relevance ({ score 0 to 1, on_topic, entity }), with data.relevance counting the off-topic results and listing their ids. Nothing is dropped or reordered and the price does not change. off returns the response without these blocks. (on | off) |
| switch_from | No | Optional brand. data.switch_receipts lists results where the author says they personally left that brand, with the verbatim text, the reason, where they went, and the link. There is no percentage. Rows that are not a switch stay in items. Included in the flat 20 credits. |
Example Response
What does the Universal Search Everywhere API return?
Every response follows one unified schema. Here is a real, unmodified response body, so you can see the exact fields you get back before spending a credit.
Example response
{
"success": true,
"platform": "search",
"endpoint": "/v1/search/everywhere",
"data": {
"request_id": "meta-1790945284064-0bmfbq",
"query": "kanye west",
"plan": {
"intent": "factual",
"freshness_mode": "balanced_recent",
"cluster_mode": "story",
"subqueries": [
{
"label": "primary",
"search_query": "kanye west",
"ranking_query": "What recent evidence from the last 30 days is most relevant to kanye west?",
"sources": [
"reddit",
"youtube"
],
"weight": 0.19801980198019803
},
{
"label": "reddit discourse and updates",
"search_query": "Kanye West controversy album tour merch",
"ranking_query": "What are the most cited current talking points about Kanye West on Reddit, and what context do users give for each claim?",
"sources": [
"reddit"
],
"weight": 0.19801980198019803,
"topic_term": null
}
],
"source_weights": {
"reddit": 0.25,
"youtube": 0.25,
"youtube-hashtag": 0.25,
"github": 0.25
},
"notes": [
"deterministic-plan",
"llm-plan"
]
},
"items": [
{
"candidate_id": "https://reddit.com/r/music/comments/1wv1ox9/kanye_wests_soldout_russia_concerts_canceled",
"item_id": "1wv1ox9",
"source": "reddit",
"title": "Kanye West's sold-out Russia concerts canceled",
"url": "https://www.reddit.com/r/Music/comments/1wv1ox9/kanye_wests_soldout_russia_concerts_canceled/",
"snippet": "",
"subquery_labels": [
"primary"
],
"native_ranks": {
"primary:reddit": 1
},
"local_relevance": 0.2222222222222222,
"freshness": 89,
"engagement": 77,
"source_quality": 0.6,
"rrf_score": 0.0008115565654926149,
"sources": [
"reddit"
],
"source_items": [
{
"item_id": "1wv1ox9",
"source": "reddit",
"title": "Sample comment text (redacted).",
"body": "Sample comment text (redacted).",
"url": "https://www.reddit.com/r/Music/comments/1wv1ox9/kanye_wests_soldout_russia_concerts_canceled/",
"author": "user_d83d82",
"published_at": "2026-10-01T14:59:27.000Z",
"date_confidence": "low",
"engagement": {
"score": 782,
"num_comments": 68
},
"relevance_hint": 0.5,
"why_relevant": "",
"snippet": "Sample comment text (redacted).",
"metadata": {
"top_comments": [
{
"score": 544,
"excerpt": "Sample comment text (redacted).",
"truncated": false,
"author": "user_8164a5",
"url": "https://example.com/redacted",
"date": "2026-10-01T15:42:25.433Z"
},
{
"score": 216,
"excerpt": "Sample comment text (redacted).",
"truncated": false,
"author": "user_f3bf8b",
"url": "https://example.com/redacted",
"date": "2026-10-01T15:52:14.228Z"
}
]
},
"media": {
"thumbnail_url": null,
"video_url": null,
"duration_s": null,
"image_urls": [],
"aspect_ratio": null,
"author_avatar_url": null,
"favicon_url": null
},
"local_relevance": 0.2222222222222222,
"freshness": 89,
"engagement_score": 77,
"source_quality": 0.6,
"local_rank_score": 0.4439444444444444
}
],
"final_score": 66.20088914137315,
"metadata": {
"provenance": [
{
"source": "reddit",
"subquery_label": "primary",
"native_rank": 1,
"item_id": "1wv1ox9"
}
]
},
"rerank_score": 81.83,
"explanation": "Directly discusses the query's entity or question",
"computed": {
"relevance": {
"score": 0.74,
"on_topic": true,
"entity": 0.97
},
"stance": {
"label": "negative",
"probabilities": {
"positive": 0,
"negative": 0.96,
"neutral": 0.03,
"no_opinion": 0.01
},
"confidence": 0.94
}
},
"cluster_id": "cluster-1"
},
{
"candidate_id": "https://youtube.com/watch?v=qxoewuahoiw",
"item_id": "qxOeWuAHOiw",
"source": "youtube",
"title": "Joe Rogan Experience #1554 - Kanye West",
"url": "https://www.youtube.com/watch?v=qxOeWuAHOiw",
"snippet": "",
"subquery_labels": [
"primary"
],
"native_ranks": {
"primary:youtube": 13
},
"local_relevance": 0.2222222222222222,
"freshness": 0,
"engagement": 43,
"source_quality": 0.85,
"rrf_score": 0.0006781500067815001,
"sources": [
"youtube"
],
"source_items": [
{
"item_id": "qxOeWuAHOiw",
"source": "youtube",
"title": "Sample comment text (redacted).",
"body": "Sample comment text (redacted).",
"url": "https://www.youtube.com/watch?v=qxOeWuAHOiw",
"author": "user_e3b0c4",
"published_at": "2021-10-02T12:48:09.738Z",
"date_confidence": "med",
"engagement": {
"views": 21128467
},
"relevance_hint": 0.5,
"why_relevant": "",
"snippet": "Sample comment text (redacted).",
"metadata": {
"top_comments": [
{
"score": 7,
"excerpt": "Sample comment text (redacted).",
"truncated": false,
"author": "user_446c78",
"url": null,
"date": "2026-07-18T06:28:44.000Z"
},
{
"score": 6,
"excerpt": "Sample comment text (redacted).",
"truncated": false,
"author": "user_bd900e",
"url": null,
"date": "2026-07-17T05:47:30.000Z"
}
]
},
"media": {
"thumbnail_url": "https://i.ytimg.com/vi/qxOeWuAHOiw/hqdefault.jpg?sqp=-oaymwEnCOADEI4CSFryq4qpAxkIARUAAIhCGAHYAQHiAQoIGBACGAY4AUAB&rs=AOn4CLBTYB39onRUPWxvKQWCPtVgaBR9LA",
"video_url": null,
"duration_s": null,
"image_urls": [],
"aspect_ratio": null,
"author_avatar_url": "https://example.com/avatars/user_7c6736.png",
"favicon_url": null
},
"local_relevance": 0.2222222222222222,
"freshness": 0,
"engagement_score": 43,
"source_quality": 0.85,
"local_rank_score": 0.18744444444444444
}
],
"final_score": 62.56753750169537,
"metadata": {
"provenance": [
{
"source": "youtube",
"subquery_label": "primary",
"native_rank": 13,
"item_id": "qxOeWuAHOiw"
}
]
},
"rerank_score": 88.58,
"explanation": "Directly discusses the query's entity or question",
"computed": {
"relevance": {
"score": 0.85,
"on_topic": true,
"entity": 0.97
},
"stance": {
"label": "positive",
"probabilities": {
"positive": 0.95,
"negative": 0.03,
"neutral": 0.02,
"no_opinion": 0
},
"confidence": 0.93
}
},
"cluster_id": "cluster-2"
}
],
"items_by_source": {
"youtube-hashtag": [
{
"item_id": "jSrQBGbw4jo",
"source": "youtube-hashtag",
"title": "Sample comment text (redacted).",
"body": "Sample comment text (redacted).",
"url": "https://www.youtube.com/watch?v=jSrQBGbw4jo",
"author": "user_1114db",
"published_at": "2026-05-02T12:48:09.564Z",
"date_confidence": "med",
"engagement": {
"views": 1816022
},
"relevance_hint": 0.5,
"why_relevant": "",
"snippet": "Sample comment text (redacted).",
"metadata": {},
"media": {
"thumbnail_url": "https://i.ytimg.com/vi/jSrQBGbw4jo/hq720.jpg?sqp=-oaymwFBCNAFEJQDSFryq4qpAzMIARUAAIhCGAHYAQHiAQoIGBACGAY4AUAB8AEB-AH-CYAC0AWKAgwIABABGGUgZShlMA8=&rs=AOn4CLDqUfH2OrVPMbTsimnFaqOqdtlCZA",
"video_url": null,
"duration_s": 310,
"image_urls": [],
"aspect_ratio": null,
"author_avatar_url": "https://example.com/avatars/user_8cfd54.png",
"favicon_url": null
},
"local_relevance": 0.19841269841269835,
"freshness": 0,
"engagement_score": 50,
"source_quality": 0.85,
"local_rank_score": 0.17896825396825394
}
],
"youtube": [
{
"item_id": "PsO6ZnUZI0g",
"source": "youtube",
"title": "Sample comment text (redacted).",
"body": "Sample comment text (redacted).",
"url": "https://www.youtube.com/watch?v=PsO6ZnUZI0g",
"author": "user_609f62",
"published_at": "2009-10-02T12:48:09.738Z",
"date_confidence": "med",
"engagement": {
"views": 558565305
},
"relevance_hint": 0.5,
"why_relevant": "",
"snippet": "Sample comment text (redacted).",
"metadata": {
"top_comments": [
{
"score": 50,
"excerpt": "Sample comment text (redacted).",
"truncated": false,
"author": "user_28677c",
"url": null,
"date": "2026-09-19T05:56:11.000Z"
},
{
"score": 11,
"excerpt": "Sample comment text (redacted).",
"truncated": false,
"author": "user_4eaf1d",
"url": null,
"date": "2026-09-24T20:35:45.000Z"
}
]
},
"media": {
"thumbnail_url": "https://i.ytimg.com/vi/PsO6ZnUZI0g/hq720.jpg?sqp=-oaymwEnCNAFEJQDSFryq4qpAxkIARUAAIhCGAHYAQHiAQoIGBACGAY4AUAB&rs=AOn4CLCk_duirSSQANf5RrOjzwWqGNVNRg",
"video_url": null,
"duration_s": 267,
"image_urls": [],
"aspect_ratio": null,
"author_avatar_url": "https://example.com/avatars/user_9b9273.png",
"favicon_url": null
},
"local_relevance": 0.2222222222222222,
"freshness": 0,
"engagement_score": 100,
"source_quality": 0.85,
"local_rank_score": 0.24444444444444444
},
{
"item_id": "6vwNcNOTVzY",
"source": "youtube",
"title": "Sample comment text (redacted).",
"body": "Sample comment text (redacted).",
"url": "https://www.youtube.com/watch?v=6vwNcNOTVzY",
"author": "user_609f62",
"published_at": "2009-10-02T12:48:09.738Z",
"date_confidence": "med",
"engagement": {
"views": 425271144
},
"relevance_hint": 0.5,
"why_relevant": "",
"snippet": "Sample comment text (redacted).",
"metadata": {
"top_comments": [
{
"score": 7,
"excerpt": "Sample comment text (redacted). Sample comment text (redacted). Sample comment text (redacted). Sample comment text (redacted).",
"truncated": false,
"author": "user_876555",
"url": null,
"date": "2026-09-10T12:39:36.000Z"
},
{
"score": 3,
"excerpt": "Sample comment text (redacted).",
"truncated": false,
"author": "user_7ed478",
"url": null,
"date": "2026-09-29T18:48:17.000Z"
}
]
},
"media": {
"thumbnail_url": "https://i.ytimg.com/vi/6vwNcNOTVzY/hq720.jpg?sqp=-oaymwEnCNAFEJQDSFryq4qpAxkIARUAAIhCGAHYAQHiAQoIGBACGAY4AUAB&rs=AOn4CLAJlkmdS4UoEq0Dq4w21KP9oBy3sg",
"video_url": null,
"duration_s": 222,
"image_urls": [],
"aspect_ratio": null,
"author_avatar_url": "https://example.com/avatars/user_9b9273.png",
"favicon_url": null
},
"local_relevance": 0.2222222222222222,
"freshness": 0,
"engagement_score": 95,
"source_quality": 0.85,
"local_rank_score": 0.23944444444444443
}
],
"reddit": [
{
"item_id": "1wv1ox9",
"source": "reddit",
"title": "Sample comment text (redacted).",
"body": "Sample comment text (redacted).",
"url": "https://www.reddit.com/r/Music/comments/1wv1ox9/kanye_wests_soldout_russia_concerts_canceled/",
"author": "user_d83d82",
"published_at": "2026-10-01T14:59:27.000Z",
"date_confidence": "low",
"engagement": {
"score": 782,
"num_comments": 68
},
"relevance_hint": 0.5,
"why_relevant": "",
"snippet": "Sample comment text (redacted).",
"metadata": {
"top_comments": [
{
"score": 544,
"excerpt": "Sample comment text (redacted).",
"truncated": false,
"author": "user_8164a5",
"url": "https://example.com/redacted",
"date": "2026-10-01T15:42:25.433Z"
},
{
"score": 216,
"excerpt": "Sample comment text (redacted).",
"truncated": false,
"author": "user_f3bf8b",
"url": "https://example.com/redacted",
"date": "2026-10-01T15:52:14.228Z"
}
]
},
"media": {
"thumbnail_url": null,
"video_url": null,
"duration_s": null,
"image_urls": [],
"aspect_ratio": null,
"author_avatar_url": null,
"favicon_url": null
},
"local_relevance": 0.2222222222222222,
"freshness": 89,
"engagement_score": 77,
"source_quality": 0.6,
"local_rank_score": 0.4439444444444444
},
{
"item_id": "1wqtof6",
"source": "reddit",
"title": "Sample comment text (redacted).",
"body": "Sample comment text (redacted).",
"url": "https://www.reddit.com/r/hiphopheads/comments/1wqtof6/kanye_west_joins_chris_brown_and_usher_for/",
"author": "user_f971f7",
"published_at": "2026-09-26T15:41:54.000Z",
"date_confidence": "low",
"engagement": {
"score": 1102,
"num_comments": 250
},
"relevance_hint": 0.5,
"why_relevant": "",
"snippet": "Sample comment text (redacted).",
"metadata": {
"top_comments": [
{
"score": 1341,
"excerpt": "Sample comment text (redacted).",
"truncated": false,
"author": "user_75c03e",
"url": "https://example.com/redacted",
"date": "2026-09-26T15:52:58.895Z"
},
{
"score": 1174,
"excerpt": "Sample comment text (redacted).",
"truncated": false,
"author": "user_b81fcb",
"url": "https://example.com/redacted",
"date": "2026-09-26T15:46:45.381Z"
}
]
},
"media": {
"thumbnail_url": null,
"video_url": null,
"duration_s": null,
"image_urls": [],
"aspect_ratio": null,
"author_avatar_url": null,
"favicon_url": null
},
"local_relevance": 0.21555555555555553,
"freshness": 82,
"engagement_score": 93,
"source_quality": 0.6,
"local_rank_score": 0.4381111111111111
}
]
},
"clusters": [
{
"cluster_id": "cluster-1",
"title": "Kanye West's sold-out Russia concerts canceled",
"candidate_ids": [
"https://reddit.com/r/music/comments/1wv1ox9/kanye_wests_soldout_russia_concerts_canceled"
],
"representative_ids": [
"https://reddit.com/r/music/comments/1wv1ox9/kanye_wests_soldout_russia_concerts_canceled"
],
"sources": [
"reddit"
],
"score": 66.20088914137315,
"uncertainty": null
},
{
"cluster_id": "cluster-2",
"title": "Joe Rogan Experience #1554 - Kanye West",
"candidate_ids": [
"https://youtube.com/watch?v=qxoewuahoiw"
],
"representative_ids": [
"https://youtube.com/watch?v=qxoewuahoiw"
],
"sources": [
"youtube"
],
"score": 62.56753750169537,
"uncertainty": null
}
],
"sources_called": [
"youtube-hashtag",
"youtube"
],
"sources_failed": {},
"sources_succeeded": [
"youtube-hashtag",
"youtube"
],
"sources_pruned": {
"github": 0.04
},
"coverage": 1,
"topic_coverage": {
"verdict": "ok",
"best_p": 0.86
},
"partial_failure": false,
"relevance": {
"mode": "score",
"rows": 17,
"judged": 17,
"unjudged": 0,
"off_topic": 0,
"off_topic_ids": [],
"methodology_version": "search-everywhere-relevance/1"
},
"stance_split": {
"methodology_version": "search-everywhere-stance/1",
"confidence_floor": 0.5,
"overall": {
"posts": 17,
"counted": 7,
"positive": 3,
"negative": 4,
"neutral": 0,
"shares": {
"positive": 0.429,
"negative": 0.571,
"neutral": 0
},
"no_opinion": 5,
"uncertain": 5,
"unjudged": 0,
"ids": {
"positive": [
"https://youtube.com/watch?v=qxoewuahoiw",
"https://youtube.com/watch?v=zohhamvk-xm"
],
"negative": [
"https://reddit.com/r/music/comments/1wv1ox9/kanye_wests_soldout_russia_concerts_canceled",
"https://reddit.com/r/nostupidquestions/comments/1tt2tfu/how_did_kanye_west_become_uncancellable"
],
"neutral": [],
"no_opinion": [
"https://youtube.com/watch?v=l63jt6__agw",
"https://youtube.com/watch?v=dr_ytq0syva"
],
"uncertain": [
"https://reddit.com/r/hiphopheads/comments/1wqtof6/kanye_west_joins_chris_brown_and_usher_for",
"https://reddit.com/r/kanye/comments/1wufnn0/50_cent_baby_mama_daphne_joy_confirms_the_rumor"
],
"unjudged": []
}
},
"by_platform": {
"reddit": {
"posts": 7,
"counted": 3,
"positive": 0,
"negative": 3,
"neutral": 0,
"shares": {
"positive": 0,
"negative": 1,
"neutral": 0
},
"no_opinion": 0,
"uncertain": 4,
"unjudged": 0,
"ids": {
"positive": [],
"negative": [
"https://reddit.com/r/music/comments/1wv1ox9/kanye_wests_soldout_russia_concerts_canceled",
"https://reddit.com/r/nostupidquestions/comments/1tt2tfu/how_did_kanye_west_become_uncancellable"
],
"neutral": [],
"no_opinion": [],
"uncertain": [
"https://reddit.com/r/hiphopheads/comments/1wqtof6/kanye_west_joins_chris_brown_and_usher_for",
"https://reddit.com/r/kanye/comments/1wufnn0/50_cent_baby_mama_daphne_joy_confirms_the_rumor"
],
"unjudged": []
}
},
"youtube": {
"posts": 10,
"counted": 4,
"positive": 3,
"negative": 1,
"neutral": 0,
"shares": {
"positive": 0.75,
"negative": 0.25,
"neutral": 0
},
"no_opinion": 5,
"uncertain": 1,
"unjudged": 0,
"ids": {
"positive": [
"https://youtube.com/watch?v=qxoewuahoiw",
"https://youtube.com/watch?v=zohhamvk-xm"
],
"negative": [
"https://youtube.com/watch?v=6vwncnotvzy"
],
"neutral": [],
"no_opinion": [
"https://youtube.com/watch?v=l63jt6__agw",
"https://youtube.com/watch?v=dr_ytq0syva"
],
"uncertain": [
"https://youtube.com/watch?v=pso6znuzi0g"
],
"unjudged": []
}
}
}
},
"dropped": 0
},
"credits_used": 20,
"request_id": "req_example000000",
"cached": false,
"pagination": {
"next_cursor": null,
"has_more": false,
"page_size": 17
},
"credits_remaining": 9999,
"source": "captured",
"captured_at": "2026-10-02T12:48:16.419Z",
"redacted": true
}Live sample illustrating the unified response shape. Field values reflect the record you query.
API Details
How does the Universal Search Everywhere API work?
Send a GET request with your API key and get back clean, structured JSON in our unified schema. Supported computed fields are populated when the source provides the required inputs.
Method
GET
Response
JSON
Developer First
How do you scrape social media data in seconds?
The fastest social media scraping API for developers. Scrape profiles, posts, comments, and analytics from 69 platforms covering 10B+ monthly active users.
One schema, every platform
Query 69 platforms with identical response structures. Write your integration once.
Computed fields, not just scraped
When an endpoint supports these metrics and the source provides the required inputs, the normalized record includes engagement_rate, estimated_reach, content_category, and language. Ready to use.
See your data before you code
Visual Data Explorer. Paste any URL, get rich result cards, sortable tables, CSV export.
import requests
response = requests.get(
'https://www.socialcrawl.dev/v1/tiktok/profile',
params={'handle': 'charlidamelio'},
headers={'x-api-key': 'sc_YOUR_API_KEY'}
)
data = response.json()[ .JSON ]
{
"success": true,
"platform": "tiktok",
"data": {
"author": {
"username": "charlidamelio",
"followers": 152400000
},
"engagement": {
"likes": 12400000000,
"engagement_rate": 0.087
},
"metadata": {
"language": "en",
"content_category": "lifestyle"
}
}
}FAQ
Have a question? We got answers
Find answers to frequently asked questions about SocialCrawl's API, pricing, and capabilities.
Contact usHow do I use the social media search API?
Send GET /v1/search/everywhere?query=your+topic with your API key. SocialCrawl fans the query out across Reddit, X, YouTube, TikTok, Instagram, Hacker News, Polymarket, GitHub, Threads, Pinterest, Perplexity, and Tavily in parallel and returns one ranked, clustered JSON result set.
What parameters does /v1/search/everywhere accept?
query is required (1–512 characters). Optional: lookback_days (default 30), from_date and to_date for an explicit ISO date range, sources for a CSV allowlist of platforms, and exclude for a CSV blocklist. lookback_days and the date range are mutually exclusive, as are sources and exclude.
What does each search result include?
Each ranked item carries the post, its platform, engagement and freshness signals, and real top comments, up to 5 per result, sorted by score, with 300-character excerpts. Comments come from the platforms themselves: Reddit upvoted replies, YouTube and TikTok top-liked comments, GitHub issue discussion.
Can I restrict the search to specific platforms?
Yes. Pass sources as a CSV allowlist (for example sources=reddit,youtube,github) to fan out only to those platforms, or exclude as a blocklist to skip platforms. The price stays a flat 20 credits either way.
How much does one search everywhere call cost?
A flat 20 credits per call, regardless of how many sources fire or how much comment enrichment lands. If the pipeline produces zero ranked items, the credits refund automatically, sync and streaming both.
Does the endpoint support streaming responses?
Yes. With Accept: text/event-stream you get typed SSE chunks, including comments_enriched chunks as each candidate's comments land, before the terminal done event. With Accept: application/json (the default) you get one synchronous envelope.
Ask AI about SocialCrawl
Ready to scrape Universal Search Everywhere data?
Get your API key and start pulling Universal Search data in under 60 seconds.