Response Schema

Every SocialCrawl response uses the same envelope, on success and on error. Write one parser and it handles every platform.

Successful response

{
  "success": true,
  "platform": "tiktok",
  "endpoint": "/v1/tiktok/profile",
  "data": {
    "author": { "username": "charlidamelio", "followers": 156800000 },
    "computed": {
      "engagement_rate": null,
      "language": null,
      "content_category": null,
      "estimated_reach": null
    }
  },
  "credits_used": 1,
  "credits_remaining": 4999,
  "request_id": "req-XXXXX",
  "cached": false
}

The computed leaves are null here because this abridged author carries no likes_count and no bio, and because estimated_reach is always null on a profile. Every computed field is a real value or an honest null, never a filled-in guess, so type your parser for T | null on all four. Computed fields states the exact rule per field.

Envelope fields

• successbooleanrequired

  true for successful responses, false for errors.

• platformstringrequired

  Platform identifier such as tiktok or instagram. Account endpoints like /v1/credits/balance report meta.

• endpointstringrequired

  The full endpoint path that was called.

• dataobjectrequired

  Platform-specific, normalised payload. Lists are returned as items plus optional next_cursor and total. The computed sub-object is attached to every Author and Post, and judged lists add labels to their rows and a report such as data.labels.

• credits_usedintegerrequired

  Credits consumed by this request. 0 on cache hits and idempotent replays.

• credits_remaininginteger | nullrequired

  Your current committed balance when known. null when a response cannot establish it without an auxiliary balance lookup.

• request_idstringrequired

  Unique identifier in the form req-XXXXX, for debugging and log lookup.

• cachedbooleanrequired

  true when served from cache. Cache hits cost 0 credits.

• age_sinteger | null

  Cache hits only: whole seconds since the copy was fetched, also sent as the X-Cache-Age header. null for a copy stored before ages were recorded. Absent on a live call. Send max_age_s to set how old a copy you accept.

• credits_remaining_notestring

  Cache hits only: why credits_remaining is null there (a free hit does not read the balance) and where to read it.

• paginationobject

  List endpoints only. The uniform pagination contract next_cursor, has_more, page_size, identical on every platform, plus typical_page_size (rows a full page holds) and next_page_credits (what the next page costs). A creator post list called with since or stop_at_id also carries stopped_at.

• metaobject

  Present only when the response carries a hint. meta.hint names one call that does what your recent calls did in fewer steps. Absent on almost every response.

• idempotent_replayboolean

  Present and true only when this response is an idempotency replay. Absent on a normal response.

A few more fields live inside data rather than at the envelope root: data.dropped, data.empty_reason with data.empty_reason_note, and data._warnings[].

data.dropped: Discarded-row counter

On list endpoints, data.dropped counts upstream items omitted because they could not be repaired to the endpoint schema. A healthy list response carries "dropped": 0. It is the only signal that a page came back short, so read it on every page of a drain rather than inferring completeness from items.length.

data.empty_reason: Why a list is empty

Every list response whose data.items is empty says why, so "nothing new since your last poll" never looks like "the source broke". data.empty_reason is one code from the table below, and data.empty_reason_note is one or two sentences saying what the code means and what to do next, written to be shown to a person or read by an agent. Both sit next to items, and a page with rows carries neither.

{
  "success": true,
  "platform": "instagram",
  "endpoint": "/v1/instagram/profile/posts",
  "data": {
    "items": [],
    "dropped": 0,
    "empty_reason": "nothing_new_since",
    "empty_reason_note": "The walk reached your since or stop_at_id boundary before any newer row, so nothing was published after it. Poll again later with the same boundary."
  },
  "credits_used": 1,
  "credits_remaining": 4998,
  "request_id": "req-XXXXX",
  "cached": false,
  "pagination": {
    "next_cursor": null,
    "has_more": false,
    "page_size": 0,
    "stopped_at": "known_id"
  }
}

Branch on empty_reason, never on the note, whose wording can improve. Read it together with pagination.has_more, which still decides whether to keep paging: a filtered page in the middle of a list is empty and has more pages after it. A private account is never an empty list: it answers 404 with error.details.reason set to account_private.

data._warnings[]: Partial-data channel

When the field-map or computed-field pipeline hits ambiguous upstream data, it attaches a human-readable notice to data._warnings: string[]:

{
  "data": {
    "author": { "username": "..." },
    "computed": { "engagement_rate": 1.0 },
    "_warnings": [
      "computed.engagement_rate: value exceeded 1.0 (raw: 1.42); clamped"
    ]
  }
}

Treat entries as advisory. The response is still valid. Empty arrays are omitted, so _warnings is present only when there is something to report.

A query parameter the endpoint does not declare is never forwarded under that name. The house names sort, country, limit, since and until work on every endpoint that takes the same idea under another name (sort_by, order, region, location, page_size, max_results, start_date, from), and so do the common other spellings (order, sort_by, region, geo, market, count, date_from). The value is applied under the endpoint's own name and _warnings says what was applied:

A value moves only when the endpoint's own parameter accepts it, so a request that worked never becomes an error. newest, latest and recent reach the endpoint's own word for newest first, and popular and top its word for most popular. country never lands on a region that only picks the country the request is routed through, because that would not filter the rows.

Any other undeclared parameter is ignored and named in _warnings with what the endpoint does support, for example "days was ignored: did you mean recent_days?", "sentiment was ignored: this endpoint has no sentiment parameter; label=sentiment is free here.", or, on a list whose page size is fixed, "limit was ignored: this endpoint's page size is fixed. To get more rows in one call, send max_items with the number of rows you want (up to 5,000).". An identity alias (username for handle on tiktok/profile, video_id for url on youtube/video) is applied with no line.

profile/posts on every platform

GET /v1/{platform}/profile/posts is each platform's posts feed. Where the feed has another name, that path is served by it: /v1/tiktok/profile/posts answers as /v1/tiktok/profile/videos, /v1/twitter/profile/posts as /v1/twitter/user/tweets, and /v1/threads/profile/posts, /v1/bluesky/profile/posts and /v1/truthsocial/profile/posts as their user/posts. It is the same call at the same price, served from the same cache entry. The response's endpoint and its Content-Location header name the path that answered, and _warnings says "GET /v1/tiktok/profile/posts was served as GET /v1/tiktok/profile/videos, this platform's name for it.". /v1/youtube/profile/posts answers as /v1/youtube/channel/videos too, with the same _warnings line. Use the native path in new code.

Some entries start with a stable token before the colon, so code can match on it. session_priced: appears on tiktokshop/search outside region=US: TikTok prices each shopper session separately there, so the same product can return a different price.current on each call, and price.original appears only when that session was offered a promotion.

Product prices (product.price.current, product.price.original) are always decimal amounts in the major unit of product.price.currency, on every platform and in every region: 10.90 with EUR means 10.90 euros, never 1,090 cents.

Data-level reports

Lists that carry SocialCrawl labels add a report next to items, so you can tell a judged page from an unjudged one without reading every row. All of them are on by default, cost nothing, and disappear with judgments=off.

status is one of three values: complete (every row that can be judged carries its judgment), partial (some rows are null, either still being judged, counted in pending, or not judged this time), or skipped (judging was switched off for this request because capacity was short, and the page is exactly what judgments=off returns plus the report). A pending row is filled on your next call for the same page, including a free cache hit. See Labels for the details, including the search endpoints, which do not fill late rows.

meta.hint: A shorter path for what you just did

When a run of your calls repeats what one other call already does, the response to the call that completes the run can carry a top-level meta.hint naming that call. It appears only on a successful GET:

{
  "success": true,
  "platform": "youtube",
  "endpoint": "/v1/youtube/search",
  "data": {
    "items": [
      /* ... */
    ],
    "dropped": 0
  },
  "credits_used": 1,
  "credits_remaining": 4998,
  "request_id": "req-XXXXX",
  "cached": false,
  "meta": {
    "hint": {
      "code": "search_multi",
      "message": "You searched \"matcha latte\" on 3 platforms one call at a time; /v1/search/multi returns each platform's own search rows in one call at the same price per platform, and a platform with no rows is not charged.",
      "method": "GET",
      "path": "/v1/search/multi?query=matcha%20latte&platforms=tiktok,instagram,youtube"
    }
  }
}

method and path are the call to make, and message says why in one sentence. When the suggested call costs more than the calls it replaces, the message states both prices. A key receives each hint at most once per UTC day, and a response carries at most one. A hint never appears on an error or on an idempotent replay, it costs nothing, and the rest of the response is unchanged. Send the request header X-SocialCrawl-Hints: off to never receive one. If your parser rejects unknown keys, allow an optional meta object at the envelope root.

Error response

{
  "success": false,
  "error": {
    "type": "INSUFFICIENT_CREDITS",
    "message": "Your account has 0 credits remaining. This endpoint requires 1 credits.",
    "status": 402,
    "doc_url": "https://www.socialcrawl.dev/docs/errors#insufficient-credits"
  },
  "credits_used": 0,
  "credits_remaining": 0,
  "request_id": "req-abc123"
}

credits_used is the net committed charge for the failed request. It is 0 when no deduction occurred or the refund committed. If an immediate refund fails, it remains non-zero while exact compensation is queued; keep the request_id for reconciliation.

doc_url is always the anchor form /docs/errors#<code-slug>, never a per-code sub-path. See Errors for the full error-code table and refund rules.

Response headers

List endpoints

List archetypes (PostList, CommentList, SearchResult) are normalised to a consistent shape regardless of upstream key names:

{
  "data": {
    "items": [
      /* ... */
    ],
    "next_cursor": "eyJwYWdlIjoyfQ==",
    "total": 1247
  }
}

The data-level next_cursor and total are legacy per-payload fields, included only when upstream provides them.

Use the top-level pagination block for pagination. Every list response also carries a uniform pagination object at the envelope root. It is identical on all platforms and all three pagination styles (cursor, page, offset):

{
  "data": {
    "items": [
      /* ... */
    ],
    "dropped": 0
  },
  "pagination": {
    "next_cursor": "sc.eyJ2IjoyLCJjIjoiMTc3NTM5NDk1MDAwMCJ9",
    "has_more": true,
    "page_size": 30,
    "typical_page_size": 30,
    "next_page_credits": 1
  }
}

The loop is the same everywhere. Read pagination.next_cursor, send it back as the cursor query parameter, and stop when pagination.has_more is false. You never handle a per-platform input-param name yourself. The API rewrites the sc. cursor to the endpoint's native param, max_cursor or after for example, on the way in. For the full drain recipe, see Pagination.

Two fields help you budget before you send the cursor back. pagination.typical_page_size is how many items a full page of this endpoint usually holds for your request (null when we have not verified it, as on a source whose page changes with your parameters; an endpoint that merges several sources can return more), so a smaller page_size with has_more: true reads as a short page from the source rather than the end. pagination.next_page_credits is what the next page costs, priced from that next request by the same pricer that charges it, and null when there is no next page. A metered endpoint refunds what the page does not use, and a cached page is free. See What the next page costs.

On a list in date order, a call that sends since or stop_at_id gets one more field, pagination.stopped_at. It is known_id or since when the walk reached that boundary, end when the list ran out first, and null while there is more to walk. A call that sends neither, or a list that is not in date order, has no stopped_at key. See Incremental sync and Date window on every list.

Idempotent requests

Send an Idempotency-Key header, a UUIDv4 for example, and a retry with the same key returns the stored answer instead of running again. A replay keeps the first call's body except for the billing fields: credits_used is 0, idempotent_replay is true, credits_remaining is your balance now (or null when it cannot be read, with X-Credits-Remaining left out), and request_id stays the first call's.

curl 'https://www.socialcrawl.dev/v1/tiktok/profile?handle=charlidamelio' \
  -H "x-api-key: YOUR_API_KEY" \
  -H "Idempotency-Key: 7a5e1b4c-2d8f-4a3b-9c1e-6e8b4d2a1f3c"

Not every call takes a key: seven POST calls ignore it and cohort writes require it. A GET that streams with Accept: text/event-stream cannot carry one. Idempotency lists which calls take a key, how long it is kept, and the errors it can produce.

Force a fresh fetch

Responses from cache-enabled endpoints are cached and the cache is shared, so a repeat call returns instantly and costs 0 credits (X-Cache: HIT). To bypass the cache for a single call, send the standard Cache-Control: no-cache request header:

curl 'https://www.socialcrawl.dev/v1/tiktok/profile?handle=charlidamelio' \
  -H "x-api-key: YOUR_API_KEY" \
  -H "Cache-Control: no-cache"

The request fetches live from the source and is billed at the normal endpoint cost, a forced MISS, so X-Cache: MISS and a non-zero X-Credits-Used. The fresh result is written back, so your next plain call gets a free HIT. Only the no-cache directive triggers it. Cache-Control: no-store on its own does not. max_age_s=0 in the query does the same for a client that cannot set headers. Registry-marked uncached endpoints skip this cache contract.

To choose how old a copy you accept instead, send max_age_s in seconds: max_age_s=3600 serves a profile or post copy up to an hour old for 0 credits, and max_age_s=60 fetches fresh when the copy is older than a minute. The hit says its age in age_s. See Choose how old a copy you accept.

For the freshness windows per data type, the shared-cache model, and guidance on when to bypass, see Caching.

Next steps