Caching

Every successful response from a cache-enabled endpoint is cached, and the cache is shared. Cache hits cost 0 credits, which makes caching the main lever for controlling both spend and latency.

A minority of endpoints disable caching and run live on every request. Each endpoint's exact cache contract is stated on its own page and in llms.txt.

Caching is on by default

There is nothing to enable or configure. When you call any /v1/* endpoint:

1. If a warm copy exists for the same endpoint and the same parameters, you get it instantly for 0 credits (X-Cache: HIT).
2. If not, the source is fetched live, the endpoint's normal cost is billed (X-Cache: MISS), and the result is stored for the next caller.

You never pay twice for the same data within its freshness window.

Reading the cache signals

Every response tells you whether it came from cache, in both the body and the headers:

age_s is absent on a live call. It is null on the rare hit whose copy was stored before ages were recorded, and X-Cache-Age is then left out. The standard Age header on our responses is set by our CDN, not by the API, and is not the age of the copy.

A cache hit deducts 0 credits. No transaction row is written and your balance is unchanged. The upstream provider was already paid when the original was fetched, so charging again for the same bytes would be double-billing. Idempotent replays are free for the same reason. See Credits.

Freshness windows

How long an entry stays warm depends on the kind of data. After the window passes, the next call is a live MISS that re-warms the entry for the next caller:

Shorter windows on fast-moving data (comments, search) keep results current. Longer windows on slower-moving data (profiles, analytics) maximise free hits. Single-video transcripts are immutable once published, so they stay warm for 30 days. Empty transcripts are never cache-written.

Those are the category defaults, and a handful of endpoints override them where the data justifies it:

Around 35 endpoints do not cache at all. Do not assume a category default when you are budgeting a specific job. The effective window for a given endpoint is stated on that endpoint's page, and in llms.txt for machine consumers.

Choose how old a copy you accept

The windows above are what a plain call accepts. Send max_age_s to set it yourself, in seconds, on any cached GET:

curl 'https://www.socialcrawl.dev/v1/tiktok/profile?handle=charlidamelio&max_age_s=3600' \
  -H "x-api-key: YOUR_API_KEY"

• Accept an older copy, for free. Profile and post copies are kept for an hour, past their 15- and 10-minute windows. With max_age_s=3600, a copy up to an hour old is served as a cache hit for 0 credits, with its age in age_s. This is the one to use when you poll the same profiles or posts and an answer a few minutes older is fine.
• Require a younger copy. max_age_s=60 refuses a copy older than a minute: the call fetches fresh, is billed as a miss, and writes the new copy back for the next caller.
• Always fetch live. max_age_s=0 accepts no cached copy, the same as Cache-Control: no-cache.

Other categories keep their copies for exactly their window, so max_age_s can tighten them but cannot reach further back. A value past what an endpoint keeps is read as that limit, and data._warnings says so, for example max_age_s=7200 was read as 3600: this endpoint keeps a cached copy for at most 3600 seconds. On an endpoint that does not cache, max_age_s does nothing and data._warnings says that too.

max_age_s takes a whole number, 0 or more. Any other value is a free 400. It is never part of the cache key, so max_age_s=3600 and a plain call read the same copy. max_age on web/scrape and web/extract is a different parameter: the scraper's own window, in milliseconds.

The cache is shared

The cache key is derived from the endpoint and its sorted, normalised query parameters, not from your API key and not from your account. Three consequences follow:

• Shared hits. Any identical request already served inside the freshness window comes back to you as a free hit, whether it was your own earlier call, another job on your account, or another customer's traffic. Public platform data is the same bytes for everyone, so the upstream is paid once and the fetch is billed once.
• Freshness is a property of the entry, not of your traffic. "Nobody on my team has called this handle recently" does not mean you are getting a live fetch. When you need a guaranteed-live read, send Cache-Control: no-cache.
• Unknown params cannot bust it. A query parameter the endpoint does not declare is dropped before cache lookup, billing, and upstream dispatch, so it never reaches the cache key. data._warnings names it, with the parameter you probably meant when there is one. Empty undeclared values are treated as omitted and get no line.

Force a fresh fetch

When you need guaranteed-live data, send the standard Cache-Control: no-cache request header:

curl 'https://www.socialcrawl.dev/v1/tiktok/profile?handle=charlidamelio' \
  -H "x-api-key: YOUR_API_KEY" \
  -H "Cache-Control: no-cache"

const res = await fetch(
  "https://www.socialcrawl.dev/v1/tiktok/profile?handle=charlidamelio",
  {
    headers: {
      "x-api-key": process.env.SOCIALCRAWL_API_KEY!,
      "Cache-Control": "no-cache",
    },
  },
);

console.log(res.headers.get("x-cache"), res.headers.get("x-credits-used"));

import os

import requests

res = requests.get(
    "https://www.socialcrawl.dev/v1/tiktok/profile",
    params={"handle": "charlidamelio"},
    headers={
        "x-api-key": os.environ["SOCIALCRAWL_API_KEY"],
        "Cache-Control": "no-cache",
    },
)

print(res.headers["X-Cache"], res.headers["X-Credits-Used"])

The request skips the cache lookup and fetches live from the source. It is billed at the endpoint's normal cost, a forced MISS, so X-Cache: MISS and a non-zero X-Credits-Used. The fresh result is written back to the cache. The pattern is "pay once to refresh, then ride free hits until the window expires".

Details worth knowing:

• The header affects only the single request it rides on. It never disables caching for your key.
• Only the no-cache directive triggers it. Cache-Control: no-store on its own does not. A composite value like no-store, no-cache does, and matching is case-insensitive.
• For a client that cannot set headers (an agent's fetch tool, a no-code HTTP step), max_age_s=0 does the same. Its name says what it does, so it cannot be sent by accident. A cache-buster such as ?fresh=1 or ?_cb= is still dropped and does nothing.
• Idempotency-Key takes precedence. A replayed idempotent request returns the stored response even if it also carries Cache-Control: no-cache. See Idempotent requests.

When should I bypass the cache?

Most workloads should leave caching on and let hits accrue. Reach for Cache-Control: no-cache only when staleness is a correctness problem, for example:

• Reconciliation or audit jobs that must reflect the source at the exact moment of the call.
• Reading data back immediately after a user action or webhook changed it upstream.
• Debugging a field-map or freshness question where you want to compare cached against live.

When "fresh within the last minute" is enough, send max_age_s=60 instead: you pay only when the copy is older than that. For everything else, reusing identical requests within the freshness window is the cheapest and fastest path, and max_age_s=3600 on profiles and posts makes it cheaper still.

Cache-Control and Idempotency-Key are listed as optional headers on every endpoint in the API Reference "Try it" panel, so you can experiment against your own key and watch X-Cache and X-Credits-Used change.

A row join reads the cache row by row

Some list endpoints take an opt-in join token, such as include=profile on the LinkedIn people lanes, which looks each row up separately behind your one request. Those lookups read the same shared cache a direct call would, so a row whose subject is already warm is filled for 0 extra credits and only the rows that had to be fetched are charged. Ask for the same joined page again inside the freshness window and the whole response is an ordinary cache hit at 0 credits. data.hydration reports how many rows came from cache. LinkedIn member and company URLs are normalised before the key is built, so a country subdomain, a trailing slash, and ?trk= tracking parameters all reach the same entry. See Credits for how the held credits settle.

Labels on a cache hit

A cached page still carries its SocialCrawl labels, at 0 credits. Labels are attached on every read, hit or miss, from a separate per-row store: one entry per comment, post or review, kept for 30 days and keyed by the row's text, so an edited comment is judged again. A row that was still pending when the first response was sent is usually labelled by the time the next caller, or your own repeat call, reads the cached page.

The four universal search endpoints are the exception: their judgments are stored with the cached response, so a result that was not judged in time stays null until that cached response expires. judgments=off is part of the cache key, so a request with it never receives a labelled cached page.

Next steps