Error Handling

Every failure returns the same JSON envelope: a stable error.type to branch on, an HTTP status, an error.retryable flag, and a doc_url pointing at that code's entry on this page. Six failures are worth retrying. credits_used always reports the charge committed at response time; automatic compensation covers the rare case where an immediate refund cannot commit.

{
  "success": false,
  "error": {
    "type": "INSUFFICIENT_CREDITS",
    "message": "Your account has 0 credits remaining. This endpoint requires 1 credits. Retrying this request will not succeed until your account balance changes.",
    "status": 402,
    "doc_url": "https://www.socialcrawl.dev/docs/errors#insufficient-credits",
    "retryable": false,
    "details": {
      "reason": "balance_too_low",
      "retry_will_succeed": false,
      "next_step": "Top up your credits on the billing page (or turn on auto-recharge there), then send the request again.",
      "top_up_url": "https://www.socialcrawl.dev/dashboard/billing/payments",
      "credits_required": 1
    }
  },
  "credits_used": 0,
  "credits_remaining": 0,
  "request_id": "req-abc123"
}

Four fields carry the debugging weight:

• error.type is what you branch on. It is stable for a given failure and never changes meaning under /v1.
• error.retryable is true when sending the identical request again can succeed, and false when it will fail the same way until you change something. It matches the Retryable column in the table below, so a client can decide without keeping its own list of codes.
• credits_used is the net committed charge for the failed request. It is 0 when no deduction occurred or the refund committed. A non-zero value on a retryable failure means the immediate refund did not commit and automatic compensation is pending; retain request_id.
• request_id matches the X-Request-Id header, which is present on every response, success or error. Include it when you contact support so we can trace the exact request in our logs.

The doc_url is an anchor on this page, so you can jump straight to a code with #insufficient-credits or #upstream-error.

Error codes

Which errors are safe to retry

The retryable failures are IDEMPOTENCY_IN_PROGRESS, RATE_LIMITED, CONCURRENCY_LIMIT, UPSTREAM_ERROR, SERVICE_UNAVAILABLE, and INTERNAL_ERROR. Every error body says the same thing in error.retryable, so you can branch on that field instead of copying this list. The flag follows the code, so a withdrawn endpoint's SERVICE_UNAVAILABLE still reads true; tell that one apart by its missing Retry-After header, as described under SERVICE_UNAVAILABLE. The one response that overrides the code's flag is described under UPSTREAM_ERROR.

Every other client error is deterministic. IDEMPOTENCY_IN_PROGRESS is the one retryable 409: wait for Retry-After, then send the same key and payload again. See Handling retries for copy-paste backoff loops.

Error reference

Each code below carries a stable anchor, the exact target of the doc_url in the error envelope.

MISSING_API_KEY

401. No API key was present on the request. Send one of your keys from Dashboard → API Keys in the x-api-key header, or as Authorization: Bearer <key>. An Authorization header with any other scheme, such as a bare key or Basic, is not read, and the message says so. No credits are deducted.

INVALID_API_KEY

401. The key is malformed, not found, revoked, or expired. Check the key in Dashboard → API Keys. No credits are deducted. An OAuth access token that is expired, revoked, or issued for another service gets this code too, with error="invalid_token" in its WWW-Authenticate header.

INSUFFICIENT_SCOPE

403. Only an OAuth access token can get this. The token does not carry a scope this endpoint needs, and the scope in its WWW-Authenticate header names the ones it does need. Sign in again asking for them. No credits are deducted. An API key never gets this code.

INSUFFICIENT_CREDITS

402. Your balance is lower than the endpoint's cost. Top up in Dashboard → Billing. No credits are deducted. credits_remaining reports your current balance.

Sending the same request again will not succeed until your balance changes, so keep this code out of your retry loop. The message ends with a sentence saying so, and error.details gives the same answer as fields:

If your team has more than one account, key_prefix and account_email_masked tell you which one ran out. The message names them as well, before the sentence about which pack to buy.

A 402 carries no Retry-After header. Waiting does not help: only a top-up or auto-recharge changes the answer, and either one takes effect on your next request.

Which pack to buy. The 402 names the pack that fits how fast the account uses credits, its price, how long it lasts, and a link that opens checkout for it. The message says the same in one sentence. In the example below the account used about 640 credits a day over the last 14 days, so Growth lasts about 31 days. The token at the end of checkout_url is cut short here.

{
  "success": false,
  "error": {
    "type": "INSUFFICIENT_CREDITS",
    "message": "Your account has 0 credits remaining. This endpoint requires 1 credits. This request was sent with the API key \"Default\" (sc_...hYFw) on the account ni***@example.com. The Growth pack (20,000 credits, £49) lasts about 31 days at your last 14 days' rate. Open checkout: https://www.socialcrawl.dev/dashboard/billing/checkout?token=eyJzdWIi... Retrying this request will not succeed until your account balance changes.",
    "status": 402,
    "doc_url": "https://www.socialcrawl.dev/docs/errors#insufficient-credits",
    "retryable": false,
    "details": {
      "reason": "balance_too_low",
      "retry_will_succeed": false,
      "next_step": "Top up your credits on the billing page (or turn on auto-recharge there), then send the request again.",
      "top_up_url": "https://www.socialcrawl.dev/dashboard/billing/payments?pack=growth",
      "credits_required": 1,
      "key_prefix": "sc_...hYFw",
      "key_name": "Default",
      "account_email_masked": "ni***@example.com",
      "recommended_pack": {
        "id": "growth",
        "name": "Growth",
        "credits": 20000,
        "price_gbp": 49,
        "price_display": "£49",
        "days_at_current_burn": 31,
        "credits_per_day": 640,
        "burn_window_days": 14
      },
      "checkout_url": "https://www.socialcrawl.dev/dashboard/billing/checkout?token=eyJzdWIi...",
      "auto_recharge": {
        "enabled": false,
        "armed": false,
        "threshold_credits": null,
        "pack_id": null,
        "has_card": false,
        "disabled_reason": null,
        "blocked_reason": null,
        "setup_url": "https://www.socialcrawl.dev/dashboard/billing/payments?autorecharge=enable#auto-recharge",
        "top_up_requested": false
      }
    }
  },
  "credits_used": 0,
  "credits_remaining": 0,
  "request_id": "req-abc123"
}

With no recent use, the sentence reads "The Growth pack (20,000 credits, £49) is the usual next step." When auto-recharge is armed, the refusal asks it for a top-up, auto_recharge.top_up_requested is true, and the sentence says the call can be retried once the top-up lands instead of naming a pack. retry_will_succeed stays false, because the identical request still fails until the top-up lands. If the offer cannot be read in time, the 402 leaves out recommended_pack, checkout_url and auto_recharge, and top_up_url is the plain billing page.

What an agent should do with this 402.

1. Read details.recommended_pack to tell the account owner which pack fits, what it costs and how long it lasts.
2. Show the owner details.checkout_url. A person opens it and pays; an agent never needs a card or a login.
3. Send the request again once the top-up lands (credits_remaining on the next response, or GET /v1/credits/balance, shows the new balance). If details.auto_recharge.top_up_requested is true, skip the first two steps and retry once the top-up lands.

When the number is a ceiling. A metered endpoint holds the most the request can cost before it runs and refunds whatever the call does not use. /v1/threads/search, for example, holds 1 credit for every search window its limit allows, plus the query relaxation searches a multi-word query can fan out to. A balance below that hold gets a 402 even though the finished call would usually have cost less, so the 402 says what the number is and how to make the call fit:

• pricing says whether the number is a ceiling. We work this out from the endpoint's recent production calls: an endpoint is a ceiling when a real share of them were charged less than they held.
• credits_typical is what a call like yours is usually charged after the refund: your hold times the median share of the hold those calls were charged, rounded, and never less than the least a successful call to that endpoint costs.
• lower_cost_options lists each parameter that would lower the hold, with the value to send and what the request would then need. credits_required_cheapest is the hold with all of them. Each option is priced by the same calculation the API charges with, so it is never a guess.

A smaller request that fits your balance can be sent straight away, and it is often enough. If you need the full request, top up.

When the API makes the call fit for you. On /v1/threads/search, a balance that covers the call with expand=false but not as sent does not get a 402. The call runs with expand=false, is charged as usual, and data._warnings says so and gives both holds. Query expansion only runs when no post matches your exact phrase, so a page with posts is the same either way. The same applies when the call has to fit your API key's credit limit or max_credits. When a max_credits page has switched to expand=false and your balance still does not cover it, the 402 quotes the price with expand=false, says so, and gives the price as sent in details.balance_fallback. In the example below, 3 credits do not cover the 7 credits that limit=100 still holds with expand=false, so the call gets a 402.

{
  "success": false,
  "error": {
    "type": "INSUFFICIENT_CREDITS",
    "message": "Your account has 3 credits remaining. This endpoint requires 11 credits. The 11 credits required are the most this call can cost: what it does not use is refunded afterwards, and calls like it are usually charged about 2. It holds fewer credits with `limit` at 15 or below (5 credits), `expand=false` (7 credits), or all of these (1 credit). Retrying this request will not succeed until your account balance changes.",
    "status": 402,
    "doc_url": "https://www.socialcrawl.dev/docs/errors#insufficient-credits",
    "retryable": false,
    "details": {
      "reason": "balance_too_low",
      "retry_will_succeed": false,
      "next_step": "Top up your credits on the billing page (or turn on auto-recharge there), then send the request again.",
      "top_up_url": "https://www.socialcrawl.dev/dashboard/billing/payments",
      "credits_required": 11,
      "pricing": "ceiling",
      "credits_typical": 2,
      "lower_cost_with": ["limit", "expand"],
      "lower_cost_options": [
        { "param": "limit", "value": "15", "credits_required": 5 },
        { "param": "expand", "value": "false", "credits_required": 7 }
      ],
      "credits_required_cheapest": 1
    }
  },
  "credits_used": 0,
  "credits_remaining": 3,
  "request_id": "req-abc123"
}

Stopping a runaway client. If one script or agent spends the balance that your other keys depend on, give its key a per-key credit limit. That key then stops at its own cap with KEY_BUDGET_EXCEEDED, and the rest of the account keeps working.

KEY_BUDGET_EXCEEDED

402. The API key you used has its own credit limit, and this request would take it past that limit. Your account balance is untouched and nothing was deducted.

This is not the same problem as INSUFFICIENT_CREDITS, and topping up will not fix it. A per-key limit is a cap you set yourself so that one key (typically a test, CI, or staging key) can never spend more than a fixed amount of the shared account balance. Hitting it means that key did its job.

To resolve it, open Dashboard → API Keys and either raise the key's limit, reset its usage counter back to zero, or remove the limit entirely. You can also switch to a key with no limit set, which is how production keys are usually configured.

credits_remaining still reports your account balance, so a large number there alongside this error is expected. It is the clearest signal that the cap, not the balance, is what stopped the request.

{
  "success": false,
  "error": {
    "type": "KEY_BUDGET_EXCEEDED",
    "message": "This API key has a limit of 500 credits and has used 498. This endpoint requires 5 credits. Your account balance (12480 credits) was not charged. Raise or reset this key's limit in Dashboard → API Keys, or use a key with no limit. Retrying this request will not succeed until this key's credit limit is raised, removed or reset.",
    "status": 402,
    "doc_url": "https://www.socialcrawl.dev/docs/errors#key-budget-exceeded",
    "retryable": false,
    "details": {
      "reason": "key_budget_reached",
      "retry_will_succeed": false,
      "next_step": "Raise or remove this key's credit limit, or reset its usage, on the API keys page, or send the request with a key that has no limit.",
      "key_settings_url": "https://www.socialcrawl.dev/dashboard/api/keys",
      "credits_required": 5
    }
  },
  "credits_used": 0,
  "credits_remaining": 12480,
  "request_id": "req-abc123"
}

error.details has the same keys as INSUFFICIENT_CREDITS, with three differences: reason is key_budget_reached, key_settings_url (the API keys page) replaces top_up_url, and there is no recommended_pack, checkout_url or auto_recharge, because buying credits does not lift a key's own limit.

Do not retry this one on a timer. The counter is cumulative rather than windowed, so it never clears on its own. Only a limit change or a usage reset will let the key spend again. See Authentication.

INVALID_REQUEST

400. A required parameter is missing, a value failed format validation, or no oneOf member was satisfied. Check the endpoint's required parameters in the API Reference. A rejected request costs 0 credits.

Most rejections run before billing, so nothing is deducted. A few can only be detected by the data source after the call has been charged, such as asking for a page past the end of a search. Those are refunded in full before the response is sent, so credits_used is still 0.

Some rejections carry error.details you can branch on. It has two shapes and they never appear together, so branch on which key is present.

METHOD_NOT_ALLOWED

405. The method is not accepted on this path. Registry read paths advertise Allow: GET, HEAD; POST-only routes advertise Allow: POST. The JSON body and X-Request-Id are still canonical. No credits are deducted.

The exceptions are the POST calls (the batch, job, Discover, cohort and webhook endpoint calls, each listed on Idempotency) and the /v1/web/* job surface, which takes POST on job, monitor, and session creation and PATCH or DELETE on monitors, sessions, and jobs.

ENDPOINT_NOT_FOUND

404. The platform or resource is not supported. Check the API Reference for the correct path. No credits are deducted.

Listening has no estimate route: POST /v1/listening/estimate and POST /v1/listening/rules/validate answer 404 with did_you_mean naming POST /v1/listening/streams?dry_run=1, which returns the same estimate, the rule as a tree and in words, and the plan. Send the same body there.

RESOURCE_NOT_FOUND

404. The upstream returned 404, or the resource exists but contains no usable data, for example a nonexistent handle. Verify the handle or URL. The deduction is refunded in full, so credits_used is 0.

Three endpoint families narrow the cause with error.details.reason, so you can tell a permanent gap from a fixable mistake.

PAYLOAD_TOO_LARGE

413. The request's JSON body exceeded the 1 MB size limit and was rejected before parsing, so no credits were deducted. Reduce the batch size, for example by splitting a large ids or urls array across several requests.

IDEMPOTENCY_KEY_CONFLICT

409. The Idempotency-Key conflicts with an incompatible active reservation. Generate a fresh key, ideally a UUIDv4. Keys are scoped per account; this error does not reveal or depend on another account's keys. No credits are deducted.

IDEMPOTENCY_IN_PROGRESS

409. Another request with this key and the same payload is still running. This retry was not dispatched or charged. Wait for the Retry-After seconds, then retry with the same key. The wait is 1 second while the original is young and grows to a quarter of the time it has been running, up to 30, so a long call is not polled every second. Once the original finishes, the retry replays its stored result at 0 credits.

IDEMPOTENCY_REPLAY_UNAVAILABLE

409. The original request completed and its billing outcome is known, but its response exceeded the idempotent replay storage limit: a response over 64 KB is stored compressed and replays normally, unless it is still over 160 KB compressed. This retry was not dispatched or charged. The same key cannot reconstruct the original body. Send the request with a new key to get the data again: inside the endpoint's cache window (freshness.cache_ttl_s on /v1/utility/endpoint) it is served from the cache at 0 credits, and after the window it is a new billable request.

IDEMPOTENCY_KEY_PAYLOAD_MISMATCH

422. You reused an Idempotency-Key with a different payload, meaning different query parameters on a GET or a different JSON body on a POST. Use a new key for the new payload. No credits are deducted.

COHORT_LIMIT_EXCEEDED

400. The account already holds the maximum of 100 cohorts under this API key. Delete an unused cohort (DELETE /v1/cohorts/{cohortId}) before creating another.

COHORT_MEMBER_LIMIT_EXCEEDED

400. The member upload would exceed the cohort limit. Reduce the upload or remove members before retrying.

COHORT_IDENTITY_PLATFORM_UNSUPPORTED

400. The supplied identity platform is not supported for cohort queries. Use a supported platform.

COHORT_IDENTITY_CONFLICT

409. The normalized platform identity is already assigned to another external ID in this cohort. Use the existing external ID or a different identity.

COHORT_QUERY_NOT_CANCELLABLE

409. The query is already terminal and cannot be cancelled. Read its status or results instead.

COHORT_QUERY_NOT_READY

409. The query has not completed, so results are not available yet. Wait for a terminal status before reading results.

COHORT_RESULT_TOO_LARGE

413. One decrypted result exceeds the 1,000,000-byte response-page ceiling. No partial result is returned.

RATE_LIMITED

429. More than 600 requests in a 1-minute sliding window on the same API key. Slow down and retry after Retry-After. The response carries X-RateLimit-Limit, X-RateLimit-Remaining, and X-RateLimit-Reset. No credits are deducted. See Rate Limits.

CONCURRENCY_LIMIT

429. More than 50 concurrent requests were in flight on the same API key. Reduce concurrency and retry. No credits are deducted. The response carries X-Concurrency-Limit and X-Concurrency-Remaining, not X-RateLimit-*. See Rate Limits.

UPSTREAM_ERROR

502. The upstream platform returned an error, after our own automatic retries. Retry after a short backoff. Credits are refunded.

When error.details.retry_with is present, the same request will fail the same way: the response reads retryable: false, carries no Retry-After, and retry_with names the parameter to change (for example {"proxy": "auto"} on /v1/web/scrape). The same applies to a 504 that carries retry_with.

These error.details.reason values can accompany an UPSTREAM_ERROR (or, for url_unreachable, a RESOURCE_NOT_FOUND). All of them are refunded:

504. A GET call has 48 seconds to finish its upstream work; a few long-running endpoints, such as transcripts and Discover, have longer limits. When the 48 seconds run out, the call stops and answers 504 UPSTREAM_ERROR without Retry-After, and its credits are refunded; retry with backoff. If a call is still running at 54 seconds, it is answered with a 504 UPSTREAM_ERROR and Retry-After, its credits are refunded, and the work is left to finish. Retrying after Retry-After may then return it from the cache at no charge.

Every GET call is answered within 60 seconds, so a client timeout of 60 seconds never gives up before we answer. If your client gives up earlier, the call still finishes on our side and is charged, and the result is cached for the endpoint's cache window (freshness.cache_ttl_s on /v1/utility/endpoint). Sending the identical request again inside that window joins the call that is still running, or reads its result from the cache, at 0 credits. A request that sends Cache-Control: no-cache skips the cache and is charged again. latency_ms.recommended_timeout_s on the same endpoint is 1.5 times the measured p99 of successful uncached calls over the 7 days named in latency_ms.sampled, and it is null when fewer than 30 calls were measured.

To tell an upstream outage from a transient blip, check error.incident on the response, or GET /v1/status/{platform}. See Status on 5xx responses.

SERVICE_UNAVAILABLE

503. Several causes. An open circuit breaker, an upstream provider throttle, or an unavailable data source carries Retry-After with the number of seconds after which a retry can succeed. The value depends on the cause, so always read the header rather than assume a fixed wait. If the idempotency claim store is temporarily unavailable, a keyed request fails closed before billing with Retry-After: 1; retry with the same key. No balance lookup is attempted on that path, so credits_remaining is null and X-Credits-Remaining is omitted.

A 503 without a Retry-After header is not transient. It means the endpoint has been withdrawn because its upstream no longer serves it, and the message names the reason. Retrying will never succeed, so branch on the presence of Retry-After rather than on the status alone. Nothing is deducted in the first place on a withdrawn endpoint, since the router rejects it before billing, so there is no refund to wait for. Check the changelog for the replacement.

A Listening request answers 503 with Retry-After and details.reason set to judge_unavailable when the check of a listener's rules and context for a refused use could not answer. Nothing is written or charged; retry the same request after the delay. A Listening preview that fails after its pages were billed refunds every billed page (reason preview_not_served) and answers 503 with credits_used: 0, X-Credits-Used: 0 and Retry-After: 5: suppression_unavailable when the privacy check could not run (if it cannot run before the first page, nothing is fetched at all), and store_unavailable for any other failure, whose message says how many billed pages were refunded. A retry under the same Idempotency-Key runs the preview again and bills its pages again; the failed attempt's pages were refunded. If the refund itself fails, the 503 keeps its reason but credits_used and X-Credits-Used carry the pages' credits, and the message asks you to write to socialcrawl@ridio.io with the request_id so they are refunded by hand.

INTERNAL_ERROR

500. An unexpected error occurred on our side. Retry, and if it persists, contact support with the request_id. The server attempts the refund before responding. If it cannot commit, credits_used remains non-zero and the exact outstanding remainder is queued for automatic compensation rather than falsely reported as refunded.

RUN_NOT_FOUND

404. No /v1/prism/discover run with this run_id belongs to your account. A run started by any active key of your account is readable with any other; another account's run answers this code too, so a run id never confirms that someone else's run exists. details.reason is unknown_run. Start a run, or list your runs with GET /v1/prism/discover/runs. No credits are deducted.

RUN_STATE_CONFLICT

409. The run's state does not allow this call. details.reason names the state: run_completed (a completed run cannot be resumed, and a completed or partial run cannot be cancelled), run_failed or run_cancelled (it cannot be cancelled again), or run_not_completed (POST .../more needs a completed run; resume continues the others). details.next_step names the call that is valid in this state. No credits are deducted.

RUN_EXPIRED

410. The run's retention_days passed, so its rows were deleted. details.reason is retention_passed. Every run route answers this code for an expired run. A queued, running or paused run never expires while it is live. Start a new run. No credits are deducted.

NEEDS_ASYNC

422. A GET /v1/prism/discover request sent with async=false is over what one call can answer: more than 36 searches (keywords, platforms, lane kinds and pages, past-window pages included), more than 50 joins, comments read inline on more than 3 posts per group, or a limit over the call's cap. details.reason is one of legs, joins, comment_posts and limit, details.over lists every one, details.legs and details.joins give the counts against details.legs_max and details.joins_max, and details.causes names the request fields behind them, with details.estimate and a details.suggested_url without async=false, which runs the same request in the background and answers 202 with the run. No credits are deducted.

CURSOR_EXPIRED

410. A GET /v1/events cursor or a named consumer's position is older than the event log's 30-day retention, so events after it were deleted. details.reason is retention_passed. details.oldest_cursor is the position before the oldest event still kept, and details.missed_from and details.missed_until bound the window you missed (missed_from is null when the cursor carries no time). Restart from cursor=oldest (or details.oldest_cursor) and tell the user which window was missed. For a consumer the 410 commits nothing: its next read restarts at the oldest retained event and returns restarted_from once. See Events feed. No credits are deducted.

SCHEDULE_NOT_FOUND

404. No /v1/prism/discover schedule with this schedule_id belongs to your account. A schedule created with any active key of your account is readable with any other; another account's schedule, and one you deleted, answer this code too. details.reason is schedule_not_found. List your schedules with GET /v1/prism/discover/schedules. No credits are deducted.

STREAM_NOT_FOUND

404. No /v1/listening/streams listener with this id belongs to the API key that sent the request. A listener is read only by the key that created it, so another key of your account, another account, and an id that does not exist all answer this code. details.reason is unknown_stream. List this key's listeners with GET /v1/listening/streams. No credits are deducted.

STREAM_STATE_CONFLICT

409. The listener's state does not allow this call. details.reason is stream_deleted when the listener is deleted (a PATCH, pause or resume of it): create a new listener instead. It is resume_from_dashboard when the listener is stopped because its key was revoked or expired: an API key cannot resume it, and the account owner rebinds it to another active key from the dashboard. No credits are deducted.

Idempotency

Send an Idempotency-Key header and a retry of the same request with the same key replays the first answer for 0 credits (X-Idempotent-Replay: true, Idempotent-Replayed: true) instead of running again. Every GET takes a key, and so do the batch, job-submit, webhook endpoint and cohort POST calls; cohort writes require one. Seven POST calls ignore it, among them POST /v1/web/sessions and POST /v1/web/agent, so never retry those blind after a timeout. Idempotency lists every call in each group, how long a key is kept, and when a call is not replayed.

The errors a key can produce are all free:

• IDEMPOTENCY_IN_PROGRESS (409): the first call is still running. A GET retry waits up to 20 seconds before this answer; a POST gets it at once. Retry after Retry-After (1 to 30 seconds) with the same key.
• IDEMPOTENCY_KEY_PAYLOAD_MISMATCH (422): the same key with a different query or body. Use a new key.
• IDEMPOTENCY_REPLAY_UNAVAILABLE (409): the first answer was over 160 KB compressed and was not kept. Send it with a new key.
• SERVICE_UNAVAILABLE (503) with Retry-After: 1: the key store was unavailable, and nothing ran. Retry with the same key.

Refund rules

Credits are refunded automatically when:

• Upstream returns a 5xx error (UPSTREAM_ERROR, 502 or 504).
• The circuit breaker rejects the request, or the upstream provider rate-limits us (SERVICE_UNAVAILABLE, 503).
• An unexpected server error occurs (INTERNAL_ERROR, 500).
• Upstream returns 200 with an empty body, for example a nonexistent handle, triggering RESOURCE_NOT_FOUND (404).

Credits are never deducted for these outcomes, so no refund is needed:

• Cache hits (X-Cache: HIT, X-Credits-Used: 0).
• METHOD_NOT_ALLOWED (405), IDEMPOTENCY_KEY_CONFLICT (409), IDEMPOTENCY_IN_PROGRESS (409), IDEMPOTENCY_REPLAY_UNAVAILABLE (409), and IDEMPOTENCY_KEY_PAYLOAD_MISMATCH (422).
• KEY_BUDGET_EXCEEDED (402). The request is refused before the charge lands, and the key's own usage counter is left where it was.
• Idempotent replays. The original charge already appears on your earlier request.

Client errors (MISSING_API_KEY, INVALID_API_KEY, INVALID_REQUEST, ENDPOINT_NOT_FOUND) are rejected before any deduction. The one exception is an INVALID_REQUEST that only the data source can detect, such as a page past the end of a search: that call is charged and then refunded in full before the response is sent. See Credits for the full refund matrix.

Handling retries

Retry only the errors marked retryable above: IDEMPOTENCY_IN_PROGRESS (409), RATE_LIMITED or CONCURRENCY_LIMIT (429), and transient 500, 502, 503, or 504 responses. Honor Retry-After: idempotency contention sends 1 to 30 (it grows with the original request's running time), claim-store outages send 1, other transient 503s send a wait that depends on the cause, rate-limit 429s send the seconds until reset, and concurrency 429s send a short hint. Always read the header rather than assume a fixed wait.

Otherwise back off exponentially with full jitter, so a pool of your own workers does not resynchronize and stampede the cap together. Cap the attempts and surface the error rather than retrying forever. Never retry a 4xx client error other than 429: the request is deterministic and will fail the same way.

Retryable failures attempt an immediate refund; if that database mutation fails, exact compensation is queued and the response keeps a non-zero committed credits_used. Add an Idempotency-Key header (see Idempotency) so a network-level retry of a successful sync call replays rather than re-runs.

Each loop below retries only the transient statuses (429, 500, 502, 503, 504), honors Retry-After when present, falls back to exponential backoff with jitter, and gives up after five attempts.

#!/usr/bin/env bash
url="https://www.socialcrawl.dev/v1/tiktok/profile?handle=charlidamelio"
max_retries=5
attempt=0

while :; do
  body=$(mktemp); headers=$(mktemp)
  status=$(curl -s -o "$body" -D "$headers" -w '%{http_code}' \
    -H "x-api-key: $SOCIALCRAWL_API_KEY" "$url")

  if [ "$status" -lt 400 ]; then
    cat "$body"; rm -f "$body" "$headers"; break
  fi

  case "$status" in
    429|500|502|503|504) ;;  # retryable, fall through
    *) echo "Non-retryable $status"; cat "$body"; rm -f "$body" "$headers"; exit 1 ;;
  esac

  attempt=$((attempt + 1))
  if [ "$attempt" -gt "$max_retries" ]; then
    echo "Gave up after $max_retries retries (last status $status)"
    rm -f "$body" "$headers"; exit 1
  fi

  # Retry-After (seconds) wins; else exponential backoff 2^attempt with jitter.
  retry_after=$(grep -i '^retry-after:' "$headers" | tr -d '\r' | awk '{print $2}')
  if [ -n "$retry_after" ]; then
    delay="$retry_after"
  else
    delay=$(awk "BEGIN{srand(); print (2 ^ $attempt) * (0.5 + rand() / 2)}")
  fi
  rm -f "$body" "$headers"
  sleep "$delay"
done

import os
import random
import time

import requests  # pip install requests

BASE_URL = "https://www.socialcrawl.dev/v1"
RETRYABLE = {429, 500, 502, 503, 504}


def get_with_retry(path, params=None, max_retries=5):
    headers = {"x-api-key": os.environ["SOCIALCRAWL_API_KEY"]}
    for attempt in range(max_retries + 1):
        resp = requests.get(f"{BASE_URL}{path}", params=params, headers=headers)
        # Success, or a client error we must not retry: hand it back / raise.
        if resp.ok or resp.status_code not in RETRYABLE or attempt == max_retries:
            resp.raise_for_status()
            return resp.json()
        # Retry-After (seconds) wins; else exponential backoff with full jitter.
        retry_after = resp.headers.get("Retry-After")
        delay = float(retry_after) if retry_after else (2 ** attempt) * (0.5 + random.random() / 2)
        time.sleep(delay)


data = get_with_retry("/tiktok/profile", {"handle": "charlidamelio"})
print(data)

// Node 18+ (built-in fetch). No dependencies.
const BASE_URL = "https://www.socialcrawl.dev/v1";
const RETRYABLE = new Set([429, 500, 502, 503, 504]);
const sleep = (ms) => new Promise((r) => setTimeout(r, ms));

async function getWithRetry(path, { params = {}, maxRetries = 5 } = {}) {
  const url = new URL(BASE_URL + path);
  for (const [k, v] of Object.entries(params)) url.searchParams.set(k, v);
  const headers = { "x-api-key": process.env.SOCIALCRAWL_API_KEY };

  for (let attempt = 0; ; attempt++) {
    const res = await fetch(url, { headers });
    if (res.ok || !RETRYABLE.has(res.status)) {
      if (!res.ok)
        throw new Error(`SocialCrawl ${res.status}: ${await res.text()}`);
      return res.json();
    }
    if (attempt >= maxRetries) {
      throw new Error(
        `Gave up after ${maxRetries} retries (last status ${res.status})`,
      );
    }
    // Retry-After (seconds) wins; else exponential backoff with full jitter.
    const retryAfter = Number(res.headers.get("Retry-After"));
    const delayMs =
      Number.isFinite(retryAfter) && retryAfter > 0
        ? retryAfter * 1000
        : 2 ** attempt * 1000 * (0.5 + Math.random() / 2);
    await sleep(delayMs);
  }
}

const data = await getWithRetry("/tiktok/profile", {
  params: { handle: "charlidamelio" },
});
console.log(data);

Status on 5xx responses

Every 500, 502, 503 and 504 body carries two additive fields inside error:

• error.status_url: the public status page, filtered to the platform when one is known, for example https://www.socialcrawl.dev/status?platform=youtube.
• error.incident: present only when an open public incident explains the failure. It has id, title, status, impact and url.

{
  "status_url": "https://www.socialcrawl.dev/status?platform=youtube",
  "incident": {
    "id": "k3v9w2m8q1zt",
    "title": "Elevated request failures on YouTube",
    "status": "investigating",
    "impact": "major",
    "url": "https://www.socialcrawl.dev/status/incidents/k3v9w2m8q1zt"
  }
}

When error.incident is present, retrying sooner will not help: wait for the incident to clear, then resume. Without it, treat the failure as transient and honour Retry-After. Status has the full recipe in cURL, TypeScript and Python, and explains how the status page decides each state.

Debugging tips

• Every error envelope carries a doc_url pointing at the matching section on this page, for example https://www.socialcrawl.dev/docs/errors#insufficient-credits.
• request_id matches the X-Request-Id header and the request_id column on Dashboard → Activity Logs.
• Persistent 502s on a known-good input usually indicate an upstream outage. Check error.incident on the response, the status page, or GET /v1/status/{platform}.
• A high retry count on a slow-but-successful call is surfaced in the X-Upstream-Retries response header.

Next steps