Most people write API code for the happy path, and then a scheduled job dies at 3am because one private profile returned a 403 and the whole loop threw. Social data APIs fail in specific, predictable ways, and handling those failures well is the difference between a scraper that runs unattended for months and one you're constantly babysitting. It also directly saves money, because the wrong retry strategy burns credits on requests that were never going to succeed.

Here's how to handle errors and retries properly.

Know which errors mean what

The first rule: not all errors should be retried. Retrying an error that will never succeed just wastes credits and time. Group responses into three buckets:

Retry these (transient): timeouts, connection resets, and 5xx server errors. These are temporary; the same request may work in a moment.
Never retry these (client errors): 400 (bad params), 401 (bad key), 402 (out of credits), 404/403 (not found or private). Retrying won't help, fix the input or stop.
Slow down on these: 429 (too many requests). Retry, but only after backing off.

Baking this distinction into your code is what separates a resilient client from one that hammers a doomed request five times.

A retry wrapper that respects the difference

Here's a Python wrapper with exponential backoff that only retries the retryable, and never wastes attempts on a 402 or 404. Base URL https://api.sociavault.com/v1, x-api-key header, payload under data:

import os, time, random, requests

API_KEY = os.environ["SOCIAVAULT_API_KEY"]
BASE = "https://api.sociavault.com/v1"

RETRYABLE_STATUS = {429, 500, 502, 503, 504}
FATAL_STATUS = {400, 401, 402, 403, 404}  # retrying these is pointless

class FatalAPIError(Exception):
    pass

def call(path, params, max_retries=4):
    for attempt in range(max_retries + 1):
        try:
            r = requests.get(f"{BASE}{path}", headers={"x-api-key": API_KEY},
                             params=params, timeout=60)
        except (requests.Timeout, requests.ConnectionError):
            if attempt == max_retries:
                raise
            _backoff(attempt)          # transient network issue -> retry
            continue

        if r.status_code == 200:
            return r.json().get("data")

        if r.status_code in FATAL_STATUS:
            # do NOT retry: fix input or stop. 402 = out of credits.
            raise FatalAPIError(f"{r.status_code} on {path}: {r.text[:200]}")

        if r.status_code in RETRYABLE_STATUS and attempt < max_retries:
            _backoff(attempt, is_rate_limit=(r.status_code == 429))
            continue

        r.raise_for_status()           # anything else: surface it

def _backoff(attempt, is_rate_limit=False):
    # exponential backoff with jitter; longer for rate limits
    base = 2 ** attempt
    if is_rate_limit:
        base *= 2
    time.sleep(base + random.uniform(0, 1))  # jitter avoids thundering herd

Two details that matter: jitter (the random fraction) stops many parallel workers from retrying in lockstep and stampeding, and treating 402 as fatal means the instant you're out of credits you stop cleanly instead of retrying into the void.

Don't let one bad item kill the batch

The second big rule: in a loop over many accounts, one failure shouldn't abort everything. Isolate per-item failures, log them, and keep going:

def collect(handles):
    results, failures = [], []
    for h in handles:
        try:
            data = call("/scrape/tiktok/profile", {"handle": h})
            results.append((h, data))
        except FatalAPIError as e:
            failures.append((h, str(e)))   # private/not-found: skip, note it
        except Exception as e:
            failures.append((h, f"unexpected: {e}"))
        time.sleep(0.5)                     # gentle pacing
    return results, failures

Returning both results and failures means a run over 500 accounts completes with 490 successes and a clean list of the 10 that were private or gone, instead of crashing on account #37. That's the behavior you want from an unattended job.

Handle the "success but empty" case

A subtle one: a 200 response with empty or partial data isn't an error, but it's not always a real result either. A private profile might return a valid response with an account_status flag rather than a 403. Read the payload, not just the status code, and decide what "no data" means for your use case, log it, skip it, or flag it, but don't treat it as a hard failure and retry.

The honest limits

Retries can multiply credit spend. Each retry of a retryable error is another call and another credit. Cap retries, and never retry fatal errors, that's where waste hides.
Backoff adds latency. Exponential backoff means a flaky endpoint slows your whole run. That's the right trade for reliability, but size max_retries sensibly.
Status codes aren't the whole story. A 200 can still carry "empty" or "private" payloads. Inspect the body, not just the code.
Idempotency is fine here, but be careful elsewhere. Read requests are safe to retry; if you ever build write operations on top, retries need more care.
Log failures, don't swallow them. A silent except: pass hides real problems (like a bad key or exhausted credits). Always record what failed and why.

Frequently Asked Questions

Which API errors should I retry?

Only transient ones: timeouts, connection errors, and 5xx server errors, plus 429 (rate limit) after a backoff. Never retry client errors like 400, 401, 402 (out of credits), 403, or 404, retrying those wastes credits and can't succeed.

What's the right retry strategy?

Exponential backoff with jitter: wait longer after each failed attempt, add a small random delay so parallel workers don't retry in lockstep, and cap the number of retries. Back off extra on 429 rate-limit responses.

How do I stop one failure from crashing a whole batch?

Wrap each item's call in its own try/except inside the loop, collect failures into a separate list, and keep going. Return both successes and failures so a run over hundreds of accounts completes instead of aborting on the first private profile.

Why treat a 402 as fatal?

Because 402 means you're out of credits, retrying can't fix that and just spins pointlessly. Treating it as fatal makes your job stop cleanly and surface the problem instead of hammering doomed requests.

Is a 200 response always a success?

Not entirely. A 200 can carry empty or partial data, or flag a private account in the payload rather than returning a 403. Inspect the response body, not just the status code, and decide what "no data" should mean for your use case.

Do retries cost extra credits?

Yes, every retry of a retryable error is another billed call. That's exactly why you cap retries and never retry fatal errors. Sensible retry limits keep reliability high without quietly inflating your credit spend.

Want to build scrapers that run unattended without babysitting? Start free with 50 credits, no card required. Pair this with the credit-saving patterns for a client that's both resilient and cheap.

Error Handling and Retries for Social Data APIs (Do It Right)