Anthropic 529 overloaded_error: What It Means and How to Handle It

Anthropic returns HTTP 429 rate_limit_error when your organization exceeds its own requests-per-minute or input/output tokens-per-minute budget, and HTTP 529 overloaded_error when Anthropic itself is temporarily at capacity. The bodies are what distinguish them: read error.type, not just the status code.

That difference decides who can fix it. A 429 is yours to manage: smooth the traffic, or raise the limit. A 529 is not about you at all, so a limit increase is useless; the only correct responses are patient retries and a fallback path. Treating every 529 as a quota problem sends teams chasing a limit increase that would not have helped.

Use the SDK exception classes rather than string-matching the message. In the Python SDK a 429 raises RateLimitError and any status at or above 500 (including 529) raises InternalServerError; check status_code or the .type property to separate a plain 500 from an overload.

Catch the most specific class first and let genuinely non-retryable errors (400, 401, 404) propagate immediately instead of being swallowed by a broad handler.

The Anthropic SDKs already retry connection errors, 408, 409, 429, and 5xx (529 included) with exponential backoff; max_retries defaults to 2. Raising it to 4 or 5 absorbs most transient overloads with no other change.

When you write the loop yourself, use full jitter: sleep a random duration in the window rather than the full backoff interval. Fixed backoff makes every throttled worker retry at the same instant, which recreates the overload at each step. Bound total attempts as well: an unbounded retry against a saturated service is a denial-of-service you inflict on yourself.

Prompt caching stores a stable prefix (a long system prompt, tool definitions, a retrieved document) so later requests reuse it instead of reprocessing it. Cache reads are billed at roughly a tenth of the base input rate, and writes at about 1.25x, so the break-even is around two requests on the five-minute TTL. Smaller requests also finish faster, which reduces exposure to capacity blips.

Caching is a prefix match, so anything volatile placed early in the prompt silently invalidates it. Keep timestamps, request IDs, and per-user context after the cache breakpoint, and verify with usage.cache_read_input_tokens: a persistent zero means something in the prefix changes on every call.

Backoff buys seconds; a fallback buys availability. Keep an ordered list of deployments (a second Anthropic model, the same model on a different platform, or another vendor entirely) and walk it when the primary returns 529 after its retry budget is spent.

Two rules keep fallback honest. Fail over only on capacity and overload errors, never on a 400 or 401 that will fail identically everywhere. And record which provider actually served each request: a silent fallback changes cost, latency, and output characteristics without anyone noticing until a bill or an eval score moves.

The Message Batches API accepts up to 100,000 requests per batch, returns results within 24 hours (most finish inside an hour), and costs 50% less per token. It runs on its own queue, so it does not compete with your interactive traffic.

Results come back in arbitrary order, so key them by custom_id rather than position. Evaluations, dataset labelling, document enrichment, and nightly summarisation all belong here: moving them off the synchronous path removes a large share of the requests that were meeting 529s in the first place.