Why does LiteLLM return a 500 with no useful error message?

Because a failure during upstream preparation or authentication can be normalised into a generic server error before any provider detail reaches your client. Restart the proxy with `--detailed_debug` (or `LITELLM_LOG=DEBUG`) and reproduce the request: the logs carry the provider's verbatim response, which is usually a 401 from a missing or wrong API key.

What causes a LiteLLM BadRequestError about the model?

Either the alias your client sent is not present as a `model_name` in `model_list`, or the matching `litellm_params.model` is missing its provider prefix. LiteLLM needs `openai/gpt-4o` or `anthropic/claude-sonnet-4-5` to know which backend to call; a bare model string produces "LLM Provider NOT provided".

What is the difference between model_name and litellm_params.model?

`model_name` is the public alias clients request; `litellm_params.model` is the actual upstream target including its provider prefix. Using the same `model_name` on several entries is intentional: it creates a load-balanced pool of deployments behind one alias, which is how you spread traffic across regions or vendors.

How do I configure fallbacks in LiteLLM?

Set `fallbacks` under `router_settings` as a list mapping one alias to a list of backup aliases, and use `context_window_fallbacks` for prompts too large for the primary model. Both must reference `model_name` aliases rather than provider model strings: pointing them at a provider string is a silent misconfiguration you only discover mid-incident.

Why does LiteLLM say the budget is exceeded when the provider is fine?

LiteLLM enforces its own per-key, per-team, and per-user budgets and RPM/TPM limits, and rejects the request before it ever reaches a provider. Raise `max_budget` or the key's `rpm_limit`/`tpm_limit`, or set a `budget_duration` so the allowance resets on a cycle: routing to a different provider will not help.

Error reference

LiteLLM Proxy Errors: Diagnosing Auth, Config, and Routing Failures

Updated July 27, 2026

In short

LiteLLM sits between your client and the provider, so its errors are almost always a provider or config problem wearing an OpenAI-shaped exception. Run the proxy with --detailed_debug to see the underlying provider response: an opaque 500 is usually a provider auth failure, a BadRequestError usually means the requested model is missing from model_list or litellm_params.model lacks its provider prefix, and an APIConnectionError means the request never reached the provider at all.

start here

litellm --config config.yaml --detailed_debug

Diagnosis

Causes at a glance

Cause	How to recognize it	Fix
Provider credentials missing or wrong	Opaque 500 from the proxy; --detailed_debug shows the provider returning 401	Set the provider environment variable and confirm with GET /health
Requested model is not in model_list	BadRequestError naming an invalid model, or "LLM Provider NOT provided"	Add the alias under model_name and prefix litellm_params.model with the provider
Network or DNS failure to the provider	APIConnectionError with no provider request id anywhere in the logs	Check api_base, container egress, and proxy environment variables from inside the pod
Request exceeds the configured timeout	Timeout exception at a suspiciously round number of seconds, under load only	Set timeout and num_retries in router_settings; stream long generations
Key, team, or user budget exhausted	Error text mentions budget or max_budget rather than a provider limit	Raise max_budget or rpm_limit/tpm_limit on the key, or route to another deployment

LiteLLM errors are provider errors in a wrapper

LiteLLM normalises every backend into OpenAI-compatible exception types: AuthenticationError, BadRequestError, RateLimitError, Timeout, ContextWindowExceededError, APIConnectionError, ServiceUnavailableError. That uniformity is the point of the proxy, and it is also why the surfaced error can look nothing like the real one.

The mapping is lossy in one specific direction: a failure that happens while the proxy is preparing or authenticating the upstream call can surface to your client as a generic 500 with no provider detail. Debug from the proxy logs, not the client exception.

Step 1

Step 1: turn on detailed debug and read the real error

Start the proxy with --detailed_debug (or set LITELLM_LOG=DEBUG) and reproduce the request. The logs then show the exact outbound request, the resolved provider and model string, and the provider's verbatim response body, which is where the actual cause lives.

Do this before changing anything. Most time lost to LiteLLM issues is spent editing config against a guessed cause when one debug run would have named it precisely.

litellm --config config.yaml --detailed_debug

# or, when running in a container
export LITELLM_LOG=DEBUG

# reproduce through the proxy, not against the provider directly
curl -sS http://localhost:4000/v1/chat/completions -H "Content-Type: application/json" -H "Authorization: Bearer $LITELLM_MASTER_KEY" -d '{"model":"gpt-4o","messages":[{"role":"user","content":"ping"}]}'

Step 2

Step 2: fix model_name versus litellm_params.model

These two fields are the single most common source of LiteLLM BadRequestErrors, and they mean different things. model_name is the public alias your clients send in the request body. litellm_params.model is the upstream target and must carry its provider prefix: openai/gpt-4o, anthropic/claude-sonnet-4-5, azure/<deployment-name>. A missing prefix produces "LLM Provider NOT provided", and an alias your client requests that is absent from model_list produces an invalid-model error.

Repeating the same model_name across several entries is deliberate, not a mistake: identical aliases become a load-balanced pool of deployments for that name, which is how you spread traffic across regions or vendors.

model_list:
  - model_name: gpt-4o                     # what clients ask for
    litellm_params:
      model: openai/gpt-4o                 # provider/model — prefix required
      api_key: os.environ/OPENAI_API_KEY

  - model_name: gpt-4o                     # same alias = second deployment
    litellm_params:
      model: azure/my-gpt4o-deployment
      api_base: os.environ/AZURE_API_BASE
      api_key: os.environ/AZURE_API_KEY

  - model_name: claude
    litellm_params:
      model: anthropic/claude-sonnet-4-5
      api_key: os.environ/ANTHROPIC_API_KEY

Step 3

Step 3: verify credentials and reachability with /health

The /health endpoint sends a minimal request to every deployment in model_list and reports which ones answer. It separates the two failure modes that look identical from the client: a bad or missing key (the provider answers with 401) and an unreachable endpoint (nothing answers at all).

Values written as os.environ/VAR_NAME are resolved by the proxy process, so the variable has to exist in that process: a key present on your laptop but absent from the container is a classic cause of an opaque 500 that only appears in production.

curl -sS http://localhost:4000/health -H "Authorization: Bearer $LITELLM_MASTER_KEY"

# unhealthy entries name the deployment and the provider error:
# {"healthy_endpoints":[...],"unhealthy_endpoints":[{"model":"azure/my-gpt4o-deployment","error":"AuthenticationError ..."}]}

# confirm the variable exists in the proxy process, not just your shell
docker exec -it litellm printenv | grep -E "OPENAI|ANTHROPIC|AZURE"

Step 4

Step 4: configure timeouts, retries, and fallbacks

A proxy without an explicit timeout inherits a very long default, so a stalled upstream ties up a worker for minutes and looks like the proxy itself is hanging. Set timeout and num_retries in router_settings, and use allowed_fails with cooldown_time so a deployment that starts failing is pulled from rotation instead of absorbing every retry.

fallbacks reroutes to a different alias when a deployment fails, and context_window_fallbacks handles the specific case of a prompt too large for the primary model by sending it to a larger-window one. Both reference model_name aliases, not provider model strings: pointing them at openai/gpt-4o instead of the alias is a silent misconfiguration that only shows up during an incident.

litellm_settings:
  drop_params: true            # silently drop params a provider rejects

router_settings:
  routing_strategy: usage-based-routing-v2
  num_retries: 3
  timeout: 30                  # seconds per attempt
  allowed_fails: 3             # failures before a deployment is cooled down
  cooldown_time: 30
  fallbacks:
    - {"gpt-4o": ["claude"]}                  # aliases, not provider strings
  context_window_fallbacks:
    - {"gpt-4o": ["claude"]}

Budget and rate-limit errors from the proxy itself

LiteLLM enforces its own per-key, per-team, and per-user budgets and rate limits, and those rejections are easy to mistake for provider throttling. The tell is the wording: a proxy-side rejection names a budget or an exceeded key limit, while a provider rejection carries the provider's own message and error type.

When a key hits its ceiling the fix is on the proxy: raise max_budget, adjust rpm_limit/tpm_limit, or set a budget_duration so the allowance resets on a cycle. Sending the same traffic to another provider will not help, because nothing ever reached a provider.

curl -sS http://localhost:4000/key/generate -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" -d '{"models":["gpt-4o","claude"],"max_budget":50,"budget_duration":"30d","rpm_limit":60,"tpm_limit":100000}'

# inspect what a key has actually spent
curl -sS "http://localhost:4000/key/info?key=sk-..." -H "Authorization: Bearer $LITELLM_MASTER_KEY"

Beyond the one-off fix

A proxy solves the routing problem and creates a visibility one: every team now shares a handful of virtual keys, and the provider bill arrives as one line. Nexus captures agent actions and token_usage events at the runtime layer rather than the gateway, so spend attributes back to the user, repo, and terminal that actually made the call: regardless of which alias or fallback the proxy chose. The same capture flags providers and models nobody approved, which is exactly what a permissive `model_list` and an unreviewed fallback chain tend to accumulate.

Explore Swfte Nexus

Common questions

Why does LiteLLM return a 500 with no useful error message?: Because a failure during upstream preparation or authentication can be normalised into a generic server error before any provider detail reaches your client. Restart the proxy with `--detailed_debug` (or `LITELLM_LOG=DEBUG`) and reproduce the request: the logs carry the provider's verbatim response, which is usually a 401 from a missing or wrong API key.
What causes a LiteLLM BadRequestError about the model?: Either the alias your client sent is not present as a `model_name` in `model_list`, or the matching `litellm_params.model` is missing its provider prefix. LiteLLM needs `openai/gpt-4o` or `anthropic/claude-sonnet-4-5` to know which backend to call; a bare model string produces "LLM Provider NOT provided".
What is the difference between model_name and litellm_params.model?: `model_name` is the public alias clients request; `litellm_params.model` is the actual upstream target including its provider prefix. Using the same `model_name` on several entries is intentional: it creates a load-balanced pool of deployments behind one alias, which is how you spread traffic across regions or vendors.
How do I configure fallbacks in LiteLLM?: Set `fallbacks` under `router_settings` as a list mapping one alias to a list of backup aliases, and use `context_window_fallbacks` for prompts too large for the primary model. Both must reference `model_name` aliases rather than provider model strings: pointing them at a provider string is a silent misconfiguration you only discover mid-incident.
Why does LiteLLM say the budget is exceeded when the provider is fine?: LiteLLM enforces its own per-key, per-team, and per-user budgets and RPM/TPM limits, and rejects the request before it ever reaches a provider. Raise `max_budget` or the key's `rpm_limit`/`tpm_limit`, or set a `budget_duration` so the allowance resets on a cycle: routing to a different provider will not help.