A fallback chain says: “if this model fails or rate-limits, try this one next, then this one.” The gateway handles it transparently — your code just sees a successful response.
For the conceptual overview see Fallback chains. This guide walks through configuring one end-to-end.
1. Pick a source model
The source is the model your application calls. Common choices:
gpt-4o — your primary high-quality modeltext-embedding-3-large — your primary embeddings modelgemini-2.5-flash — your latency-sensitive model
You can attach one chain per source slug per workspace.
2. Pick fallback targets
Order them by your preference — typically same family / same tier first, then a different provider, then a cheaper model. Three to four levels is the sweet spot; deeper chains rarely fire and add latency to debug.
Examples:
For embeddings, mixing providers in a fallback breaks vector compatibility — the dimensions and embedding spaces differ. Keep fallbacks within the same model family or downgrade to a smaller model from the same provider.
3. Create the chain
Settings → Fallbacks → Add Override, once for each fallback model you want in
the chain:
- Pick the source model
- Pick a fallback model and give it a priority
- Repeat for each additional fallback model
Each override is created immediately — there’s no separate save step. There’s also no
in-place edit: to change a fallback’s priority, delete the override and add it again.
4. Test it
Force the primary to fail to confirm the chain works:
Look at the response headers:
x-fallback-from is present only when a fallback fired, naming the model you
requested; x-model-used always names whichever model actually answered. There’s no
header reporting how many levels deep the chain went — only whether it moved at all.
Then restore the rate limit.
5. Observe in production
Usage has no fallback-specific view — no per-chain rate, no depth
distribution, no added-latency metric. What it does give you: the By model panel
shows spend shifting toward a fallback target if one starts absorbing traffic, and
clicking a row in Recent requests shows whether a fallback fired and which model
actually served that call. See Usage & invoices.
Budget alerts trigger on total credits spent per period —
there’s no way to key one on fallback activity.
What does not fall back
The gateway only retries on transient/upstream errors:
429 rate limit, 502/503/504 upstream gateway, network timeout- Provider-specific transient codes we map to retryable
It does not retry on:
4xx from your code (400 invalid_request_error, 401, 403, 422)403 insufficient_credits- Validation failures
These are caller errors — fallback wouldn’t help.
Disabling for a test
There is no per-request header to force primary-only — fallback is a workspace-wide
setting, in Settings → Fallbacks:
- Turn Enable Fallback off to stop the gateway from ever substituting a different
model.
- Pick the Retry Only strategy to keep retrying the primary’s own source — useful
when you want to confirm “is the primary actually up right now” rather than “did
anything succeed” — without ever falling through to a different model.
Both are also on the API: PATCH /workspaces/{id}/fallback-config with
{ "enabled": false } or { "strategy": "retry_only" }.
Cost
You pay for the model that served the request, not the attempts. Failed primary attempts are free; the cheaper fallback shows up on your invoice.
Patterns
Cost-tier descent: every step cheaper than the last. Best when quality differences are tolerable for the small fraction of fallback traffic.
Provider diversification: every step a different provider. Best for resilience; quality may drift but availability is preserved.
Region failover (Enterprise): every step a different region. Best for data-residency-aware deployments.
You can mix these — chain gpt-4o → gpt-4o-mini (cheaper, same provider) → claude-sonnet-4.5 (different provider, similar quality) → gemini-2.5-flash (different provider, faster, last resort). 
