Your AI Gateway fallback worked, but the answer came from a different model

An application stays online when its primary model provider returns a 429 or server error. That is useful resilience, but the replacement model may behave differently on the customer's task. Databricks documents ordered AI Gateway fallback for these response classes. The operational pain is easy to miss because the application still gets a successful response.

Record the route behind each answer

Keep the requested endpoint, actual served model, fallback reason, request ID and customer workflow together. Databricks exposes routing information in system.ai_gateway.usage. Its usage-tracking documentation covers usage, latency and cost records; those records are typically available within about an hour, so they are useful for review but not an instant circuit-breaker.

Define acceptable substitution

Run the same task set through primary and fallback models. Compare required fields, refusal behavior, groundedness, latency and unit economics. If a workflow requires a particular capability or regulatory boundary, do not silently route to a model that lacks it. Decide which requests may degrade gracefully and which should fail visibly.

Rehearse the failure path

Trigger a controlled provider error in staging. Confirm fallback order, session behavior, trace retention and the alert that tells operators a substitution occurred. A green availability chart should be paired with a routing chart and a small quality check on affected tasks.

Adovo’s view

Availability is not the whole service promise. A model switch changes the product experienced by the customer, even if the HTTP status stays green. Resilience earns trust when the substitution is measured and bounded.