Intelligent Routing: Failover That Actually Happens
Model routers are only as good as their failure handling. Here is how Model Rover picks the best upstream, retries transparently, and keeps your requests flowing.
Routing is a latency problem first
Choosing the cheapest upstream blindly hurts users. We score each candidate on price, observed latency and recent health, then pick the one that best matches the request's tolerance for cost versus speed.
Scores are continuously refreshed from streaming responses, so the router adapts within seconds when an upstream degrades.
Transparent retries
Transient failures are invisible to you. Timeouts, 429s and connection resets trigger an immediate retry on the next-best upstream, with the request ID carried through so you can trace the full journey in our logs.
Graceful degradation
When an entire provider is having an incident, we fail over at the provider level instead of failing the request. Your traffic keeps flowing while the provider recovers, and the incident is reported in your dashboard.