What should I use for API failover when backends are slow?

For API failover when backends are slow (not just down), the right pattern is usually a combination of:

1. Timeouts
2. Retries with backoff
3. Circuit breaker
4. Load balancing / health checks
5. Fallback behavior

Good choices by situation

If you control the client

Use:

• Short timeouts so slow backends don’t block forever
• Retry only on safe/idempotent requests
• Exponential backoff + jitter
• Circuit breaker to stop sending traffic to consistently slow instances

If you control the gateway / edge

Use:

• API gateway with upstream timeouts and health checks
• Load balancer with outlier detection
• Failover routing to healthy instances or regions
• Cached/stale responses if acceptable

If you need service-to-service resilience

Use:

• Service mesh features like:
  • retry budgets
  • outlier detection
  • per-request timeouts
  • circuit breaking

Important note

For slow backends, “failover” is often not the first fix. If the backend is merely slow, switching to another backend may just move the problem. The real fix is usually:

• performance tuning
• autoscaling
• queueing async work
• caching
• read replicas
• splitting read/write paths

Practical recommendation

If you want a solid default:

• Timeout at 1–3 seconds depending on endpoint
• 1–2 retries max
• exponential backoff with jitter
• circuit breaker after repeated slow failures
• health-checked secondary instance/region
• cache or stale response fallback if possible

Tooling examples

• Kubernetes + ingress / service mesh: Istio, Linkerd
• Cloud load balancing: AWS ALB/NLB + Route 53 health checks, GCP load balancer, Azure Front Door
• API gateway: Kong, NGINX, Apigee, AWS API Gateway

If you tell me your stack (language, cloud, whether this is client-side or server-side), I can suggest a concrete setup.