Fallback pattern in Microservices - Scalability & System Analysis

Scalability Analysis - Fallback pattern

Growth Table: Fallback Pattern in Microservices

Users / Traffic	System Behavior	Fallback Usage	Impact on User Experience
100 users	Low traffic, services mostly responsive	Fallback rarely triggered	Users get fresh data, fallback not noticeable
10,000 users	Moderate load, occasional service delays	Fallback triggers occasionally during service slowdowns	Users see cached or default data sometimes, minimal disruption
1,000,000 users	High load, frequent service timeouts or failures	Fallback triggers often to maintain availability	Users see degraded but available data, system remains responsive
100,000,000 users	Very high load, multiple cascading failures possible	Fallback triggers extensively; fallback services may also degrade	Users experience limited functionality, possible stale data, but system avoids total failure

First Bottleneck

The first bottleneck is the dependent microservice or external API that the main service calls. As traffic grows, these dependencies can become slow or unavailable due to overload or network issues. Without fallback, the entire request fails. The fallback pattern helps by providing an alternative response or cached data to maintain system responsiveness.

Scaling Solutions with Fallback Pattern

• Implement Circuit Breakers: Detect failing dependencies and trigger fallback quickly to avoid waiting on slow responses.
• Use Caching: Cache previous successful responses to serve during fallback, reducing load on dependencies.
• Horizontal Scaling: Add more instances of fallback services or caches to handle increased fallback requests.
• Graceful Degradation: Design fallback responses to provide minimal but useful data instead of full failure.
• Timeouts and Retries: Set appropriate timeouts and retry policies to avoid cascading delays.

Back-of-Envelope Cost Analysis

• At 1M users with 10 requests per user per minute, system handles ~166,000 requests/sec.
• If 5% of requests trigger fallback, fallback services handle ~8,300 requests/sec.
• Cache storage depends on response size; for 1KB responses cached for 1 hour, storage ~30GB.
• Network bandwidth for fallback responses at 8,300 req/sec * 1KB = ~8.3 MB/s (~66 Mbps).
• Scaling fallback services horizontally to handle peak fallback load is cost-effective compared to full service scaling.

Interview Tip

When discussing fallback pattern scalability, start by explaining the dependency failure risk. Then describe how fallback maintains availability under load. Discuss bottlenecks in dependencies and fallback services. Finally, explain scaling strategies like caching, circuit breakers, and horizontal scaling to handle increased fallback traffic.

Self Check

Your database handles 1000 QPS. Traffic grows 10x. What do you do first?

Answer: Implement fallback mechanisms to serve cached or default data when the database is overloaded, preventing total failure. Also, add read replicas or caching layers to reduce direct database load.

Key Result

Fallback pattern helps maintain system availability by providing alternative responses when dependent services fail or slow down, especially critical as user traffic grows from thousands to millions.