Fallback pattern in Microservices - Scalability & System Analysis

Scalability Analysis - Fallback pattern

Growth Table: Fallback Pattern in Microservices

Users / Traffic	System Behavior	Fallback Usage	Impact on User Experience
100 users	Low traffic, services mostly responsive	Fallback rarely triggered	Users get fresh data, fallback not noticeable
10,000 users	Moderate load, occasional service delays	Fallback triggers occasionally during service slowdowns	Users see cached or default data sometimes, minimal disruption
1,000,000 users	High load, frequent service timeouts or failures	Fallback triggers often to maintain availability	Users see degraded but available data, system remains responsive
100,000,000 users	Very high load, multiple cascading failures possible	Fallback triggers extensively; fallback services may also degrade	Users experience limited functionality, possible stale data, but system avoids total failure

First Bottleneck

The first bottleneck is the dependent microservice or external API that the main service calls. As traffic grows, these dependencies can become slow or unavailable due to overload or network issues. Without fallback, the entire request fails. The fallback pattern helps by providing an alternative response or cached data to maintain system responsiveness.

Scaling Solutions with Fallback Pattern

• Implement Circuit Breakers: Detect failing dependencies and trigger fallback quickly to avoid waiting on slow responses.
• Use Caching: Cache previous successful responses to serve during fallback, reducing load on dependencies.
• Horizontal Scaling: Add more instances of fallback services or caches to handle increased fallback requests.
• Graceful Degradation: Design fallback responses to provide minimal but useful data instead of full failure.
• Timeouts and Retries: Set appropriate timeouts and retry policies to avoid cascading delays.

Back-of-Envelope Cost Analysis

• At 1M users with 10 requests per user per minute, system handles ~166,000 requests/sec.
• If 5% of requests trigger fallback, fallback services handle ~8,300 requests/sec.
• Cache storage depends on response size; for 1KB responses cached for 1 hour, storage ~30GB.
• Network bandwidth for fallback responses at 8,300 req/sec * 1KB = ~8.3 MB/s (~66 Mbps).
• Scaling fallback services horizontally to handle peak fallback load is cost-effective compared to full service scaling.

Interview Tip

When discussing fallback pattern scalability, start by explaining the dependency failure risk. Then describe how fallback maintains availability under load. Discuss bottlenecks in dependencies and fallback services. Finally, explain scaling strategies like caching, circuit breakers, and horizontal scaling to handle increased fallback traffic.

Self Check

Your database handles 1000 QPS. Traffic grows 10x. What do you do first?

Answer: Implement fallback mechanisms to serve cached or default data when the database is overloaded, preventing total failure. Also, add read replicas or caching layers to reduce direct database load.

Key Result

Fallback pattern helps maintain system availability by providing alternative responses when dependent services fail or slow down, especially critical as user traffic grows from thousands to millions.

Practice

1. What is the main purpose of the fallback pattern in microservices?

easy

A. To provide a backup response when a service call fails

B. To increase the number of service calls

C. To replace the main service permanently

D. To log all service requests for auditing

Solution

1. Step 1: Understand the fallback pattern role

   The fallback pattern is designed to handle failures gracefully by providing an alternative response.

2. Step 2: Identify the main goal

   Its main goal is to keep the system responsive and avoid cascading failures by returning backup data or default messages.

3. Final Answer:

   To provide a backup response when a service call fails -> Option A

4. Quick Check:

   Fallback pattern = backup response [OK]

Hint: Fallback means backup response on failure [OK]

Common Mistakes:

• Thinking fallback increases service calls
• Confusing fallback with permanent service replacement
• Assuming fallback is for logging only

2. Which of the following is a correct way to implement a fallback method in a microservice?

easy

A. Ignore the failure and return an error to the user

B. Call the main service repeatedly until it succeeds

C. Return cached data or a default message when the main service fails

D. Restart the entire microservice on failure

Solution

1. Step 1: Review fallback implementation options

   Fallback should provide a quick alternative response like cached data or default messages.

2. Step 2: Eliminate incorrect options

   Repeated calls can cause delays, ignoring failure hurts user experience, and restarting service is costly and slow.

3. Final Answer:

   Return cached data or a default message when the main service fails -> Option C

4. Quick Check:

   Fallback = cached or default response [OK]

Hint: Fallback returns cached or default data on failure [OK]

Common Mistakes:

• Retrying endlessly instead of fallback
• Returning errors instead of fallback data
• Restarting services unnecessarily

3. Consider this pseudocode for a microservice call with fallback:

response = callMainService()
if response.failed:
    response = fallbackResponse()
print(response)

What will be printed if callMainService() fails?

medium

A. The fallback response

B. The original failed response

C. An error message and no response

D. Nothing, the program crashes

Solution

1. Step 1: Analyze the failure condition

   If callMainService() fails, the code assigns fallbackResponse() to response.

2. Step 2: Determine printed output

   The printed output will be the fallback response, not the failed original response or an error.

3. Final Answer:

   The fallback response -> Option A

4. Quick Check:

   Failed main call triggers fallback output [OK]

Hint: Failed call triggers fallback print [OK]

Common Mistakes:

• Assuming failed response is printed
• Expecting program crash on failure
• Confusing fallback with error message

4. This code snippet tries to implement a fallback but has a bug:

def get_data():
    try:
        return call_service()
    except:
        call_fallback()

What is the bug here?

medium

A. The code does not catch exceptions

B. The try block does not call the service

C. The except block should raise an error

D. The fallback function is not returned

Solution

1. Step 1: Check try-except behavior

   The try block returns the service call result, but except calls fallback without returning it.

2. Step 2: Identify missing return

   Without returning fallback's result, the function returns None on failure instead of fallback data.

3. Final Answer:

   The fallback function is not returned -> Option D

4. Quick Check:

   Missing return in except causes None [OK]

Hint: Always return fallback result in except block [OK]

Common Mistakes:

• Forgetting to return fallback data
• Misunderstanding try-except flow
• Assuming fallback raises error

5. You design a microservice that calls a payment gateway. To avoid delays, you want to use the fallback pattern. Which fallback strategy is best to keep the system responsive and safe?

hard

A. Return a generic error message without fallback

B. Return a cached success response immediately and update later asynchronously

C. Retry the payment gateway call 10 times before fallback

D. Restart the payment microservice on failure

Solution

1. Step 1: Understand fallback goals for payment service

   Fallback should keep system responsive and avoid blocking user with delays.

2. Step 2: Evaluate options for responsiveness and safety

   Returning cached success immediately and updating asynchronously balances responsiveness and eventual consistency.

3. Step 3: Eliminate risky or slow options

   Retries cause delays, generic errors hurt UX, restarting service is costly and slow.

4. Final Answer:

   Return a cached success response immediately and update later asynchronously -> Option B

5. Quick Check:

   Cached immediate fallback with async update = best practice [OK]

Hint: Use cached immediate fallback plus async update [OK]

Common Mistakes:

• Excessive retries causing delays
• No fallback causing poor user experience
• Restarting services on failure