Fallback pattern in Microservices - System Design Exercise

Design: Microservices with Fallback Pattern

Design the fallback pattern implementation in a microservices architecture including fallback triggers, fallback response types, monitoring, and integration with service calls. Out of scope: detailed UI design, database schema beyond caching fallback data.

Functional Requirements

FR1: The system consists of multiple microservices communicating over the network.

FR2: When a microservice call fails or is slow, the system should provide a fallback response to maintain user experience.

FR3: Fallback responses can be default static data, cached data, or a degraded service response.

FR4: The system should log fallback events for monitoring and alerting.

FR5: Fallbacks should not cause cascading failures or block the system.

FR6: The system should handle up to 10,000 concurrent requests with p99 latency under 300ms.

Non-Functional Requirements

NFR1: Availability target: 99.9% uptime (max 8.77 hours downtime per year).

NFR2: Fallback responses must be returned within 100ms after detecting failure or timeout.

NFR3: Microservices communicate via REST or gRPC.

NFR4: Fallback logic should be configurable per service.

NFR5: System must avoid blocking threads during fallback handling.

Think Before You Design

Questions to Ask

❓ Question 1

❓ Question 2

❓ Question 3

❓ Question 4

❓ Question 5

❓ Question 6

Key Components

API Gateway or Service Mesh for routing and fallback logic

Circuit Breaker component to detect failures

Cache layer for storing fallback data

Logging and Monitoring system

Configuration management for fallback policies

Design Patterns

Circuit Breaker pattern

Bulkhead pattern

Cache-Aside pattern

Timeout and Retry pattern

Fallback pattern

Reference Architecture

Client
  |
  v
API Gateway / Service Mesh
  |
  +--> Microservice A ---+
  |                     |
  |                     +--> Cache (fallback data)
  |                     |
  |                     +--> Logging & Monitoring
  |
  +--> Microservice B ---+
                        |
                        +--> Cache

Fallback logic is implemented in API Gateway or Service Mesh using Circuit Breaker and Timeout detection.

Components

API Gateway / Service Mesh

Envoy, Istio, or custom gateway

Route requests to microservices, implement fallback logic, circuit breaker, and timeout detection

Microservices

Any language/framework (e.g., Spring Boot, Node.js)

Provide business functionality, respond to client requests

Cache

Redis or Memcached

Store fallback data to serve when microservice calls fail

Circuit Breaker

Resilience4j, Hystrix, or built-in service mesh feature

Detect failures and open circuit to trigger fallback

Logging & Monitoring

Prometheus, Grafana, ELK stack

Track fallback events, latency, errors, and alert on anomalies

Request Flow

1. Client sends request to API Gateway.

2. API Gateway routes request to target microservice.

3. If microservice responds successfully within timeout, response is returned to client.

4. If microservice call fails or times out, Circuit Breaker opens and triggers fallback.

5. API Gateway returns fallback response from cache or default static data.

6. Fallback event is logged and monitored.

7. Client receives fallback response quickly, maintaining user experience.

Database Schema

Entities: - FallbackCache: stores key-value pairs of fallback data per service endpoint. Relationships: - Each microservice endpoint can have associated fallback cache entries. - Logging system stores fallback event records with timestamp, service name, and failure reason.

Scaling Discussion

Bottlenecks

Cache becoming a single point of failure or performance bottleneck under high load.

Circuit Breaker state synchronization across distributed instances.

Logging system overwhelmed by high volume of fallback events.

API Gateway becoming a bottleneck due to fallback logic processing.

Solutions

Use distributed cache clusters with replication and sharding to scale cache layer.

Use local circuit breaker instances with eventual consistency or centralized state store with low latency.

Implement log sampling and aggregation to reduce logging load; use scalable log storage solutions.

Scale API Gateway horizontally with stateless design and load balancing.

Interview Tips

Time: Spend 10 minutes understanding requirements and clarifying fallback scenarios, 20 minutes designing architecture and data flow, 10 minutes discussing scaling and trade-offs, 5 minutes summarizing.

Explain why fallback pattern improves system resilience and user experience.

Describe how circuit breaker and timeout detection trigger fallback.

Discuss different fallback response types and when to use each.

Highlight importance of monitoring fallback events for reliability.

Address scaling challenges and solutions for cache, circuit breaker, and gateway.

Show awareness of trade-offs between consistency, latency, and complexity.

Practice

1. What is the main purpose of the fallback pattern in microservices?

easy

A. To provide a backup response when a service call fails

B. To increase the number of service calls

C. To replace the main service permanently

D. To log all service requests for auditing

Solution

1. Step 1: Understand the fallback pattern role

   The fallback pattern is designed to handle failures gracefully by providing an alternative response.

2. Step 2: Identify the main goal

   Its main goal is to keep the system responsive and avoid cascading failures by returning backup data or default messages.

3. Final Answer:

   To provide a backup response when a service call fails -> Option A

4. Quick Check:

   Fallback pattern = backup response [OK]

Hint: Fallback means backup response on failure [OK]

Common Mistakes:

• Thinking fallback increases service calls
• Confusing fallback with permanent service replacement
• Assuming fallback is for logging only

2. Which of the following is a correct way to implement a fallback method in a microservice?

easy

A. Ignore the failure and return an error to the user

B. Call the main service repeatedly until it succeeds

C. Return cached data or a default message when the main service fails

D. Restart the entire microservice on failure

Solution

1. Step 1: Review fallback implementation options

   Fallback should provide a quick alternative response like cached data or default messages.

2. Step 2: Eliminate incorrect options

   Repeated calls can cause delays, ignoring failure hurts user experience, and restarting service is costly and slow.

3. Final Answer:

   Return cached data or a default message when the main service fails -> Option C

4. Quick Check:

   Fallback = cached or default response [OK]

Hint: Fallback returns cached or default data on failure [OK]

Common Mistakes:

• Retrying endlessly instead of fallback
• Returning errors instead of fallback data
• Restarting services unnecessarily

3. Consider this pseudocode for a microservice call with fallback:

response = callMainService()
if response.failed:
    response = fallbackResponse()
print(response)

What will be printed if callMainService() fails?

medium

A. The fallback response

B. The original failed response

C. An error message and no response

D. Nothing, the program crashes

Solution

1. Step 1: Analyze the failure condition

   If callMainService() fails, the code assigns fallbackResponse() to response.

2. Step 2: Determine printed output

   The printed output will be the fallback response, not the failed original response or an error.

3. Final Answer:

   The fallback response -> Option A

4. Quick Check:

   Failed main call triggers fallback output [OK]

Hint: Failed call triggers fallback print [OK]

Common Mistakes:

• Assuming failed response is printed
• Expecting program crash on failure
• Confusing fallback with error message

4. This code snippet tries to implement a fallback but has a bug:

def get_data():
    try:
        return call_service()
    except:
        call_fallback()

What is the bug here?

medium

A. The code does not catch exceptions

B. The try block does not call the service

C. The except block should raise an error

D. The fallback function is not returned

Solution

1. Step 1: Check try-except behavior

   The try block returns the service call result, but except calls fallback without returning it.

2. Step 2: Identify missing return

   Without returning fallback's result, the function returns None on failure instead of fallback data.

3. Final Answer:

   The fallback function is not returned -> Option D

4. Quick Check:

   Missing return in except causes None [OK]

Hint: Always return fallback result in except block [OK]

Common Mistakes:

• Forgetting to return fallback data
• Misunderstanding try-except flow
• Assuming fallback raises error

5. You design a microservice that calls a payment gateway. To avoid delays, you want to use the fallback pattern. Which fallback strategy is best to keep the system responsive and safe?

hard

A. Return a generic error message without fallback

B. Return a cached success response immediately and update later asynchronously

C. Retry the payment gateway call 10 times before fallback

D. Restart the payment microservice on failure

Solution

1. Step 1: Understand fallback goals for payment service

   Fallback should keep system responsive and avoid blocking user with delays.

2. Step 2: Evaluate options for responsiveness and safety

   Returning cached success immediately and updating asynchronously balances responsiveness and eventual consistency.

3. Step 3: Eliminate risky or slow options

   Retries cause delays, generic errors hurt UX, restarting service is costly and slow.

4. Final Answer:

   Return a cached success response immediately and update later asynchronously -> Option B

5. Quick Check:

   Cached immediate fallback with async update = best practice [OK]

Hint: Use cached immediate fallback plus async update [OK]

Common Mistakes:

• Excessive retries causing delays
• No fallback causing poor user experience
• Restarting services on failure