Fallback pattern in Microservices - System Design Exercise

Design: Microservices with Fallback Pattern

Design the fallback pattern implementation in a microservices architecture including fallback triggers, fallback response types, monitoring, and integration with service calls. Out of scope: detailed UI design, database schema beyond caching fallback data.

Functional Requirements

FR1: The system consists of multiple microservices communicating over the network.

FR2: When a microservice call fails or is slow, the system should provide a fallback response to maintain user experience.

FR3: Fallback responses can be default static data, cached data, or a degraded service response.

FR4: The system should log fallback events for monitoring and alerting.

FR5: Fallbacks should not cause cascading failures or block the system.

FR6: The system should handle up to 10,000 concurrent requests with p99 latency under 300ms.

Non-Functional Requirements

NFR1: Availability target: 99.9% uptime (max 8.77 hours downtime per year).

NFR2: Fallback responses must be returned within 100ms after detecting failure or timeout.

NFR3: Microservices communicate via REST or gRPC.

NFR4: Fallback logic should be configurable per service.

NFR5: System must avoid blocking threads during fallback handling.

Think Before You Design

Questions to Ask

❓ Question 1

❓ Question 2

❓ Question 3

❓ Question 4

❓ Question 5

❓ Question 6

Key Components

API Gateway or Service Mesh for routing and fallback logic

Circuit Breaker component to detect failures

Cache layer for storing fallback data

Logging and Monitoring system

Configuration management for fallback policies

Design Patterns

Circuit Breaker pattern

Bulkhead pattern

Cache-Aside pattern

Timeout and Retry pattern

Fallback pattern

Reference Architecture

Client
  |
  v
API Gateway / Service Mesh
  |
  +--> Microservice A ---+
  |                     |
  |                     +--> Cache (fallback data)
  |                     |
  |                     +--> Logging & Monitoring
  |
  +--> Microservice B ---+
                        |
                        +--> Cache

Fallback logic is implemented in API Gateway or Service Mesh using Circuit Breaker and Timeout detection.

Components

API Gateway / Service Mesh

Envoy, Istio, or custom gateway

Route requests to microservices, implement fallback logic, circuit breaker, and timeout detection

Microservices

Any language/framework (e.g., Spring Boot, Node.js)

Provide business functionality, respond to client requests

Cache

Redis or Memcached

Store fallback data to serve when microservice calls fail

Circuit Breaker

Resilience4j, Hystrix, or built-in service mesh feature

Detect failures and open circuit to trigger fallback

Logging & Monitoring

Prometheus, Grafana, ELK stack

Track fallback events, latency, errors, and alert on anomalies

Request Flow

1. Client sends request to API Gateway.

2. API Gateway routes request to target microservice.

3. If microservice responds successfully within timeout, response is returned to client.

4. If microservice call fails or times out, Circuit Breaker opens and triggers fallback.

5. API Gateway returns fallback response from cache or default static data.

6. Fallback event is logged and monitored.

7. Client receives fallback response quickly, maintaining user experience.

Database Schema

Entities: - FallbackCache: stores key-value pairs of fallback data per service endpoint. Relationships: - Each microservice endpoint can have associated fallback cache entries. - Logging system stores fallback event records with timestamp, service name, and failure reason.

Scaling Discussion

Bottlenecks

Cache becoming a single point of failure or performance bottleneck under high load.

Circuit Breaker state synchronization across distributed instances.

Logging system overwhelmed by high volume of fallback events.

API Gateway becoming a bottleneck due to fallback logic processing.

Solutions

Use distributed cache clusters with replication and sharding to scale cache layer.

Use local circuit breaker instances with eventual consistency or centralized state store with low latency.

Implement log sampling and aggregation to reduce logging load; use scalable log storage solutions.

Scale API Gateway horizontally with stateless design and load balancing.

Interview Tips

Time: Spend 10 minutes understanding requirements and clarifying fallback scenarios, 20 minutes designing architecture and data flow, 10 minutes discussing scaling and trade-offs, 5 minutes summarizing.

Explain why fallback pattern improves system resilience and user experience.

Describe how circuit breaker and timeout detection trigger fallback.

Discuss different fallback response types and when to use each.

Highlight importance of monitoring fallback events for reliability.

Address scaling challenges and solutions for cache, circuit breaker, and gateway.

Show awareness of trade-offs between consistency, latency, and complexity.