Circuit breaker pattern in HLD - System Design Guide

Problem Statement

When a downstream service or resource becomes slow or unresponsive, continuing to send requests causes long delays and resource exhaustion. This leads to cascading failures where the entire system slows down or crashes because it waits indefinitely for failing components.

Solution

The circuit breaker pattern monitors the success and failure of requests to a service. When failures exceed a threshold, it stops sending requests to that service temporarily, returning errors immediately. After a cooldown period, it tests the service again to see if it has recovered, preventing system overload and improving overall resilience.

Architecture

Client

→Circuit Breaker

↓

Failure Count

Failure Count

This diagram shows the client sending requests through the circuit breaker to the downstream service. The circuit breaker tracks failures and decides whether to allow or block requests.

Trade-offs

✓ Pros

→

Prevents system overload by stopping requests to failing services.

→

Improves system responsiveness by failing fast instead of waiting for timeouts.

→

Allows automatic recovery by periodically testing the service after failures.

✗ Cons

→

Adds complexity to client or service communication logic.

→

Requires tuning of thresholds and timeout durations for optimal performance.

→

May cause temporary denial of service if the breaker trips too aggressively.

Use when your system depends on unreliable or slow external services and you want to avoid cascading failures, especially at scales above hundreds of requests per second.

Avoid when your system has very low traffic (under 100 requests per second) or when downstream services are highly reliable and fast, as the added complexity may not justify the benefits.

Real World Examples

Netflix

Netflix uses circuit breakers to isolate failures in its microservices, preventing one failing service from causing widespread outages.

Amazon

Amazon applies circuit breakers to manage calls to external payment gateways, avoiding delays when those services are down.

Uber

Uber uses circuit breakers to handle unreliable third-party APIs, ensuring the main app remains responsive even if external services fail.

Alternatives

Retry pattern

Retries failed requests a fixed number of times before giving up, without stopping requests entirely.

Use when: Use when failures are transient and likely to succeed on retry, but not when failures are prolonged or cause cascading issues.

Bulkhead pattern

Isolates resources into separate pools to contain failures, rather than stopping requests entirely.

Use when: Use when you want to limit failure impact by resource isolation rather than blocking requests.

Summary

Circuit breaker pattern prevents system overload by stopping requests to failing services.

It improves system resilience by failing fast and allowing recovery testing.

It is essential for systems relying on unreliable or slow external services at scale.

Practice

1. What is the primary purpose of the circuit breaker pattern in system design?

easy

A. To prevent repeated calls to a failing service and improve system stability

B. To increase the number of requests sent to a service

C. To store user session data efficiently

D. To encrypt data during transmission

Solution

1. Step 1: Understand the circuit breaker pattern role

   The circuit breaker pattern is designed to stop sending requests to a service that is failing repeatedly to avoid wasting resources and cascading failures.

2. Step 2: Identify the main benefit

   By preventing repeated calls to a failing service, it helps maintain overall system stability and improves user experience by failing fast.

3. Final Answer:

   To prevent repeated calls to a failing service and improve system stability -> Option A

4. Quick Check:

   Circuit breaker purpose = prevent repeated failing calls [OK]

Hint: Circuit breaker stops calls to failing services fast [OK]

Common Mistakes:

• Confusing circuit breaker with caching
• Thinking it increases request volume
• Mixing it up with encryption or session management

2. Which of the following correctly describes the open state in a circuit breaker?

easy

A. The circuit breaker allows all requests to pass through

B. The circuit breaker resets all counters to zero

C. The circuit breaker tests a limited number of requests

D. The circuit breaker blocks all requests to the failing service

Solution

1. Step 1: Recall the circuit breaker states

   The circuit breaker has three states: closed (normal operation), open (blocking requests), and half-open (testing requests).

2. Step 2: Define the open state behavior

   In the open state, the circuit breaker blocks all requests to the failing service to prevent further failures.

3. Final Answer:

   The circuit breaker blocks all requests to the failing service -> Option D

4. Quick Check:

   Open state = block requests [OK]

Hint: Open state means block all requests [OK]

Common Mistakes:

• Confusing open with closed or half-open states
• Thinking open state allows requests
• Assuming counters reset in open state

3. Consider this simplified pseudocode for a circuit breaker:

if failure_count > threshold:
    state = 'open'
if state == 'open':
    return 'fail fast'
else:
    call_service()

What will happen if failure_count exceeds the threshold?

medium

A. The service call will continue normally

B. The circuit breaker will enter half-open state

C. The circuit breaker will return 'fail fast' without calling the service

D. The failure count will reset automatically

Solution

1. Step 1: Analyze the condition for failure count

   If failure_count is greater than threshold, the state is set to 'open'.

2. Step 2: Check behavior when state is 'open'

   When state is 'open', the code returns 'fail fast' and does not call the service.

3. Final Answer:

   The circuit breaker will return 'fail fast' without calling the service -> Option C

4. Quick Check:

   Failure count > threshold = fail fast [OK]

Hint: Open state returns fail fast, no service call [OK]

Common Mistakes:

• Assuming service call still happens
• Confusing open with half-open state
• Thinking failure count resets automatically

4. A circuit breaker is stuck in the open state and never transitions to half-open. What is the most likely cause?

medium

A. The timeout to reset the circuit breaker is missing or too long

B. The service is always healthy

C. The failure count threshold is set too low

D. The circuit breaker is not counting failures

Solution

1. Step 1: Understand state transitions in circuit breaker

   The circuit breaker moves from open to half-open after a timeout period to test if the service has recovered.

2. Step 2: Identify cause of stuck open state

   If the timeout is missing or set too long, the circuit breaker will never try half-open state and remain open indefinitely.

3. Final Answer:

   The timeout to reset the circuit breaker is missing or too long -> Option A

4. Quick Check:

   Missing timeout causes stuck open state [OK]

Hint: Timeout missing or too long keeps breaker open [OK]

Common Mistakes:

• Confusing failure threshold with timeout
• Assuming service health affects state directly
• Ignoring the role of failure counting

5. You design a system using the circuit breaker pattern to call a payment service. The service fails intermittently. How should you configure the circuit breaker to balance availability and fault tolerance?

hard

A. Set a high failure threshold and long timeout to avoid blocking the service too soon

B. Set a low failure threshold and short timeout to quickly block and retry the service

C. Disable the circuit breaker to avoid blocking any requests

D. Set failure threshold to zero to block all requests immediately

Solution

1. Step 1: Understand intermittent failure impact

   Intermittent failures mean the service sometimes works and sometimes fails, so quick detection and retry is important.

2. Step 2: Choose configuration for balance

   A low failure threshold and short timeout allow the circuit breaker to quickly block failing calls and retry soon, improving fault tolerance and availability.

3. Final Answer:

   Set a low failure threshold and short timeout to quickly block and retry the service -> Option B

4. Quick Check:

   Low threshold + short timeout balances availability and fault tolerance [OK]

Hint: Low threshold and short timeout balance retries and blocking [OK]

Common Mistakes:

• Setting threshold too high delays failure detection
• Disabling circuit breaker risks cascading failures
• Setting threshold zero blocks all requests unnecessarily