Redundancy and fault tolerance in HLD - Architecture Diagram

System Overview - Redundancy and fault tolerance

This system is designed to remain available and reliable even when some parts fail. It uses multiple copies of components and smart routing to avoid downtime. The goal is to keep the service running smoothly without interruption.

Architecture Diagram

          +----------------+
          |     Users      |
          +--------+-------+
                   |
           +-------v-------+
           | Load Balancer |<----------------+
           +-------+-------+                 |
                   |                         |
    +--------------+--------------+          |
    |                             |          |
+---v---+                     +---v---+      |
| API 1 |                     | API 2 |      |
+---+---+                     +---+---+      |
    |                             |          |
    +-------------+---------------+          |
                  |                          |
          +-------v-------+                  |
          | Message Queue |------------------+
          +-------+-------+
                  |
          +-------v-------+
          | Worker Nodes  |
          +-------+-------+
                  |
          +-------v-------+
          |  Database(s)  |
          +-------+-------+
                  |
          +-------v-------+
          |     Cache     |
          +---------------+

Components

Users

client

End users who send requests to the system

Load Balancer

load_balancer

Distributes incoming requests evenly across API servers

API 1

service

Handles user requests and business logic

API 2

service

Backup API server to ensure availability if API 1 fails

Message Queue

queue

Decouples request processing and enables retry on failure

Worker Nodes

service

Processes tasks asynchronously from the queue

Database(s)

database

Stores persistent data with replication for fault tolerance

Cache

cache

Speeds up data retrieval and reduces load on the database

Request Flow - 10 Hops

Users → Load Balancer

Load Balancer → API 1

API 1 → Cache

Cache → API 1

API 1 → Message Queue

Message Queue → Worker Nodes

Worker Nodes → Database(s)

Database(s) → Cache

API 1 → Load Balancer

Load Balancer → Users

Failure Scenario

Component Fails:API 1

Impact:Requests routed to API 1 fail, causing service disruption

Mitigation:Load balancer detects failure and routes traffic to API 2 automatically

Architecture Quiz - 3 Questions

What component ensures requests are evenly distributed to multiple API servers?

A Message Queue

B Load Balancer

C Cache

D Worker Nodes

Design Principle

This architecture uses redundancy by having multiple API servers and fault tolerance through components like load balancers, caches, and message queues. These ensure the system stays available and responsive even if some parts fail.

Practice

1. What is the main purpose of redundancy in system design?

easy

A. To simplify the system architecture

B. To increase the speed of data processing

C. To have backup components to avoid single points of failure

D. To reduce the cost of hardware

Solution

1. Step 1: Understand redundancy concept

   Redundancy means having extra components or backups so if one fails, the system still works.

2. Step 2: Identify the main goal of redundancy

   The goal is to avoid single points of failure by having backups ready.

3. Final Answer:

   To have backup components to avoid single points of failure -> Option C

4. Quick Check:

   Redundancy = Backup components [OK]

Hint: Redundancy means backups to prevent failure [OK]

Common Mistakes:

• Confusing redundancy with performance improvement
• Thinking redundancy reduces cost
• Assuming redundancy simplifies architecture

2. Which of the following is a correct example of a fault-tolerant system component?

easy

A. A load balancer distributing traffic to multiple servers

B. A single server with no backup

C. A database without replication

D. A system that shuts down on any error

Solution

1. Step 1: Identify fault tolerance features

   Fault tolerance means the system can keep working even if some parts fail.

2. Step 2: Evaluate options for fault tolerance

   A load balancer sends traffic to multiple servers, so if one fails, others handle requests.

3. Final Answer:

   A load balancer distributing traffic to multiple servers -> Option A

4. Quick Check:

   Fault tolerance = multiple servers with load balancer [OK]

Hint: Fault tolerance needs multiple working parts, not single points [OK]

Common Mistakes:

• Choosing single server as fault tolerant
• Ignoring replication or backups
• Confusing fault tolerance with error shutdown

3. Consider a system with two identical servers running in active-active mode for fault tolerance. If one server fails, what happens to the incoming requests?

medium

A. The system shuts down to prevent errors

B. All requests are lost until the failed server is fixed

C. Requests are queued indefinitely with no processing

D. Requests are automatically handled by the remaining active server

Solution

1. Step 1: Understand active-active mode

   In active-active, both servers handle requests simultaneously to share load and provide backup.

2. Step 2: Analyze failure impact

   If one server fails, the other continues handling all requests without interruption.

3. Final Answer:

   Requests are automatically handled by the remaining active server -> Option D

4. Quick Check:

   Active-active means failover to active server [OK]

Hint: Active-active means all servers handle requests; failure shifts load [OK]

Common Mistakes:

• Thinking requests are lost on failure
• Assuming system queues requests indefinitely
• Believing system shuts down on failure

4. A system uses a single database without replication. It crashes unexpectedly. What is the main fault tolerance issue here?

medium

A. Load balancer misconfiguration

B. No backup database to take over on failure

C. Too many replicas causing conflicts

D. Excessive redundancy causing delays

Solution

1. Step 1: Identify fault tolerance problem

   Fault tolerance requires backups or replicas to continue working if one fails.

2. Step 2: Analyze the system setup

   Single database without replication means no backup exists to handle failure.

3. Final Answer:

   No backup database to take over on failure -> Option B

4. Quick Check:

   No replication = no fault tolerance backup [OK]

Hint: No replication means no fault tolerance backup [OK]

Common Mistakes:

• Confusing replication with too many replicas
• Blaming load balancer for database crash
• Thinking redundancy causes delays here

5. You are designing a fault-tolerant web service that must remain available even if one data center fails. Which design choice best supports this requirement?

hard

A. Deploy the service in two geographically separate data centers with data replication

B. Deploy the service in one data center with multiple servers

C. Use a single powerful server with backup power supply

D. Deploy the service in one data center and rely on daily backups

Solution

1. Step 1: Understand availability requirement

   The service must stay available even if one entire data center fails.

2. Step 2: Evaluate design options for fault tolerance

   Deploying in two separate data centers with data replication ensures if one fails, the other serves requests.

3. Step 3: Compare other options

   Single data center or single server cannot handle full data center failure; daily backups do not provide immediate availability.

4. Final Answer:

   Deploy the service in two geographically separate data centers with data replication -> Option A

5. Quick Check:

   Geo-redundancy + replication = high availability [OK]

Hint: Use multiple data centers with replication for fault tolerance [OK]

Common Mistakes:

• Relying on single data center for fault tolerance
• Confusing backup power with data center failure protection
• Thinking daily backups ensure immediate availability