Event-driven design in LLD - Architecture Diagram

System Overview - Event-driven design

This system uses event-driven design to handle tasks asynchronously. When a user triggers an action, an event is created and sent to a message queue. Services listen for these events and process them independently, allowing the system to be scalable and responsive.

Architecture Diagram

User
  |
  v
Load Balancer
  |
  v
API Gateway
  |
  v
Event Producer Service
  |
  v
Message Queue
 /   \
v     v
Worker Service A  Worker Service B
  |               |
  v               v
Database A       Database B
  ^               ^
  |               |
Cache A         Cache B
  

Components

User

user

Initiates requests to the system

Load Balancer

load_balancer

Distributes incoming requests evenly to API Gateway instances

API Gateway

api_gateway

Receives user requests and forwards them to the Event Producer Service

Event Producer Service

service

Creates events based on user requests and sends them to the Message Queue

Message Queue

message_queue

Holds events until Worker Services consume and process them asynchronously

Worker Service A

service

Consumes events from the queue and processes tasks related to Database A

Worker Service B

service

Consumes events from the queue and processes tasks related to Database B

Database A

database

Stores data processed by Worker Service A

Database B

database

Stores data processed by Worker Service B

Cache A

cache

Speeds up data retrieval for Database A

Cache B

cache

Speeds up data retrieval for Database B

Request Flow - 13 Hops

User → Load Balancer

Load Balancer → API Gateway

API Gateway → Event Producer Service

Event Producer Service → Message Queue

Message Queue → Worker Service A

Worker Service A → Cache A

Worker Service A → Database A

Worker Service A → Cache A

Worker Service A → Message Queue

Message Queue → Worker Service B

Worker Service B → Cache B

Worker Service B → Database B

Worker Service B → Cache B

Failure Scenario

Component Fails:Message Queue

Impact:Events cannot be delivered to Worker Services, causing processing delays and possible data loss

Mitigation:Use a replicated and durable message queue with retry mechanisms and dead-letter queues to handle failures

Architecture Quiz - 3 Questions

Which component is responsible for holding events until workers process them?

A API Gateway

B Load Balancer

C Message Queue

D Cache

Design Principle

This architecture shows how event-driven design decouples components using a message queue, enabling asynchronous processing and better scalability. It also demonstrates how caches improve performance by reducing database load.

Practice

1. What is the main purpose of event-driven design in system architecture?

easy

A. To allow systems to react to actions as they happen asynchronously

B. To process all tasks sequentially in a fixed order

C. To store data permanently in a database

D. To create static web pages without user interaction

Solution

1. Step 1: Understand event-driven design concept

   Event-driven design focuses on reacting to events or actions as they occur, rather than processing everything in a fixed sequence.

2. Step 2: Compare options with concept

   To allow systems to react to actions as they happen asynchronously matches this idea by describing asynchronous reaction to actions. Other options describe unrelated concepts like sequential processing, data storage, or static content.

3. Final Answer:

   To allow systems to react to actions as they happen asynchronously -> Option A

4. Quick Check:

   Event-driven design = react asynchronously [OK]

Hint: Event-driven means reacting to events as they happen [OK]

Common Mistakes:

• Confusing event-driven with sequential processing
• Thinking event-driven is about data storage
• Assuming event-driven means static content

2. Which of the following is the correct sequence in an event-driven system?

easy

A. Consumer -> Producer -> Queue

B. Producer -> Consumer -> Queue

C. Queue -> Producer -> Consumer

D. Producer -> Queue -> Consumer

Solution

1. Step 1: Identify roles in event-driven flow

   Producers create events, queues hold events, and consumers process events.

2. Step 2: Arrange correct order

   The correct order is Producer sends event to Queue, then Consumer reads from Queue.

3. Final Answer:

   Producer -> Queue -> Consumer -> Option D

4. Quick Check:

   Producer creates, Queue holds, Consumer processes [OK]

Hint: Events flow: Producer to Queue to Consumer [OK]

Common Mistakes:

• Mixing up producer and consumer order
• Placing queue after consumer
• Ignoring the queue role

3. Consider this simplified event-driven code snippet:

event_queue = []

def produce(event):
    event_queue.append(event)

def consume():
    if event_queue:
        return event_queue.pop(0)
    return None

produce('A')
produce('B')
print(consume())
print(consume())
print(consume())

What is the output?

medium

A. None None None

B. B A None

C. A B None

D. A None B

Solution

1. Step 1: Trace event production

   Two events 'A' and 'B' are added to the queue in order: ['A', 'B'].

2. Step 2: Trace event consumption

   consume() removes and returns the first event: first 'A', then 'B', then None when empty.

3. Final Answer:

   A B None -> Option C

4. Quick Check:

   FIFO queue returns A then B then None [OK]

Hint: Queue pops first-in event first (FIFO) [OK]

Common Mistakes:

• Assuming LIFO instead of FIFO
• Forgetting to check empty queue
• Mixing order of events

4. In an event-driven system, a developer wrote this code snippet:

def consume(event_queue):
    event = event_queue.pop()
    process(event)

What is the main issue with this code?

medium

A. It does not check if the queue is empty before popping

B. It adds events instead of removing them

C. It uses an undefined function 'process'

D. It processes events in reverse order, not FIFO

Solution

1. Step 1: Analyze pop usage without check

   pop() removes last item but no check if queue is empty, risking error.

2. Step 2: Identify error risk

   Calling pop() on empty list causes runtime error; code lacks safety check.

3. Final Answer:

   It does not check if the queue is empty before popping -> Option A

4. Quick Check:

   pop() on empty list causes error [OK]

Hint: Always check queue not empty before pop() [OK]

Common Mistakes:

• Ignoring empty queue check
• Confusing pop() order with error
• Assuming process() is undefined error

5. You are designing a scalable event-driven system for a social media app. Which approach best improves scalability and fault tolerance?

hard

A. Store all events in a database and process them synchronously

B. Use a distributed message queue with multiple consumers processing events in parallel

C. Use a single queue and one consumer to ensure event order

D. Send events directly from producer to consumer without queue

Solution

1. Step 1: Understand scalability and fault tolerance needs

   Social media apps have high event volume; parallel processing and fault tolerance are key.

2. Step 2: Evaluate options for scalability

   Distributed queues with multiple consumers allow load balancing and fault tolerance. Single consumer limits throughput. Synchronous processing blocks system. Direct send lacks buffering and fault tolerance.

3. Final Answer:

   Use a distributed message queue with multiple consumers processing events in parallel -> Option B

4. Quick Check:

   Distributed queues + parallel consumers = scalable & fault tolerant [OK]

Hint: Parallel consumers on distributed queue scale best [OK]

Common Mistakes:

• Choosing single consumer limits throughput
• Ignoring asynchronous processing benefits
• Skipping queue leads to lost events