Event-driven design in LLD - System Design Guide

Problem Statement

When components in a system are tightly coupled, a change or failure in one part can cause cascading failures or slow down the entire system. This tight coupling also makes it hard to add new features or scale parts independently, leading to slow development and fragile systems.

Solution

Event-driven design decouples components by having them communicate through events. When something happens, a component emits an event, and other components listen and react to those events independently. This allows parts of the system to work asynchronously and scale separately without waiting for each other.

Architecture

Producer

→Event Broker

↓

Consumer

Consumer

This diagram shows a producer sending events to an event broker, which then forwards those events to one or more consumers that have subscribed to them.

Trade-offs

✓ Pros

→

Decouples components, allowing independent development and deployment.

→

Improves scalability by enabling asynchronous processing and load distribution.

→

Enhances system resilience since failures in one component do not directly block others.

✗ Cons

→

Increases complexity in event management and debugging due to asynchronous flows.

→

Requires careful design of event schemas and handling of eventual consistency.

→

May introduce latency as events propagate through the system asynchronously.

Use when your system has multiple independent components that need to react to changes asynchronously, especially at scale above thousands of events per second or when you want to improve modularity and fault tolerance.

Avoid when your system requires strict synchronous operations or immediate consistency, or when the event volume is very low (under hundreds per second) making the added complexity unnecessary.

Real World Examples

Uber

Uber uses event-driven design to decouple services like ride requests, driver matching, and payment processing, allowing each to scale and evolve independently.

Netflix

Netflix uses event-driven architecture to handle user activity events for recommendations and monitoring, enabling real-time processing and fault isolation.

Amazon

Amazon employs event-driven design in its order processing system to asynchronously update inventory, payment, and shipping services.

Code Example

The before code shows OrderService calling payment and inventory services directly, creating tight coupling. The after code introduces an EventBus where OrderService publishes an 'order_created' event. PaymentService and InventoryService subscribe to this event and react independently, decoupling the components and enabling asynchronous processing.

LLD

### Before: tightly coupled synchronous calls
class OrderService:
    def create_order(self, order):
        payment_service.process_payment(order)
        inventory_service.update_stock(order)

### After: event-driven design with event bus
class EventBus:
    def __init__(self):
        self.listeners = {}

    def subscribe(self, event_type, listener):
        self.listeners.setdefault(event_type, []).append(listener)

    def publish(self, event_type, data):
        for listener in self.listeners.get(event_type, []):
            listener(data)

class OrderService:
    def __init__(self, event_bus):
        self.event_bus = event_bus

    def create_order(self, order):
        # Save order logic here
        self.event_bus.publish('order_created', order)

class PaymentService:
    def __init__(self, event_bus):
        event_bus.subscribe('order_created', self.process_payment)

    def process_payment(self, order):
        # Payment processing logic
        pass

class InventoryService:
    def __init__(self, event_bus):
        event_bus.subscribe('order_created', self.update_stock)

    def update_stock(self, order):
        # Stock update logic
        pass

# Usage
bus = EventBus()
payment_service = PaymentService(bus)
inventory_service = InventoryService(bus)
order_service = OrderService(bus)
order_service.create_order({'id': 123, 'items': ['book', 'pen']})

Alternatives

Request-Response

Components communicate synchronously by sending requests and waiting for responses.

Use when: Choose when operations require immediate results and tight coordination between components.

Batch Processing

Data is collected and processed in large groups at scheduled times rather than reacting to individual events.

Use when: Choose when real-time processing is not needed and throughput optimization is more important.

Summary

Event-driven design decouples system components by using events to communicate asynchronously.

It improves scalability, fault tolerance, and modularity by allowing independent processing.

This design requires careful event management and is best for systems needing asynchronous workflows at scale.