Pub/sub pattern in HLD - System Design Guide

Problem Statement

When multiple components need to communicate, tightly coupling them causes failures to cascade and makes scaling difficult. Direct communication means every sender must know every receiver, leading to complex dependencies and fragile systems.

Solution

The pub/sub pattern decouples senders and receivers by introducing a message broker. Publishers send messages to topics without knowing who subscribes. Subscribers listen to topics and receive messages asynchronously, enabling flexible, scalable communication.

Architecture

Publisher

↓

Broker

(Message

↓

Subscriber 1

This diagram shows publishers sending messages to a broker, which then forwards messages to multiple subscribers asynchronously.

Trade-offs

✓ Pros

→

Decouples components, reducing dependencies and improving system flexibility.

→

Supports asynchronous communication, improving responsiveness and scalability.

→

Enables multiple subscribers to receive the same message without extra load on the publisher.

→

Simplifies adding or removing subscribers without changing publishers.

✗ Cons

→

Introduces a broker component, adding operational complexity and a potential single point of failure.

→

Message delivery latency can increase due to asynchronous processing.

→

Requires careful handling of message ordering and duplication depending on broker capabilities.

Use when multiple independent components need to react to events or data changes asynchronously, especially at scale above thousands of messages per second or when decoupling is critical.

Avoid when communication is simple point-to-point with low message volume under 1000 messages per second or when strict synchronous request-response is required.

Real World Examples

Netflix

Uses pub/sub to decouple microservices for event-driven workflows, allowing independent scaling and failure isolation.

Uber

Employs pub/sub for real-time location updates and notifications to multiple services without tight coupling.

Amazon

Uses pub/sub in AWS SNS to fan out notifications to multiple endpoints, enabling scalable alerting and messaging.

Alternatives

Direct point-to-point messaging

Senders communicate directly with receivers without an intermediary broker.

Use when: Use when the number of components is small and tightly coupled communication is acceptable.

Message queue (point-to-point)

Messages are sent to a queue consumed by a single receiver, unlike pub/sub which supports multiple subscribers.

Use when: Choose when tasks need to be processed by only one consumer to balance load.

Event streaming

Stores and streams ordered event logs for replay and processing, unlike pub/sub which focuses on message delivery.

Use when: Use when event history and replayability are important.

Summary

Pub/sub decouples senders and receivers by using a broker to route messages asynchronously.

It enables scalable, flexible communication where multiple subscribers can receive the same message.

This pattern is ideal for event-driven systems requiring loose coupling and asynchronous processing.