Pub/sub pattern in HLD - Scalability & System Analysis

Scalability Analysis - Pub/sub pattern

Growth Table: Pub/Sub Pattern Scaling

Scale	Users/Clients	Messages per Second	System Changes
Small	100 users	100-500 msg/s	Single broker, simple topic management, direct message delivery
Medium	10,000 users	10,000-50,000 msg/s	Multiple brokers, partitioned topics, message persistence, basic load balancing
Large	1,000,000 users	1M+ msg/s	Clustered brokers, topic sharding, advanced load balancing, message replication, durable storage
Very Large	100,000,000 users	100M+ msg/s	Geo-distributed clusters, multi-region replication, CDN integration, hierarchical topic routing

First Bottleneck

At small to medium scale, the message broker is the first bottleneck. It handles all message routing and delivery. As users and message rates grow, a single broker's CPU, memory, and network limits are reached quickly.

Also, the network bandwidth between publishers, brokers, and subscribers becomes a bottleneck as message volume increases.

Scaling Solutions

• Horizontal scaling: Add more broker instances and distribute topics among them (partitioning/sharding).
• Load balancing: Use load balancers to distribute client connections evenly across brokers.
• Caching: Use subscriber-side caching or edge caches for frequently requested messages.
• Message persistence: Store messages durably to allow replay and reduce load spikes.
• Geo-distribution: Deploy brokers in multiple regions to reduce latency and network load.
• CDN integration: For static or large messages, use CDNs to offload delivery.

Back-of-Envelope Cost Analysis

Assuming 10,000 messages per second at medium scale:

• Each message ~1 KB → 10 MB/s bandwidth needed.
• Broker CPU: 1 server can handle ~5,000 msg/s → need 2+ brokers.
• Storage: For message persistence, 10,000 msg/s × 1 KB × 3600 s = ~36 GB/hour.
• Network: 1 Gbps link (~125 MB/s) sufficient for this scale.

Interview Tip

Start by explaining the pub/sub components: publishers, subscribers, and brokers.

Discuss how message volume and user count affect broker load and network.

Identify the bottleneck (broker capacity) and propose scaling with partitioning and horizontal scaling.

Mention persistence and geo-distribution for reliability and latency.

Self Check

Your message broker handles 1,000 messages per second. Traffic grows 10x to 10,000 msg/s. What do you do first?

Answer: Add more broker instances and partition topics to distribute load horizontally, because a single broker cannot handle 10x the load.

Key Result

The message broker is the first bottleneck in pub/sub systems; horizontal scaling with partitioned brokers and load balancing is key to handle growth from thousands to millions of messages per second.

Practice

1. What is the main purpose of the Pub/Sub pattern in system design?

easy

A. To decouple message senders from receivers for flexible communication

B. To tightly couple components for faster data exchange

C. To store data permanently in a database

D. To encrypt messages for security

Solution

1. Step 1: Understand the role of Pub/Sub

   Pub/Sub pattern separates the sender (publisher) and receiver (subscriber) so they do not depend on each other directly.

2. Step 2: Identify the main benefit

   This separation allows flexible communication and easier scaling of systems.

3. Final Answer:

   To decouple message senders from receivers for flexible communication -> Option A

4. Quick Check:

   Decoupling = A [OK]

Hint: Pub/Sub means senders and receivers don't know each other [OK]

Common Mistakes:

• Thinking Pub/Sub stores data permanently
• Confusing Pub/Sub with encryption
• Assuming tight coupling between components

2. Which of the following is the correct component in a Pub/Sub system?

easy

A. Sender, Receiver, Firewall

B. Client, Server, Database

C. Publisher, Subscriber, Message Broker

D. Producer, Consumer, Cache

Solution

1. Step 1: Recall Pub/Sub components

   Pub/Sub systems have publishers (senders), subscribers (receivers), and a message broker that routes messages.

2. Step 2: Match components to options

   Only Publisher, Subscriber, Message Broker lists these three correctly.

3. Final Answer:

   Publisher, Subscriber, Message Broker -> Option C

4. Quick Check:

   Pub/Sub components = D [OK]

Hint: Remember the broker routes messages between publishers and subscribers [OK]

Common Mistakes:

• Confusing Pub/Sub with client-server terms
• Mixing cache or firewall as core components
• Using generic terms like producer/consumer without broker

3. Consider a Pub/Sub system where a publisher sends messages to a topic. If three subscribers are subscribed to that topic, how many copies of each message does the broker send?

medium

A. One copy shared among all subscribers

B. Three copies, one for each subscriber

C. Zero copies, subscribers pull messages themselves

D. Depends on the message size

Solution

1. Step 1: Understand message delivery in Pub/Sub

   Each subscriber receives its own copy of the message from the broker to ensure independent processing.

2. Step 2: Calculate copies sent

   With three subscribers, the broker sends three copies, one per subscriber.

3. Final Answer:

   Three copies, one for each subscriber -> Option B

4. Quick Check:

   Copies = Subscribers count = B [OK]

Hint: Broker sends one message per subscriber, not shared [OK]

Common Mistakes:

• Assuming one copy shared by all subscribers
• Thinking subscribers pull messages without broker sending
• Believing message size affects copy count

4. A developer implemented a Pub/Sub system but notices subscribers never receive messages. Which is the most likely cause?

medium

A. Subscribers are using the wrong database

B. Publisher is sending messages too frequently

C. Message broker is encrypting messages

D. Subscribers are not subscribed to the correct topic

Solution

1. Step 1: Analyze why subscribers get no messages

   If subscribers never receive messages, a common cause is they are not subscribed to the topic where messages are published.

2. Step 2: Evaluate other options

   Sending frequency or encryption does not prevent delivery; wrong database is unrelated.

3. Final Answer:

   Subscribers are not subscribed to the correct topic -> Option D

4. Quick Check:

   Subscription mismatch = A [OK]

Hint: Check subscriber topic matches publisher topic first [OK]

Common Mistakes:

• Blaming message frequency for delivery failure
• Assuming encryption blocks delivery
• Confusing database usage with Pub/Sub messaging

5. You need to design a scalable notification system using Pub/Sub. Which approach best ensures messages are delivered even if some subscribers are temporarily offline?

hard

A. Use a message broker with persistent storage and durable subscriptions

B. Send messages directly from publisher to subscribers without broker

C. Use a broker that deletes messages immediately after sending

D. Require subscribers to poll the publisher continuously

Solution

1. Step 1: Identify requirements for offline subscriber support

   To handle offline subscribers, messages must be stored until subscribers can receive them.

2. Step 2: Choose the right Pub/Sub feature

   Persistent storage and durable subscriptions in the broker ensure messages are saved and delivered later.

3. Step 3: Evaluate other options

   Direct sending or immediate deletion loses messages; polling is inefficient and not scalable.

4. Final Answer:

   Use a message broker with persistent storage and durable subscriptions -> Option A

5. Quick Check:

   Durable subscriptions = reliable delivery = C [OK]

Hint: Durable subscriptions keep messages for offline subscribers [OK]

Common Mistakes:

• Skipping message persistence in broker
• Trying direct publisher-to-subscriber without broker
• Relying on polling instead of push delivery