Real-time features in HLD - System Design Exercise

Design: Real-time Notification System

Design covers backend architecture, message delivery, subscription management, and client communication. Out of scope are UI design and third-party integrations.

Functional Requirements

FR1: Deliver notifications to users instantly when events occur

FR2: Support 100,000 concurrent users receiving notifications

FR3: Allow users to subscribe/unsubscribe to different notification types

FR4: Ensure message delivery order is preserved per user

FR5: Provide at-least-once delivery guarantee

FR6: Support web and mobile clients

Non-Functional Requirements

NFR1: Latency for delivering notifications should be under 200ms (p99)

NFR2: System availability should be 99.9% uptime

NFR3: Handle peak loads of 10,000 notifications per second

NFR4: Support horizontal scaling for growing user base

Think Before You Design

Questions to Ask

❓ Question 1

❓ Question 2

❓ Question 3

❓ Question 4

❓ Question 5

❓ Question 6

Key Components

Event producer or source

Message broker or queue system

Subscription management service

Notification delivery service

Client communication protocols (WebSocket, HTTP/2, Push APIs)

Cache or in-memory store for fast lookup

Database for persistence

Design Patterns

Publish-Subscribe pattern

Event-driven architecture

Message queue with topic partitions

Push notification protocols

Backpressure and rate limiting

Retry and dead-letter queue handling

Reference Architecture

  +----------------+       +------------------+       +---------------------+
  | Event Producer | ----> | Message Broker   | ----> | Notification Service |
  +----------------+       +------------------+       +---------------------+
                                   |                             |
                                   |                             v
                             +-------------+             +--------------+
                             | Subscription|             | Client       |
                             | Management  |             | Communication|
                             +-------------+             +--------------+

Components

Event Producer

Any backend service or external system

Generates events that trigger notifications

Message Broker

Apache Kafka or RabbitMQ

Queues and routes notification events to consumers

Subscription Management

Relational DB (PostgreSQL) + Cache (Redis)

Stores user subscriptions and preferences

Notification Service

Microservice in Node.js or Java

Processes events, checks subscriptions, sends notifications

Client Communication

WebSocket, HTTP/2 Push, or Firebase Cloud Messaging

Delivers notifications to web and mobile clients in real-time

Request Flow

1. 1. Event Producer creates a notification event and sends it to the Message Broker.

2. 2. Message Broker stores and routes the event to the Notification Service.

3. 3. Notification Service queries Subscription Management to find users subscribed to this event type.

4. 4. Notification Service sends notifications to clients via Client Communication protocols.

5. 5. Clients receive notifications instantly and display them to users.

Database Schema

Entities: - User (user_id, name, contact_info) - Subscription (subscription_id, user_id, notification_type, is_active) - NotificationEvent (event_id, type, payload, timestamp) Relationships: - User 1:N Subscription - NotificationEvent linked to multiple Subscriptions via notification_type This schema supports tracking which users receive which notifications.

Scaling Discussion

Bottlenecks

Message Broker throughput limits under high event volume

Notification Service CPU and memory under heavy processing

Subscription Management DB read latency for large user base

Client connections limit for WebSocket servers

Solutions

Partition topics in Message Broker and scale brokers horizontally

Deploy multiple instances of Notification Service with load balancing

Use caching (Redis) to reduce DB reads for subscriptions

Use multiple WebSocket servers with sticky sessions and load balancers

Interview Tips

Time: Spend 10 minutes clarifying requirements and constraints, 20 minutes designing architecture and data flow, 10 minutes discussing scaling and trade-offs, 5 minutes summarizing.

Explain real-time delivery challenges and low latency needs

Discuss choice of message broker and pub-sub pattern

Highlight subscription management and filtering logic

Describe client communication protocols and fallback options

Address scaling bottlenecks and horizontal scaling strategies

Practice

1. Which protocol is commonly used to enable real-time communication in web applications?

easy

A. SMTP

B. HTTP/1.1

C. WebSocket

D. FTP

Solution

1. Step 1: Understand real-time communication needs

   Real-time apps require a protocol that supports two-way, instant data exchange.

2. Step 2: Identify protocol features

   WebSocket allows full-duplex communication over a single connection, unlike HTTP/1.1 which is request-response only.

3. Final Answer:

   WebSocket -> Option C

4. Quick Check:

   Real-time = WebSocket [OK]

Hint: Real-time needs two-way instant data flow: WebSocket fits best [OK]

Common Mistakes:

• Confusing HTTP with WebSocket for real-time
• Choosing FTP or SMTP which are not real-time protocols
• Thinking HTTP/2 is the same as WebSocket

2. Which component in a real-time system is responsible for distributing messages from producers to consumers?

easy

A. Broker

B. Producer

C. Consumer

D. Database

Solution

1. Step 1: Define roles in real-time messaging

   Producers send data, consumers receive data, and brokers route messages between them.

2. Step 2: Identify the distributor

   The broker acts as the middleman ensuring messages reach the right consumers.

3. Final Answer:

   Broker -> Option A

4. Quick Check:

   Message routing = Broker [OK]

Hint: Broker connects producers and consumers by routing messages [OK]

Common Mistakes:

• Confusing producer as distributor
• Thinking consumer sends messages
• Assuming database handles message routing

3. Consider a chat app using WebSocket. If the server receives a message from User A and broadcasts it to 100 connected users, what is the main bottleneck to scale this real-time feature?

medium

A. User A's device speed

B. Client browser rendering speed

C. Database read latency

D. Server CPU and network bandwidth

Solution

1. Step 1: Analyze message flow in real-time chat

   The server receives and then sends the message to all connected users, requiring CPU and network resources.

2. Step 2: Identify bottleneck

   Server CPU handles message processing; network bandwidth handles sending to many users simultaneously.

3. Final Answer:

   Server CPU and network bandwidth -> Option D

4. Quick Check:

   Scaling real-time = Server resources [OK]

Hint: Server resources limit broadcast scale, not client or DB speed [OK]

Common Mistakes:

• Blaming client device speed for server load
• Focusing on database latency which is less critical here
• Ignoring network bandwidth limits

4. A real-time notification system uses MQTT but users report delayed messages. Which is the most likely cause?

medium

A. Clients are using WebSocket instead of MQTT

B. Broker is overloaded and dropping messages

C. Messages are too small to send quickly

D. Users have disabled notifications on their devices

Solution

1. Step 1: Understand MQTT broker role

   The broker routes messages; if overloaded, it queues or drops messages causing delays.

2. Step 2: Evaluate other options

   Clients using WebSocket instead of MQTT would cause connection issues, not delays; small messages send faster; disabled notifications affect display, not delivery.

3. Final Answer:

   Broker is overloaded and dropping messages -> Option B

4. Quick Check:

   Delays usually mean broker overload [OK]

Hint: Delays often mean broker overload, not client protocol mismatch [OK]

Common Mistakes:

• Blaming client protocol mismatch for delays
• Assuming small messages cause delays
• Ignoring broker capacity limits

5. You are designing a real-time stock price update system for millions of users. Which approach best ensures scalability and low latency?

hard

A. Use a distributed message broker cluster with topic partitions and WebSocket connections

B. Store all prices in a single database and poll clients every second

C. Send updates via email to all users when prices change

D. Use HTTP long polling from clients to server for updates

Solution

1. Step 1: Understand scalability needs

   Millions of users require distributed systems to handle load and maintain low latency.

2. Step 2: Evaluate options

   Distributed brokers with topic partitions allow parallel processing; WebSocket supports instant push updates. Polling and email cause delays and high load.

3. Final Answer:

   Use a distributed message broker cluster with topic partitions and WebSocket connections -> Option A

4. Quick Check:

   Scalable real-time = Distributed broker + WebSocket [OK]

Hint: Distributed brokers + WebSocket scale best for millions [OK]

Common Mistakes:

• Choosing polling which wastes resources and adds latency
• Using email which is not real-time
• Relying on a single database causing bottlenecks