Long polling and Server-Sent Events in HLD - System Design Exercise

Design: Real-time Notification System using Long Polling and Server-Sent Events

Design covers server architecture, client-server communication using Long Polling and Server-Sent Events, connection management, and fallback mechanisms. Out of scope are message content design and authentication mechanisms.

Functional Requirements

FR1: Deliver real-time notifications from server to clients

FR2: Support up to 10,000 concurrent clients

FR3: Ensure notifications are delivered with latency under 1 second

FR4: Allow clients to receive updates without frequent polling

FR5: Support fallback from Server-Sent Events to Long Polling if SSE is not supported

FR6: Handle client reconnections gracefully without losing messages

Non-Functional Requirements

NFR1: System must maintain 99.9% uptime (about 8.77 hours downtime per year)

NFR2: API response latency p99 under 500ms for connection establishment

NFR3: Support clients on modern browsers and some legacy browsers

NFR4: Minimize server resource usage for idle connections

Think Before You Design

Questions to Ask

❓ Question 1

❓ Question 2

❓ Question 3

❓ Question 4

❓ Question 5

Key Components

HTTP server capable of handling long-lived connections

Event source endpoint for Server-Sent Events

Long polling endpoint for fallback

Message queue or broker for notification delivery

Connection manager to track active clients

Client-side logic to handle SSE and fallback to long polling

Design Patterns

Publish-Subscribe pattern for notifications

Fallback pattern from SSE to Long Polling

Heartbeat or ping messages to keep connections alive

Exponential backoff for client reconnection attempts

Reference Architecture

                    +---------------------+
                    |  Notification Source |
                    +----------+----------+
                               |
                               v
                    +---------------------+
                    |   Message Broker    |
                    +----------+----------+
                               |
        +----------------------+-----------------------+
        |                                              |
+-------v--------+                             +-------v--------+
| HTTP Server SSE |                             | HTTP Server LP |
| (Event Source)  |                             | (Long Polling) |
+-------+--------+                             +-------+--------+
        |                                              |
        |                                              |
+-------v--------+                             +-------v--------+
| Connection     |                             | Connection     |
| Manager       |                             | Manager       |
+-------+--------+                             +-------+--------+
        |                                              |
        +----------------------+-----------------------+
                               |
                               v
                      +---------------------+
                      |      Clients        |
                      +---------------------+

Components

Notification Source

Any backend service or application

Generates notifications or events to be sent to clients

Message Broker

Redis Pub/Sub or RabbitMQ

Queues and distributes notifications to HTTP servers

HTTP Server SSE

Node.js with Express and EventSource API

Handles Server-Sent Events connections, pushes events to clients

HTTP Server LP

Node.js with Express

Handles long polling requests as fallback for clients without SSE support

Connection Manager

In-memory store or Redis

Tracks active client connections and manages reconnections

Clients

Modern browsers with EventSource API and fallback JavaScript

Receive notifications via SSE or long polling, handle reconnections

Request Flow

1. 1. Notification Source publishes an event to the Message Broker.

2. 2. HTTP Servers subscribe to the Message Broker to receive notifications.

3. 3. When a notification arrives, HTTP Server SSE pushes it to connected clients via SSE.

4. 4. Clients with SSE support receive events in real-time over a persistent HTTP connection.

5. 5. Clients without SSE support send long polling requests to HTTP Server LP.

6. 6. HTTP Server LP holds the request open until a notification is available or timeout occurs.

7. 7. Upon notification, HTTP Server LP responds to the client and client immediately sends a new long polling request.

8. 8. Connection Manager tracks active connections and handles reconnections with exponential backoff.

9. 9. Heartbeat messages are sent periodically to keep connections alive and detect dropped clients.

Database Schema

Entities: - ClientConnection: {client_id (PK), connection_type (SSE or LP), last_heartbeat, status} - Notification: {notification_id (PK), content, timestamp} - Subscription: {client_id (FK), topic} Relationships: - One ClientConnection can have multiple Subscriptions - Notifications are published to topics which clients subscribe to - Connection Manager uses ClientConnection to track active clients and their connection types

Scaling Discussion

Bottlenecks

HTTP server resource exhaustion due to many open SSE connections

Message Broker overload with high notification volume

Connection Manager becoming a single point of failure

Network bandwidth limits for pushing notifications to many clients

Solutions

Use multiple HTTP server instances behind a load balancer with sticky sessions

Employ horizontal scaling and clustering for Message Broker

Distribute Connection Manager state using Redis or consistent hashing

Implement message batching and compression to reduce bandwidth usage

Interview Tips

Time: Spend 10 minutes clarifying requirements and constraints, 20 minutes designing architecture and data flow, 10 minutes discussing scaling and trade-offs, 5 minutes summarizing.

Explain difference between Long Polling and Server-Sent Events clearly

Discuss fallback strategy and client compatibility

Highlight connection management and reconnection handling

Address scalability challenges and solutions

Mention latency and availability targets and how design meets them

Practice

1. What is the main difference between long polling and Server-Sent Events (SSE)?

easy

A. Long polling closes the connection after each response; SSE keeps the connection open.

B. Long polling uses WebSockets; SSE uses HTTP/2 only.

C. Long polling requires client to open multiple connections; SSE uses multiple connections.

D. Long polling pushes data continuously; SSE waits for client requests.

Solution

1. Step 1: Understand long polling behavior

   Long polling keeps the HTTP request open until the server has data, then closes it after sending the response.

2. Step 2: Understand SSE behavior

   SSE keeps a single HTTP connection open to continuously push updates without closing it.

3. Final Answer:

   Long polling closes the connection after each response; SSE keeps the connection open. -> Option A

4. Quick Check:

   Connection persistence = A [OK]

Hint: Remember: long polling reconnects, SSE stays connected [OK]

Common Mistakes:

• Confusing SSE with WebSockets
• Thinking SSE uses multiple connections
• Believing long polling pushes continuously

2. Which HTTP method is typically used by clients to initiate a Server-Sent Events connection?

easy

A. POST

B. GET

C. PUT

D. DELETE

Solution

1. Step 1: Identify HTTP method for SSE

   SSE connections are initiated by clients using the GET method to request a stream from the server.

2. Step 2: Confirm other methods are not used

   POST, PUT, DELETE are used for data modification, not for opening SSE streams.

3. Final Answer:

   GET -> Option B

4. Quick Check:

   SSE connection method = GET [OK]

Hint: SSE uses GET to open a streaming connection [OK]

Common Mistakes:

• Choosing POST because it sends data
• Confusing PUT or DELETE with SSE initiation
• Thinking SSE uses WebSocket handshake methods

3. Consider this simplified server behavior for long polling:

while True:
  wait_for_event()
  send_response_to_client()
  client_reconnects()

What is the main downside of this approach compared to SSE?

medium

A. It cannot send data to multiple clients simultaneously.

B. It keeps the connection open indefinitely, wasting server resources.

C. It uses UDP instead of TCP, causing unreliable delivery.

D. It requires the client to repeatedly reconnect, causing overhead.

Solution

1. Step 1: Analyze long polling loop

   The server waits for an event, sends a response, then the client reconnects to wait again.

2. Step 2: Identify overhead caused by reconnects

   Each reconnect requires new HTTP handshake, increasing latency and server load.

3. Final Answer:

   It requires the client to repeatedly reconnect, causing overhead. -> Option D

4. Quick Check:

   Repeated reconnects = overhead [OK]

Hint: Long polling reconnects cause overhead, SSE keeps one open [OK]

Common Mistakes:

• Thinking long polling keeps connection open indefinitely
• Confusing TCP/UDP protocols in HTTP context
• Assuming long polling can't handle multiple clients

4. A developer implements SSE but notices the client stops receiving updates after some time. What is a likely cause?

medium

A. The client is not reconnecting after each message.

B. The client is sending multiple GET requests simultaneously.

C. The server is closing the connection prematurely.

D. The server is using POST instead of GET for SSE.

Solution

1. Step 1: Understand SSE connection behavior

   SSE expects the server to keep the connection open to push updates continuously.

2. Step 2: Identify issue with connection closing

   If the server closes the connection early, the client stops receiving updates until it reconnects.

3. Final Answer:

   The server is closing the connection prematurely. -> Option C

4. Quick Check:

   Connection closed early = no updates [OK]

Hint: SSE needs server to keep connection open, not close early [OK]

Common Mistakes:

• Thinking client must reconnect after every message
• Confusing HTTP methods for SSE
• Assuming multiple GET requests improve SSE

5. You are designing a chat app that must support thousands of users receiving real-time messages. Which approach is best and why?
A) Use long polling for all clients.
B) Use Server-Sent Events for all clients.
C) Use WebSockets for all clients.
D) Use HTTP polling every 5 seconds.

hard

A. WebSockets provide full-duplex communication, best for chat scalability.

B. Long polling scales well and reduces server load compared to SSE.

C. SSE keeps one connection per client open, efficiently pushing updates.

D. HTTP polling every 5 seconds is the most real-time and efficient.

Solution

1. Step 1: Evaluate long polling and SSE for scale

   Long polling causes overhead reconnects; SSE keeps connections open but is unidirectional.

2. Step 2: Consider WebSockets for chat

   WebSockets allow two-way communication with a single persistent connection, ideal for chat apps.

3. Step 3: Assess HTTP polling

   Polling every 5 seconds causes high latency and unnecessary load.

4. Final Answer:

   WebSockets provide full-duplex communication, best for chat scalability. -> Option A

5. Quick Check:

   Best real-time chat = WebSockets [OK]

Hint: Chat needs two-way, low-latency: choose WebSockets [OK]

Common Mistakes:

• Choosing SSE for two-way chat communication
• Thinking polling is efficient for real-time
• Assuming long polling scales better than WebSockets