WebSocket for real-time communication in HLD - System Design Exercise

Design: Real-Time Communication System using WebSocket

Design covers WebSocket server architecture, client connection management, message routing, and scaling strategies. Out of scope are client UI details and persistent message storage.

Functional Requirements

FR1: Support real-time bidirectional communication between clients and server

FR2: Handle up to 50,000 concurrent WebSocket connections

FR3: Deliver messages with p99 latency under 100ms

FR4: Support user authentication and authorization before connection

FR5: Allow broadcasting messages to multiple clients efficiently

FR6: Ensure message ordering and delivery guarantees

FR7: Provide reconnection support for clients after network interruptions

Non-Functional Requirements

NFR1: System must maintain 99.9% uptime (approx. 8.77 hours downtime per year)

NFR2: Scale horizontally to handle increasing number of connections

NFR3: Use industry standard protocols and technologies

NFR4: Secure communication with TLS encryption

NFR5: Minimize server resource usage per connection

Think Before You Design

Questions to Ask

❓ Question 1

❓ Question 2

❓ Question 3

❓ Question 4

❓ Question 5

❓ Question 6

❓ Question 7

Key Components

Load balancer to distribute incoming WebSocket connections

WebSocket server cluster to handle connections

Authentication service for validating users

Message broker or pub/sub system for routing messages

In-memory data store for session and connection state

TLS termination for secure communication

Design Patterns

Publish-Subscribe pattern for message distribution

Sticky sessions or session affinity for connection consistency

Horizontal scaling with stateless WebSocket servers

Backpressure handling to avoid server overload

Heartbeat and ping/pong for connection health checks

Reference Architecture

Client1 ---\
Client2 ----> Load Balancer ---> WebSocket Server Cluster ---> Message Broker ---> Other Servers
ClientN ---/

Authentication Service <--> WebSocket Server Cluster

In-Memory Store <--> WebSocket Server Cluster

TLS Termination at Load Balancer

Components

Load Balancer

Nginx or AWS ALB

Distributes incoming WebSocket connection requests evenly across WebSocket servers and handles TLS termination

WebSocket Server Cluster

Node.js with ws library or similar

Manages WebSocket connections, authenticates users, sends and receives messages

Authentication Service

OAuth2 or JWT based service

Validates user credentials before allowing WebSocket upgrade

Message Broker

Redis Pub/Sub or Apache Kafka

Routes messages between WebSocket servers for broadcasting and private messaging

In-Memory Data Store

Redis

Stores session data, connection metadata, and supports quick lookup for routing

Request Flow

1. Client sends HTTP request to Load Balancer to initiate WebSocket handshake.

2. Load Balancer terminates TLS and forwards request to a WebSocket server.

3. WebSocket server authenticates client using Authentication Service.

4. Upon successful authentication, WebSocket connection is established.

5. Client sends messages over WebSocket to server.

6. WebSocket server publishes messages to Message Broker for distribution.

7. Other WebSocket servers subscribed to Message Broker receive messages and forward to connected clients.

8. Heartbeat messages (ping/pong) maintain connection health.

9. If client disconnects, reconnection attempts are handled with session restoration using In-Memory Data Store.

Database Schema

Entities: - User: user_id (PK), username, password_hash, auth_token - Session: session_id (PK), user_id (FK), connection_id, last_active_timestamp - Message: message_id (PK), sender_user_id (FK), recipient_user_id (nullable), group_id (nullable), content, timestamp Relationships: - User 1:N Session - User 1:N Message (as sender) - Message may be private (recipient_user_id) or broadcast (group_id)

Scaling Discussion

Bottlenecks

WebSocket server CPU and memory limits due to large number of concurrent connections

Load balancer capacity to handle many simultaneous TLS handshakes

Message broker throughput limits when broadcasting to many clients

Network bandwidth constraints for high message volume

Session state synchronization across servers

Solutions

Scale WebSocket servers horizontally and use stateless design with shared session store

Use multiple load balancers with DNS round-robin or cloud provider scaling

Partition message topics in message broker and use sharding

Implement message compression and rate limiting to reduce bandwidth

Use distributed in-memory stores like Redis Cluster for session state replication

Interview Tips

Time: Spend 10 minutes clarifying requirements and constraints, 20 minutes designing architecture and data flow, 10 minutes discussing scaling and trade-offs, 5 minutes summarizing.

Explain why WebSocket is chosen for bidirectional real-time communication

Discuss authentication before WebSocket upgrade for security

Describe how load balancer and server cluster handle scale and availability

Highlight message broker role in decoupling and efficient message routing

Address reconnection and session management for reliability

Mention trade-offs between consistency, latency, and resource usage

Practice

1. What is the primary advantage of using WebSocket over traditional HTTP for real-time communication?

easy

A. It requires no server-side support to function.

B. It uses less bandwidth by compressing all messages automatically.

C. It keeps a persistent connection open for instant two-way data exchange.

D. It guarantees message delivery order without any additional setup.

Solution

1. Step 1: Understand WebSocket connection behavior

   WebSocket establishes a persistent connection between client and server, unlike HTTP which opens and closes connections per request.

2. Step 2: Identify the key benefit for real-time apps

   This persistent connection allows instant two-way communication, ideal for real-time updates.

3. Final Answer:

   It keeps a persistent connection open for instant two-way data exchange. -> Option C

4. Quick Check:

   Persistent connection = Instant two-way communication [OK]

Hint: WebSocket keeps connection open for fast two-way data [OK]

Common Mistakes:

• Confusing WebSocket with HTTP polling
• Assuming WebSocket compresses all messages automatically
• Thinking WebSocket works without server support

2. Which of the following is the correct syntax to open a WebSocket connection in JavaScript?

easy

A. const ws = new WebSocket('http://example.com/socket');

B. const ws = new WebSocket('ws://example.com/socket');

C. const ws = WebSocket.connect('ws://example.com/socket');

D. const ws = WebSocket.open('http://example.com/socket');

Solution

1. Step 1: Recall WebSocket URL scheme

   WebSocket URLs start with ws:// or wss:// for secure connections, not http://.

2. Step 2: Identify correct constructor usage

   The WebSocket object is created with new WebSocket(url), no .connect() or .open() methods exist.

3. Final Answer:

   const ws = new WebSocket('ws://example.com/socket'); -> Option B

4. Quick Check:

   WebSocket uses ws:// + new WebSocket() [OK]

Hint: WebSocket URLs start with ws:// or wss:// [OK]

Common Mistakes:

• Using http:// instead of ws://
• Calling non-existent connect() or open() methods
• Omitting the new keyword

3. Consider this simplified server-side pseudocode handling WebSocket messages:

onMessage(client, message) {
  broadcastToAllClients(message);
}

What will happen if 1000 clients send messages simultaneously?

medium

A. Server may become overwhelmed, causing message delays or dropped connections.

B. All messages will be broadcasted instantly to all clients without delay.

C. Only the first client's message will be broadcasted; others ignored.

D. Clients will automatically queue messages on their side until server is free.

Solution

1. Step 1: Analyze server load with simultaneous messages

   When many clients send messages at once, the server must process and broadcast each, increasing CPU and network load.

2. Step 2: Understand server resource limits

   Without scaling or message queuing, the server can get overwhelmed, causing delays or dropped connections.

3. Final Answer:

   Server may become overwhelmed, causing message delays or dropped connections. -> Option A

4. Quick Check:

   High load without scaling = delays/drops [OK]

Hint: High simultaneous messages can overload server without scaling [OK]

Common Mistakes:

• Assuming instant broadcast without delay
• Thinking clients queue messages automatically
• Believing only first message is processed

4. A developer wrote this WebSocket server snippet:

ws.on('message', (msg) => {
  ws.send(msg);
});

Why might this cause problems in a multi-client chat application?

medium

A. It closes the connection after sending the message.

B. It broadcasts the message to all clients, causing duplicates.

C. It causes syntax errors due to incorrect event name.

D. It only sends the message back to the sender, not to other clients.

Solution

1. Step 1: Understand the code behavior

   The code listens for a message from a client and sends the same message back only to that client (ws.send(msg)).

2. Step 2: Identify issue in multi-client context

   Other connected clients do not receive the message, so chat messages are not broadcasted properly.

3. Final Answer:

   It only sends the message back to the sender, not to other clients. -> Option D

4. Quick Check:

   ws.send(msg) sends only to sender [OK]

Hint: ws.send(msg) sends only to sender, not all clients [OK]

Common Mistakes:

• Assuming ws.send broadcasts to all clients
• Thinking event name 'message' is incorrect
• Believing connection closes after send

5. You are designing a scalable chat app using WebSocket. Which architectural component helps efficiently distribute messages to thousands of clients without overloading the main server?

hard

A. A message broker (e.g., Redis Pub/Sub) between WebSocket servers and clients.

B. A single powerful WebSocket server handling all connections directly.

C. Using HTTP long polling instead of WebSocket for all clients.

D. Storing all messages in a database before sending to clients.

Solution

1. Step 1: Understand scaling challenges with many clients

   A single server can become a bottleneck when handling thousands of WebSocket connections and broadcasting messages.

2. Step 2: Identify role of message broker

   A message broker like Redis Pub/Sub can distribute messages efficiently across multiple WebSocket servers, balancing load and improving scalability.

3. Final Answer:

   A message broker (e.g., Redis Pub/Sub) between WebSocket servers and clients. -> Option A

4. Quick Check:

   Message broker enables scalable message distribution [OK]

Hint: Use message broker to scale WebSocket message distribution [OK]

Common Mistakes:

• Relying on a single server for all connections
• Switching to HTTP long polling loses real-time benefits
• Storing messages in DB adds latency, not scaling