Why messaging requires real-time architecture in HLD - Design It to Understand It

Design: Real-Time Messaging System

Design focuses on real-time message delivery architecture and core messaging features. Out of scope: advanced encryption, multimedia message processing, and offline message push notifications.

Functional Requirements

FR1: Deliver messages instantly between users

FR2: Support 100,000 concurrent users

FR3: Ensure message order is preserved

FR4: Provide message delivery acknowledgments

FR5: Handle user presence status (online/offline)

FR6: Allow message history retrieval

Non-Functional Requirements

NFR1: Latency for message delivery must be under 200ms (p99)

NFR2: System availability must be 99.9%

NFR3: Support mobile and web clients

NFR4: Scale to handle peak loads without message loss

Think Before You Design

Questions to Ask

❓ Question 1

❓ Question 2

❓ Question 3

❓ Question 4

❓ Question 5

❓ Question 6

Key Components

WebSocket or persistent connection servers

Message broker or queue system

User presence service

Database for message storage

API gateway for client communication

Load balancers

Design Patterns

Publish-Subscribe pattern

Event-driven architecture

Message queue with acknowledgments

Caching for presence and recent messages

Horizontal scaling with stateless servers

Reference Architecture

Client (Web/Mobile)
    |
    | WebSocket Connection
    v
Load Balancer
    |
Real-Time Messaging Servers (handle connections, presence)
    |
Message Broker (e.g., Redis Pub/Sub or Kafka)
    |
Database (for message persistence and history)

Additional: Presence Service <-> Messaging Servers

Components

Client

WebSocket-enabled web/mobile apps

Send and receive messages in real-time

Load Balancer

Nginx or cloud LB

Distribute client connections evenly to servers

Real-Time Messaging Servers

Node.js or Go servers with WebSocket support

Maintain persistent connections, route messages, manage presence

Message Broker

Redis Pub/Sub or Kafka

Decouple message sending and delivery, enable scalable message distribution

Database

PostgreSQL or Cassandra

Store message history and user data

Presence Service

In-memory store like Redis

Track user online/offline status

Request Flow

1. Client opens WebSocket connection to messaging server via load balancer.

2. Client sends message to messaging server.

3. Messaging server publishes message to message broker.

4. Message broker distributes message to all relevant messaging servers.

5. Messaging servers push message to connected clients in real-time.

6. Messaging servers store message in database asynchronously.

7. Presence service updates user status and informs messaging servers.

Database Schema

Entities: - User (user_id PK, username, status) - Message (message_id PK, sender_id FK, receiver_id FK, content, timestamp, status) - Conversation (conversation_id PK, participant_ids) Relationships: - User to Message: One-to-many (a user sends many messages) - Conversation to Message: One-to-many (a conversation has many messages) - User to Conversation: Many-to-many (users participate in multiple conversations)

Scaling Discussion

Bottlenecks

Messaging servers overwhelmed by too many concurrent WebSocket connections

Message broker saturation under high message throughput

Database write latency for storing messages

Presence service becoming a single point of failure

Solutions

Scale messaging servers horizontally with stateless design and sticky sessions or shared session store

Use partitioned or clustered message brokers to distribute load

Implement asynchronous batch writes or use NoSQL databases optimized for writes

Deploy presence service in a distributed, replicated manner with failover

Interview Tips

Time: Spend 10 minutes understanding requirements and clarifying constraints, 20 minutes designing architecture and data flow, 10 minutes discussing scaling and trade-offs, 5 minutes summarizing.

Explain why low latency and persistent connections are critical for messaging

Discuss choice of WebSocket for real-time bidirectional communication

Highlight decoupling with message broker for scalability

Mention data consistency and message ordering challenges

Address user presence tracking importance

Talk about scaling strategies and fault tolerance

Practice

1. Why is real-time architecture important for messaging systems?

easy

A. It delivers messages instantly for smooth conversations.

B. It stores messages for long-term archival only.

C. It processes messages in batches once a day.

D. It encrypts messages without sending them.

Solution

1. Step 1: Understand messaging user experience

   Users expect messages to appear immediately during chats for natural flow.

2. Step 2: Connect real-time architecture to instant delivery

   Real-time systems use open connections to send messages instantly without delay.

3. Final Answer:

   It delivers messages instantly for smooth conversations. -> Option A

4. Quick Check:

   Instant delivery = Real-time architecture [OK]

Hint: Real-time means instant message delivery [OK]

Common Mistakes:

• Confusing real-time with batch processing
• Thinking real-time only means encryption
• Assuming real-time stores messages only

2. Which technology is commonly used to maintain open connections for real-time messaging?

easy

A. WebSockets for continuous connection

B. FTP for file transfers

C. HTTP polling every hour

D. SMTP for email delivery

Solution

1. Step 1: Identify open connection methods

   Real-time messaging needs a persistent connection to avoid delays.

2. Step 2: Match technology to persistent connection

   WebSockets keep a continuous open connection, unlike HTTP polling or FTP.

3. Final Answer:

   WebSockets for continuous connection -> Option A

4. Quick Check:

   Open connection = WebSockets [OK]

Hint: WebSockets keep connections open for real-time [OK]

Common Mistakes:

• Choosing HTTP polling which is slow
• Confusing FTP or SMTP with messaging protocols
• Thinking email protocols support real-time chat

3. Consider a messaging system using WebSockets. What happens if the server delays message delivery by 5 seconds?

medium

A. Messages are delivered in batches every minute.

B. Users see messages instantly without delay.

C. Messages get lost and never arrive.

D. Messages appear late, causing poor chat experience.

Solution

1. Step 1: Understand WebSocket behavior

   WebSockets deliver messages instantly if server sends immediately.

2. Step 2: Analyze impact of 5-second delay

   If server delays sending, users see messages late, hurting chat flow.

3. Final Answer:

   Messages appear late, causing poor chat experience. -> Option D

4. Quick Check:

   Server delay = Late messages [OK]

Hint: Server delay causes late message display [OK]

Common Mistakes:

• Assuming WebSockets fix server delays
• Thinking messages get lost due to delay
• Confusing batch delivery with real-time

4. A messaging system uses WebSockets but users report delayed messages. What is the most likely cause?

medium

A. WebSocket protocol does not support real-time.

B. Server processes messages slowly before sending.

C. Clients do not support HTML5.

D. Messages are encrypted end-to-end.

Solution

1. Step 1: Check WebSocket capabilities

   WebSockets support real-time delivery if server sends promptly.

2. Step 2: Identify delay source

   Slow server processing before sending causes message delays, not WebSocket itself.

3. Final Answer:

   Server processes messages slowly before sending. -> Option B

4. Quick Check:

   Slow server = Delayed messages [OK]

Hint: Delays usually come from server processing, not WebSocket [OK]

Common Mistakes:

• Blaming WebSocket protocol for delays
• Assuming client HTML5 support causes delay
• Confusing encryption with delivery speed

5. You design a messaging app for millions of users. Which real-time architecture choice best supports fast, reliable message delivery at scale?

hard

A. Store messages locally on user devices only.

B. Send messages via email every hour.

C. Use WebSockets with load balancers and message queues.

D. Use HTTP polling every 10 minutes.

Solution

1. Step 1: Identify scalable real-time components

   WebSockets provide instant delivery; load balancers distribute traffic; queues ensure reliability.

2. Step 2: Compare other options for scale and speed

   Email and polling are slow; local storage alone can't deliver messages to others.

3. Final Answer:

   Use WebSockets with load balancers and message queues. -> Option C

4. Quick Check:

   Scalable real-time = WebSockets + load balancers + queues [OK]

Hint: Combine WebSockets, load balancers, and queues for scale [OK]

Common Mistakes:

• Choosing slow batch methods for real-time needs
• Ignoring load balancing for millions of users
• Relying only on local device storage