Why messaging requires real-time architecture in HLD - Design It to Understand It

Design: Real-Time Messaging System

Design focuses on real-time message delivery architecture and core messaging features. Out of scope: advanced encryption, multimedia message processing, and offline message push notifications.

Functional Requirements

FR1: Deliver messages instantly between users

FR2: Support 100,000 concurrent users

FR3: Ensure message order is preserved

FR4: Provide message delivery acknowledgments

FR5: Handle user presence status (online/offline)

FR6: Allow message history retrieval

Non-Functional Requirements

NFR1: Latency for message delivery must be under 200ms (p99)

NFR2: System availability must be 99.9%

NFR3: Support mobile and web clients

NFR4: Scale to handle peak loads without message loss

Think Before You Design

Questions to Ask

❓ Question 1

❓ Question 2

❓ Question 3

❓ Question 4

❓ Question 5

❓ Question 6

Key Components

WebSocket or persistent connection servers

Message broker or queue system

User presence service

Database for message storage

API gateway for client communication

Load balancers

Design Patterns

Publish-Subscribe pattern

Event-driven architecture

Message queue with acknowledgments

Caching for presence and recent messages

Horizontal scaling with stateless servers

Reference Architecture

Client (Web/Mobile)
    |
    | WebSocket Connection
    v
Load Balancer
    |
Real-Time Messaging Servers (handle connections, presence)
    |
Message Broker (e.g., Redis Pub/Sub or Kafka)
    |
Database (for message persistence and history)

Additional: Presence Service <-> Messaging Servers

Components

Client

WebSocket-enabled web/mobile apps

Send and receive messages in real-time

Load Balancer

Nginx or cloud LB

Distribute client connections evenly to servers

Real-Time Messaging Servers

Node.js or Go servers with WebSocket support

Maintain persistent connections, route messages, manage presence

Message Broker

Redis Pub/Sub or Kafka

Decouple message sending and delivery, enable scalable message distribution

Database

PostgreSQL or Cassandra

Store message history and user data

Presence Service

In-memory store like Redis

Track user online/offline status

Request Flow

1. Client opens WebSocket connection to messaging server via load balancer.

2. Client sends message to messaging server.

3. Messaging server publishes message to message broker.

4. Message broker distributes message to all relevant messaging servers.

5. Messaging servers push message to connected clients in real-time.

6. Messaging servers store message in database asynchronously.

7. Presence service updates user status and informs messaging servers.

Database Schema

Entities: - User (user_id PK, username, status) - Message (message_id PK, sender_id FK, receiver_id FK, content, timestamp, status) - Conversation (conversation_id PK, participant_ids) Relationships: - User to Message: One-to-many (a user sends many messages) - Conversation to Message: One-to-many (a conversation has many messages) - User to Conversation: Many-to-many (users participate in multiple conversations)

Scaling Discussion

Bottlenecks

Messaging servers overwhelmed by too many concurrent WebSocket connections

Message broker saturation under high message throughput

Database write latency for storing messages

Presence service becoming a single point of failure

Solutions

Scale messaging servers horizontally with stateless design and sticky sessions or shared session store

Use partitioned or clustered message brokers to distribute load

Implement asynchronous batch writes or use NoSQL databases optimized for writes

Deploy presence service in a distributed, replicated manner with failover

Interview Tips

Time: Spend 10 minutes understanding requirements and clarifying constraints, 20 minutes designing architecture and data flow, 10 minutes discussing scaling and trade-offs, 5 minutes summarizing.

Explain why low latency and persistent connections are critical for messaging

Discuss choice of WebSocket for real-time bidirectional communication

Highlight decoupling with message broker for scalability

Mention data consistency and message ordering challenges

Address user presence tracking importance

Talk about scaling strategies and fault tolerance