Bird
Raised Fist0

Design WhatsApp - Real-Time Chat, Delivery Receipts, End-to-End Encryption

HLDhardInterview freq: high
Choose your preparation mode4 modes available
ArchitectureTraceChallenge
🎯
Design WhatsApp - Real-Time Chat, Delivery Receipts, End-to-End Encryption
45 min · senior
senior

You are tasked with designing a WhatsApp-like messaging system serving 50 million daily active users (DAU). The system must support real-time chat with delivery receipts, end-to-end encryption (E2EE), and offline message delivery. The P99 latency for message delivery should be under 200ms. Users expect messages to be delivered in order and reliably, even if they are offline for extended periods. The system must also handle group chats and support message synchronization on reconnect.

Constraints
  • End-to-end encryption must ensure that only sender and recipient(s) can read messages; servers cannot decrypt message content.
  • Offline message delivery with guaranteed eventual consistency and ordering when users reconnect.
  • P99 message delivery latency under 200ms for online users.
  • Support for group chats with up to 256 participants.
  • Delivery receipts (sent, delivered, read) must be real-time and consistent.
  • System must scale to 50M DAU with peak message throughput estimated at 10,000 messages per second.
📋
Guided Steps
1/5
Step 1
Message Storage and Ordering
How will you store messages to support ordered delivery, offline sync, and scale to 50M DAU?
💡 Hint
Consider a distributed database optimized for high write throughput and range queries by conversation.
Key Decision
Use Apache Cassandra with partition_key=conversation_id to ensure all messages in a conversation are stored on the same node, enabling efficient range scans for cursor-based sync and ordered delivery.
Interviewer Expectation
Mid-level: Names Cassandra for scalability. Senior: Explains why conversation_id is the partition key to keep messages ordered and why sender_id partitioning fails due to out-of-order delivery and cross-node coordination.
⚠️Common Wrong Answer
Using a relational database with sender_id as partition key leads to cross-shard queries and out-of-order message delivery, causing high latency and inconsistent sync.
Step 2
Real-Time Message Delivery and Offline Support
How will you deliver messages in real-time and support offline users with eventual delivery guarantees?
💡 Hint
Think about a distributed messaging system that preserves order per conversation and supports replay for offline clients.
Key Decision
Use Apache Kafka with conversation_id as the partition key to guarantee ordered message streams per conversation; consumers (delivery workers) read from Kafka and push messages to online clients or store for offline delivery.
Interviewer Expectation
Mid-level: Mentions Kafka for messaging. Senior: Explains why conversation_id as partition key ensures ordering and how Kafka's retention and replay enable offline delivery and sync.
⚠️Common Wrong Answer
Using RabbitMQ without partitioning causes unordered delivery and poor scalability at 50M DAU due to queue contention.
Step 3
End-to-End Encryption Key Management
How will you implement end-to-end encryption ensuring servers cannot decrypt messages, while supporting group chats and offline delivery?
💡 Hint
Consider the Signal Protocol for key exchange and message encryption, and how to handle offline key distribution.
Key Decision
Implement Signal Protocol for E2EE with per-device identity keys and ephemeral session keys; use a dedicated Key Management Service (KMS) to store public keys and pre-keys, enabling offline message encryption without server access to private keys.
Interviewer Expectation
Mid-level: Names Signal Protocol for E2EE. Senior: Details how pre-keys enable offline encryption, how group chats use sender keys, and how key rotation and device management are handled securely.
⚠️Common Wrong Answer
Encrypting messages only at transport layer (TLS) exposes plaintext to servers, violating E2EE requirements.
Step 4
Delivery Receipts and Read Acknowledgments
How will you implement real-time delivery and read receipts that scale and remain consistent?
💡 Hint
Consider a low-latency pub/sub system or in-memory store to track message states and notify clients.
Key Decision
Use Redis Streams with conversation_id as stream key to track delivery/read receipts in order; delivery workers update Redis and push notifications via WebSocket to clients for real-time updates.
Interviewer Expectation
Mid-level: Suggests Redis for caching receipts. Senior: Explains Redis Streams ordering guarantees, TTL for offline detection, and how receipt updates are idempotent and consistent across devices.
⚠️Common Wrong Answer
Polling the server for receipt status causes high load and latency spikes, collapsing under 50M DAU.
Step 5
Client-Server Communication Protocol
What communication protocol will you use between clients and servers to support real-time messaging and low latency?
💡 Hint
Consider persistent connections that allow bidirectional communication and low overhead.
Key Decision
Use WebSocket connections for persistent, bidirectional communication enabling real-time message push and receipt acknowledgments with P99 latency under 200ms.
Interviewer Expectation
Mid-level: Mentions WebSocket for real-time. Senior: Explains connection management, heartbeat mechanisms, reconnection strategies, and fallback options for unreliable networks.
⚠️Common Wrong Answer
HTTP polling causes excessive reconnections and high latency, overwhelming servers at scale.
🔌
API Design Challenge
Sketch 3 core API endpoints for the chat system: sending a message, syncing messages on reconnect, and acknowledging message delivery/read.
Hints
  • How does the client send an encrypted message to the server?
  • How does the client efficiently sync missed messages after offline periods?
  • How does the client notify the server about message delivery and read receipts?
📊
Scoring Rubric
▼ Show
Must Have - 60%
  • Use of conversation_id as partition key in Cassandra for ordered message storage and efficient sync.
  • Kafka with conversation_id partitioning for ordered real-time message streaming and offline replay.
  • Signal Protocol-based end-to-end encryption with offline pre-key support.
  • WebSocket for real-time bidirectional communication with low latency.
  • Delivery receipts implemented with Redis Streams ensuring ordering and consistency.
Good to Have - 30%
  • Dedicated Key Management Service for secure public key storage and distribution.
  • Heartbeat mechanism for online/offline detection and offline message sync.
Bonus - 10%
  • Optimizations for group chat message encryption and delivery.
  • Advanced reconnection and fallback strategies for unreliable networks.
Anti-patterns to Avoid
✅
Model Answer
▼ Reveal

This design outlines a WhatsApp-like real-time chat system supporting 50M DAU with low latency, offline delivery, and strong E2EE. It uses Cassandra for ordered message storage, Kafka for ordered real-time streaming and offline replay, Signal Protocol for encryption, Redis Streams for delivery receipts, and WebSocket for client-server communication.

Key Decisions
  • Use conversation_id as partition key in Cassandra to keep messages ordered and enable efficient range queries for offline sync.
  • Partition Kafka topics by conversation_id to guarantee ordered message streams and enable replay for offline clients.
  • Implement Signal Protocol with per-device identity keys and pre-keys to enable E2EE and offline encryption without server access to private keys.
  • Use Redis Streams keyed by conversation_id to track delivery and read receipts in order, ensuring consistency and low latency.
  • Use WebSocket connections for persistent, bidirectional communication supporting real-time message delivery and receipt acknowledgments with P99 latency under 200ms.
  • Implement heartbeat mechanism using Redis TTL keys to detect client online/offline status and trigger offline message sync.
  • Delivery workers handle message retries and offline storage to guarantee eventual delivery and ordering.
  • conversation_id as partition key in Apache Cassandra for ordered, scalable message storage and efficient offline sync.
  • Apache Kafka with conversation_id partitioning to guarantee ordered real-time message streams and enable offline replay.
  • Signal Protocol for end-to-end encryption with pre-key bundles enabling offline message encryption without server access to private keys.
  • Redis Streams for ordered, low-latency delivery and read receipt tracking with TTL-based offline detection.
  • WebSocket protocol for persistent, bidirectional client-server communication supporting real-time message delivery and acknowledgments.
  • Dedicated Key Management Service (KMS) to store and distribute public keys securely for E2EE without exposing private keys.
  • Delivery workers consuming Kafka partitions to push messages to online clients or store for offline delivery, ensuring ordered delivery and retries.
  • Heartbeat mechanism via Redis TTL keys to detect online/offline client status and trigger offline message sync.

Practice

(1/5)
1. A search ranking system must store and serve feature vectors for millions of users with low latency. Which component is best suited to provide fast, consistent feature retrieval at scale?
easy
A. A relational database with complex joins to compute features on demand
B. A distributed in-memory feature store with precomputed feature vectors
C. A batch processing system that recomputes features daily and stores them in object storage
D. A key-value store without versioning or consistency guarantees

Solution

  1. Step 1: Identify feature retrieval requirements

    Low latency and consistency for millions of users require fast, precomputed feature access.

  2. Step 2: Evaluate component suitability

    A relational DB with joins (A) is too slow for real-time; batch recomputation (C) is stale; key-value store without consistency (D) risks stale or inconsistent features. Distributed in-memory feature store (B) provides fast, consistent retrieval.

  3. Final Answer:

    Option B -> Option B

  4. Quick Check:

    In-memory feature store -> low latency, consistent features [check]
Hint: Precompute features in-memory for fast, consistent retrieval [check]
Common Mistakes:
2. In a collaborative editing system, at step 4 of the request flow, the OT engine receives two concurrent operations from users Alice and Bob. What does the OT engine do at this step to maintain document consistency?
easy
A. Transforms Bob's operation against Alice's operation to preserve intention and order
B. Rejects Bob's operation and requests a retry to avoid conflicts
C. Queues Bob's operation until Alice's operation is fully applied to the document
D. Applies Bob's operation directly, overwriting Alice's changes

Solution

  1. Step 1: Understand OT engine's role at concurrent operation receipt

    The OT engine must handle concurrent operations arriving out of order or simultaneously and merge them without losing user intent.

  2. Step 2: Recognize OT transformation process

    OT transforms one operation against another to adjust positions and effects, preserving both users' intentions and maintaining consistency.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    OT transforms concurrent ops to preserve intention and order [check]
Hint: OT transforms concurrent ops to preserve intention and order [OK]
Common Mistakes:
3. A distributed system needs to monitor request latency and error rates in near real-time to trigger alerts. Which observability component is best suited to handle this requirement?
easy
A. A batch job that processes logs nightly to generate reports
B. Centralized logging system that aggregates all logs for offline analysis
C. Distributed tracing system that records end-to-end request paths for debugging
D. Metrics collection system that aggregates and visualizes numerical data over time

Solution

  1. Step 1: Identify the need for near real-time numerical data aggregation

    Metrics systems collect numerical data like latency and error rates continuously and support alerting.

  2. Step 2: Eliminate logging and tracing for this use case

    Logging is better for detailed event records, not real-time aggregation; tracing is for request path debugging, not metrics aggregation.

  3. Final Answer:

    Option D -> Option D

  4. Quick Check:

    Metrics aggregate numerical data for alerts and dashboards [check]
Hint: Metrics track numbers over time for alerts [check]
Common Mistakes:
4. A search engine must quickly update document scores as new documents arrive and queries are frequent. Which component is best suited to efficiently compute term frequency and inverse document frequency in real-time?
easy
A. An in-memory inverted index with incremental TF-IDF updates
B. A relational database with full-text indexing
C. A batch processing system that recomputes TF-IDF nightly
D. A simple key-value store without indexing capabilities

Solution

  1. Step 1: Identify real-time scoring needs

    Relational DBs with full-text indexing (A) are slower for frequent updates; batch recomputation (C) is too delayed for real-time queries.

  2. Step 2: Evaluate indexing and update efficiency

    In-memory inverted index with incremental TF-IDF updates (B) supports fast updates and queries; key-value stores (D) lack indexing for relevance scoring.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    In-memory inverted index enables fast incremental TF-IDF updates [check]
Hint: In-memory inverted index enables fast TF-IDF updates [check]
Common Mistakes:
5. If the GraphQL server's query complexity analysis component is removed, what failure occurs when clients send deeply nested queries?
medium
A. The server processes deeply nested queries causing high CPU and memory usage, leading to degraded performance or crashes
B. The server rejects all queries immediately due to missing validation
C. Clients receive partial data with missing nested fields silently
D. The server caches all queries indefinitely, causing storage exhaustion

Solution

  1. Step 1: Understand query complexity analysis role

    This component limits query depth and cost to prevent resource exhaustion.

  2. Step 2: Trace failure without complexity analysis

    Without it, malicious or accidental deeply nested queries consume excessive CPU/memory, degrading or crashing the server.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Missing complexity check -> resource exhaustion from deep queries [check]
Hint: No complexity check -> server overload from deep queries [check]
Common Mistakes:
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard