Bird
Raised Fist0

Design WhatsApp - Real-Time Chat, Delivery Receipts, End-to-End Encryption

HLDhardInterview freq: high
Choose your preparation mode4 modes available
ArchitectureTraceChallenge
🏗️
Design WhatsApp - Real-Time Chat, Delivery Receipts, End-to-End Encryption
Design a highly scalable, real-time encrypted chat system with delivery receipts for 2B DAU
hardCommunication and Messaging SystemsFreq: high
Prerequisites:WebSocketsKafkaCassandra
⚙️
Functional Requirements
  • Real-time one-to-one and group chat messaging with message ordering guarantees
  • Delivery receipts: sent, delivered, read status updates per message
  • End-to-end encryption ensuring only sender and receiver(s) can read messages
  • Offline message storage and sync when users reconnect
  • Presence indication (online/offline status) for contacts
  • Message deletion and edit support (optional but common)
📊
Non-Functional Requirements
  • Support 2 billion daily active users (DAU)
  • Handle peak write throughput of ~925K messages per second
  • 99.99% system availability with multi-region failover
  • End-to-end latency under 100ms P99 for message delivery
  • Data retention for at least 30 days with encrypted storage
❓
Clarifying Questions to Ask
  • What is the expected average message size and frequency per user?
  • Are group chats supported and what is the max group size?
  • What encryption protocol is preferred or mandated?
  • What is the expected offline message sync behavior and limits?
  • Is message ordering strict or eventual consistency acceptable?
  • Are delivery receipts required for every message or optional?
Out of Scope
✗ Voice and video calling features
✗ Media (images, videos) storage and delivery
✗ User account management and authentication flows
✗ Spam detection and moderation
📌
Assumptions
  • 2 billion DAU
  • Average 40 messages sent per user per day
  • Average message size 100 bytes
  • Peak traffic concentrated in 4 hours daily (peak factor 4)
⚡
QPS Calculation
write qps2B × 40 / 86400 ≈ 925,926 msg/sec
peak write qps925,926 × 4 ≈ 3.7M msg/sec
read qpsAssuming 1.5 reads per write (delivery receipts + fetch), 1.5 × 925,926 ≈ 1.39M reads/sec
💾
Storage Estimate
daily storage2B × 40 × 100 bytes = 8 TB/day
retention 30 days8 TB × 30 = 240 TB
with replication factor 3240 TB × 3 = 720 TB total storage
🖥️
Server Estimate
message ingestionKafka cluster with 1000 brokers to handle 3.7M msg/sec peak writes
storage nodesCassandra cluster with 500 nodes to store 720 TB with replication and handle 1.4M reads/sec
🏗️
Architecture Diagram
Architecture Diagram
User Device
    |
WebSocket Gateway (Nginx + Envoy)
    |
Message Broker (Kafka Cluster)
    |
Message Processor (Stateless Microservices)
    |
Cassandra Storage Cluster
    |
Notification Service (Push + Delivery Receipts)
    |
Presence Service (Redis Cluster)
    |
Key Management Service (HSM + Signal Protocol)
🧩
Components
WebSocket Gateway
Maintain persistent WebSocket connections for real-time bidirectional messaging
Why thisNginx + Envoy supports millions of concurrent connections with low latency and TLS termination
Why notLong polling HTTP servers like Apache fail to scale to millions of concurrent connections efficiently
Kafka Cluster
Durable, ordered, partitioned message queue for ingesting and distributing chat messages
Why thisKafka scales horizontally to millions of writes/sec with strong ordering guarantees per partition
Why notRabbitMQ lacks horizontal partitioning and throughput at 3.7M msg/sec peak
Message Processor
Stateless microservices that decrypt, validate, enrich, and route messages to storage and notifications
Why thisStateless design enables easy horizontal scaling and fault isolation
Why notMonolithic services create bottlenecks and complicate scaling at 2B DAU
Cassandra Storage Cluster
Store encrypted messages with high write throughput and multi-region replication
Why thisCassandra's wide-column model supports time-series data with tunable consistency and linear write scalability
Why notPostgreSQL requires complex manual sharding at 3.7M writes/sec; B-tree writes don't scale linearly
Notification Service
Send push notifications and delivery receipt updates to user devices asynchronously
Why thisKafka-backed microservices with APNs/FCM integration handle millions of notifications reliably
Why notPolling-based notification systems increase latency and server load
Presence Service
Track and serve user online/offline presence with low latency
Why thisRedis Cluster with TTL keys natively expires stale presence without cron jobs - sub-millisecond lookup
Why notCassandra has high write latency and is unsuitable for ephemeral presence data
Key Management Service
Manage end-to-end encryption keys using Signal Protocol and Hardware Security Modules (HSMs)
Why thisSignal Protocol Double Ratchet ensures forward secrecy and post-compromise security; HSMs protect private keys
Why notCentralized key storage without HSMs risks key leakage and weakens E2E guarantees
🔄
Data Flow
1
User Device → WebSocket Gateway
Send encrypted message over WebSocket
Low-latency persistent connection for real-time delivery
2
WebSocket Gateway → Kafka Cluster
Publish encrypted message to Kafka topic partition
Kafka ensures ordered, durable ingestion at scale
3
Kafka Cluster → Message Processor
Consume message, validate, decrypt metadata, enrich with delivery info
Stateless processing enables horizontal scaling
4
Message Processor → Cassandra Storage Cluster
Write encrypted message with metadata and timestamps
Wide-column store optimized for high write throughput
5
Message Processor → Notification Service
Trigger push notifications and delivery receipt updates
Asynchronous notification to recipient devices
6
Notification Service → User Device
Deliver push notification and update delivery status
Ensures user is notified even if offline
7
User Device → Presence Service
Update online/offline presence with TTL
Presence data expires automatically to avoid stale states
8
User Device → Key Management Service
Exchange and rotate encryption keys securely
Signal Protocol Double Ratchet key updates for E2E encryption

Practice

(1/5)
1. In a system trace, at Step 3 the Storage Estimator component calculates daily storage needs. If the average object size is 2MB and the system expects 500K new objects daily, what is the storage estimate at Step 3?
easy
A. Cannot estimate without knowing retention period
B. 1TB per day, calculated as 500K objects x 2MB each
C. 2TB per day, assuming 4MB average object size instead
D. 500GB per day, assuming 1MB average object size instead

Solution

  1. Step 1: Calculate daily storage

    Multiply 500,000 objects by 2MB each = 1,000,000 MB = 1TB per day.

  2. Step 2: Eliminate other options

    500GB per day, assuming 1MB average object size instead uses wrong object size; 2TB per day, assuming 4MB average object size instead doubles size incorrectly; Cannot estimate without knowing retention period ignores that retention affects total storage, not daily estimate.

  3. Final Answer:

    Option B -> Option B

  4. Quick Check:

    500K x 2MB = 1TB daily storage [check]
Hint: Daily storage = object count x average size [check]
Common Mistakes:
2. If the Bloom filter's hash functions are not independent and produce correlated outputs, what is the exact failure mode observed?
medium
A. The bit array will saturate quickly causing excessive false positives and reduced filter effectiveness
B. The filter will have false negatives, incorrectly rejecting existing members
C. The false positive rate increases beyond the expected theoretical rate
D. Insertions will fail silently without updating the bit array

Solution

  1. Step 1: Understand hash function independence role

    Independent hash functions distribute bits uniformly; correlated hashes cause clustering.

  2. Step 2: Trace effect of correlated hashes

    Correlated hashes set overlapping bits, saturating the bit array faster, increasing false positives and reducing filter effectiveness.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Correlated hashes cause bit array saturation, raising false positives [check]
Hint: Correlated hashes cause bit array saturation and false positives [check]
Common Mistakes:
3. For the For-You feed algorithm, which architecture best balances freshness of recommendations with scalability when serving millions of users?
medium
A. Use a monolithic service that recomputes recommendations synchronously on every user request
B. Use a client-side recommendation engine to reduce server load
C. Use an offline batch job to precompute recommendations periodically and serve from cache
D. Use a real-time streaming pipeline to update recommendations instantly per user action

Solution

  1. Step 1: Evaluate synchronous recomputation

    Monolithic synchronous (A) causes high latency and poor scalability.

  2. Step 2: Compare batch vs streaming

    Streaming (C) offers freshness but is complex and costly at scale; client-side (D) lacks data for quality recommendations.

  3. Step 3: Select batch precompute

    Batch jobs (B) balance freshness and scalability by precomputing and caching recommendations.

  4. Final Answer:

    Option C -> Option C

  5. Quick Check:

    Batch precompute caches recommendations for scalable freshness [check]
Hint: Batch precompute balances freshness and scale best [OK]
Common Mistakes:
4. If the shared database in a microservices architecture is unavailable, what is the immediate failure impact on services that rely on it?
medium
A. Services queue requests internally and retry database operations asynchronously
B. Services block on database calls, causing request timeouts and degraded user experience
C. API Gateway reroutes requests to backup services without database dependency
D. Services automatically switch to local caches and continue operating normally

Solution

  1. Step 1: Understand failure impact of shared database outage

    Services depending on the shared DB cannot proceed with reads/writes, causing blocking or errors.

  2. Step 2: Evaluate fallback mechanisms

    Some microservices implement internal queuing and retry logic to handle transient DB failures asynchronously, preventing immediate request failures.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Internal queuing and retries mitigate DB outage impact [check]
Hint: Queue and retry internally to survive DB outages [check]
Common Mistakes:
5. At 10x scale, which component in the multi-tenant SaaS platform is most likely to fail first, and why?
hard
A. Feature Flag Service cache, because stale flag data causes widespread inconsistent behavior
B. Billing database, due to write amplification from synchronous updates per tenant request
C. Tenant Metadata Service, as it becomes a centralized bottleneck for tenant context lookups
D. API Gateway, because it maintains stateful WebSocket connections that cannot be easily sharded

Solution

  1. Step 1: Analyze scaling bottlenecks

    Billing DB (A) can be scaled with sharding; Tenant Metadata (C) can be cached; API Gateway (D) handles stateless HTTP mostly.

  2. Step 2: Evaluate Feature Flag Service cache at scale

    Cache invalidation and stale data grow exponentially, causing inconsistent feature exposure and client errors.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Feature flag cache stale -> inconsistent behavior at scale [check]
Hint: Feature flag cache stale -> first scale failure [check]
Common Mistakes:
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard