Bird
Raised Fist0

Design Slack - Workspaces, Channels, Real-Time Delivery at Scale, Search

HLDhardInterview freq: high
Choose your preparation mode4 modes available
ArchitectureTraceChallenge
🏗️
Design Slack - Workspaces, Channels, Real-Time Delivery at Scale, Search
Design Slack with scalable workspaces, channels, real-time messaging, and search
hardCommunication and Messaging SystemsFreq: high
Prerequisites:Inverted IndexElasticsearch
⚙️
Functional Requirements
  • Support multiple workspaces, each with multiple channels (public and private).
  • Real-time message delivery with sub-100ms P99 latency to all online members in a channel.
  • Message persistence with history accessible for search and retrieval.
  • Full-text search over messages and files within a workspace using Elasticsearch.
  • User presence and typing indicators updated in real-time.
  • Support for direct messages (DMs) and group DMs.
📊
Non-Functional Requirements
  • Support 2 billion daily active users (DAU) globally.
  • Handle peak write throughput of ~925K messages per second (derived in estimation).
  • Ensure 99.99% availability for messaging and search services.
  • Maintain 100ms P99 latency for message delivery to online users.
  • Data durability with replication and retention of at least 30 days of message history.
❓
Clarifying Questions to Ask
  • What is the expected average message size and message rate per user?
  • Are messages ordered strictly or eventually consistent ordering acceptable?
  • What is the retention period for messages and search indices?
  • Should offline users receive push notifications or only sync on reconnect?
  • Is end-to-end encryption required or is transport-level encryption sufficient?
  • What are the expected read-to-write ratios for messages and search queries?
Out of Scope
✗ File storage and sharing (only message metadata and text considered).
✗ Video/audio calls and screen sharing features.
✗ Mobile app UI/UX design.
✗ Third-party integrations and bots.
📌
Assumptions
  • 2 billion DAU globally
  • Average 40 messages per user per day
  • Average message size 100 bytes
  • Peak traffic concentrated in 8 hours/day (peak factor 3x)
⚡
QPS Calculation
write qps2B × 40 / 86400 ≈ 925,926 messages/sec
peak write qps925,926 × 3 ≈ 2,777,778 messages/sec
read qpsAssuming 5 reads per write (message fetch, presence, search), 925,926 × 5 ≈ 4,629,630 reads/sec
💾
Storage Estimate
daily storage2B × 40 × 100 bytes = 8 TB/day
retention days30
total storage8 TB × 30 = 240 TB
with replication240 TB × 3 (replication factor) = 720 TB
🖥️
Server Estimate
message ingestionKafka cluster with 100 brokers (each handles ~30K msg/sec)
search indexingElasticsearch cluster with 200 nodes (to handle indexing and query load at 4.6M QPS)
🏗️
Architecture Diagram
Architecture Diagram
           +----------------+
           |  Client (User) |
           +-------+--------+
                   | WebSocket / REST
                   v
           +-------+--------+
           |  API Gateway   |
           +-------+--------+
                   |
          +--------+---------+
          |                  |
  +-------v-------+  +-------v-------+
  | Message Ingest|  | Presence Svc  |
  |   (Kafka)     |  |  (Redis)      |
  +-------+-------+  +-------+-------+
          |                  |
  +-------v------------------v-------+
  |       Message Storage (Cassandra) |
  +-------+------------------+-------+
          |                  |
  +-------v-------+  +-------v-------+
  | Search Index  |  | Delivery Svc  |
  | (Elasticsearch)|  | (Push + WS)  |
  +---------------+  +---------------+
🧩
Components
API Gateway
Entry point for client requests; handles authentication, rate limiting, and routes to backend services.
Why thisNginx + Envoy proxy for high throughput, TLS termination, and service mesh integration.
Why notDirect client-to-service communication lacks centralized control and complicates scaling and security.
Message Ingestion (Kafka)
Durable, scalable message queue to ingest and buffer all incoming messages for asynchronous processing.
Why thisKafka supports high throughput (millions of msgs/sec), partitioning by channel/workspace, and durable storage with replication.
Why notRabbitMQ lacks Kafka's throughput and partition scalability; Redis Streams cannot handle 2.7M msg/sec reliably.
Message Storage (Apache Cassandra)
Store all messages persistently with high write throughput and horizontal scalability.
Why thisCassandra provides linear write scalability, tunable consistency, and efficient time-series data modeling for messages.
Why notPostgreSQL requires complex manual sharding at 2.7M writes/sec; B-tree writes don't scale linearly.
Search Index (Elasticsearch)
Full-text search over messages and metadata within workspaces and channels.
Why thisElasticsearch supports inverted indexes, distributed search, and near real-time indexing with horizontal scaling.
Why notSolr has higher latency and less cloud-native scaling; custom inverted index lacks mature ecosystem.
Presence Service (Redis)
Track user online/offline status and typing indicators with sub-millisecond latency.
Why thisRedis TTL natively expires stale presence without cron jobs; supports pub/sub for real-time updates.
Why notMemcached lacks pub/sub and TTL expiration features; Cassandra too slow for presence updates.
Delivery Service
Push messages and presence updates to connected clients via WebSocket or push notifications.
Why thisCustom scalable WebSocket servers with horizontal scaling and sticky sessions to maintain real-time connections.
Why notHTTP polling increases latency and server load; MQTT brokers add complexity without clear benefit.
🔄
Data Flow
1
Client → API Gateway
Send message via WebSocket or REST API
Gateway authenticates and validates message
2
API Gateway → Kafka
Publish message to Kafka topic partitioned by channel_id
Decouples ingestion from processing; supports high throughput
3
Kafka → Message Storage (Cassandra)
Consume messages and write to Cassandra with workspace_id, channel_id as partition keys
Ensures durable storage with linear scalability
4
Kafka → Search Index (Elasticsearch)
Index message content asynchronously for full-text search
Near real-time search availability
5
Kafka → Delivery Service
Push message to all online channel members via WebSocket
Maintains sub-100ms P99 delivery latency
6
Clients → Presence Service (Redis)
Update user presence and typing status
Real-time presence updates with TTL expiration
7
Delivery Service → Clients
Send presence and typing updates via WebSocket
Keeps UI in sync with user states

Practice

(1/5)
1. A high-traffic e-commerce platform expects sudden spikes of 1 million orders per minute. Which component best handles decoupling order intake from downstream inventory updates to maintain responsiveness?
easy
A. An in-memory cache with TTL eviction to buffer incoming orders
B. A relational database with strong ACID transactions for immediate consistency
C. A distributed message queue with partitioning and durable storage
D. A REST API gateway that throttles requests to downstream services

Solution

  1. Step 1: Identify decoupling need

    Immediate consistency via relational DB (A relational database with strong ACID transactions for immediate consistency) blocks on writes, causing latency spikes. In-memory cache (An in-memory cache with TTL eviction to buffer incoming orders) lacks durability and ordering guarantees. API gateway throttling (A REST API gateway that throttles requests to downstream services) limits throughput but doesn't decouple processing.

  2. Step 2: Durable queue with partitioning

    A distributed message queue (A distributed message queue with partitioning and durable storage) buffers spikes, partitions workload, and guarantees durability, enabling asynchronous downstream processing.

  3. Final Answer:

    Option C -> Option C

  4. Quick Check:

    Distributed queue partitions spikes, ensures durability, decouples processing [check]
Hint: Use durable queues to buffer spikes and decouple processing [OK]
Common Mistakes:
2. A distributed system requires generating unique IDs that are sortable by creation time and can be generated independently on multiple nodes without coordination. Which component or technology best fits this requirement?
easy
A. Use ULID or KSUID generators embedded in each service node
B. Use a centralized relational database with auto-increment IDs
C. Use UUID version 4 generated randomly on each node
D. Use a timestamp concatenated with a node ID without collision prevention

Solution

  1. Step 1: Identify requirements for sortable, distributed unique IDs

    Centralized DB auto-increment (A) is a bottleneck and not distributed; UUIDv4 (C) is unique but not sortable; timestamp+node ID without collision prevention (D) risks duplicates.

  2. Step 2: Recognize ULID/KSUID properties

    ULID and KSUID provide time-sortable, collision-resistant IDs generated independently on nodes.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    ULID/KSUID generate sortable unique IDs without coordination [check]
Hint: Sortable unique IDs without coordination -> ULID/KSUID [check]
Common Mistakes:
3. A distributed cache system needs to minimize data reshuffling when nodes are added or removed. Which hashing strategy best supports this requirement?
easy
A. Use consistent hashing with a ring and virtual nodes
B. Use modular hashing with a fixed number of buckets equal to the number of nodes
C. Use a simple hash mod large prime number unrelated to node count
D. Use random hashing with a centralized directory mapping keys to nodes

Solution

  1. Step 1: Identify modular hashing limitations

    Modular hashing requires rehashing all keys when node count changes, causing massive reshuffling.

  2. Step 2: Understand consistent hashing benefits

    Consistent hashing maps nodes and keys on a ring, minimizing key movement to only neighbors when nodes change.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Consistent hashing minimizes reshuffling on node changes [check]
Hint: Consistent hashing minimizes reshuffle on node changes [check]
Common Mistakes:
4. During a global network outage caused by BGP misconfiguration, which recovery approach balances speed and safety best for restoring connectivity without causing routing loops?
medium
A. Gradually reintroduce routes with route flap damping to prevent instability
B. Immediately re-advertise all withdrawn routes globally to restore traffic
C. Use static routing entries temporarily to bypass BGP until full recovery
D. Rely on client-side retries and exponential backoff to handle transient failures

Solution

  1. Step 1: Understand recovery trade-offs

    Immediate re-advertisement (A) risks routing loops and instability; static routes (C) don't scale globally; client retries (D) don't fix routing issues.

  2. Step 2: Evaluate gradual reintroduction

    Route flap damping (B) prevents rapid route changes causing instability, balancing speed and safety during recovery.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Route flap damping prevents loops during gradual route restoration [check]
Hint: Gradual route reintroduction with damping avoids loops [check]
Common Mistakes:
5. At 10x scale, which component in a multi-tier storage system is most likely to become the bottleneck first, and why?
hard
A. Archive storage because tape drives have limited concurrent access bandwidth
B. Metadata service because it maintains global access frequency and tiering decisions, causing high coordination overhead
C. Cold storage because object storage scales linearly with data size
D. Warm storage because HDDs have slower seek times but can be scaled horizontally easily

Solution

  1. Step 1: Identify component roles at scale

    Archive storage uses tape drives with limited concurrent access bandwidth and high latency.

  2. Step 2: Analyze scaling challenges

    At 10x scale, tape drives become bottleneck due to limited parallelism and slow seek times, unlike metadata which can be sharded.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Archive tape bandwidth limits -> first bottleneck [check]
Hint: Tape archive bandwidth limits scale poorly [check]
Common Mistakes:
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard