Bird
Raised Fist0

Scalability - Designing for 10x, 100x, 1000x Traffic Growth from Day One

HLDmediumInterview freq: high
Choose your preparation mode4 modes available
ArchitectureTraceChallenge
🎯
Scalability - Designing for 10x, 100x, 1000x Traffic Growth from Day One
45 min · senior
senior

You are building a messaging platform currently serving 50 million daily active users (DAU). The business expects rapid growth to 500 million DAU within 2 years and potentially 5 billion DAU in the longer term. The system must be designed from day one to handle 10x, 100x, and 1000x traffic growth without major rewrites. The platform requires end-to-end encryption, offline message delivery, and must maintain a P99 latency under 200ms for message send and receive operations.

Constraints
  • End-to-end encryption with zero-trust server architecture
  • Offline message delivery with guaranteed at-least-once semantics
  • P99 latency under 200ms for message send and receive
  • Support for 10x, 100x, and 1000x traffic growth from day one
  • Data durability and consistency across geo-distributed regions
📋
Guided Steps
1/5
Step 1
Choosing the Messaging Backbone
What messaging infrastructure will you use to handle 10x to 1000x message throughput growth while preserving message order and delivery guarantees?
💡 Hint
Consider partitioning keys that ensure ordering and scalability, and technologies that support high throughput with durability.
Key Decision
Use Apache Kafka with conversation_id as the partition key - this ensures all messages in a conversation are on the same partition, preserving order and enabling linear scalability with partition count.
Interviewer Expectation
Mid-level: Names Kafka as the messaging backbone. Senior: Explains why conversation_id is chosen as partition key over sender_id to avoid out-of-order delivery and cross-partition coordination.
⚠️Common Wrong Answer
Using RabbitMQ or Redis Pub/Sub without partitioning - these systems do not scale linearly at 1000x throughput and cannot guarantee ordering at massive scale.
Step 2
Data Storage for Message Persistence
Which database technology will you use to store encrypted messages for offline delivery and fast retrieval at scale?
💡 Hint
Look for a database optimized for high write throughput, range queries for cursor-based sync, and geo-distribution support.
Key Decision
Use Apache Cassandra with partition key as conversation_id - LSM-tree storage enables high write throughput and efficient range scans for message sync, supporting geo-replication for durability.
Interviewer Expectation
Mid-level: Suggests a NoSQL database like Cassandra. Senior: Explains why conversation_id as partition key enables O(1) range scans for cursor-based sync and why relational DBs or MongoDB would fail at scale.
⚠️Common Wrong Answer
Using relational databases like MySQL for message storage - they cannot handle the write volume and range queries efficiently at 1000x scale.
Step 3
Ensuring End-to-End Encryption and Zero-Trust
How will you design the encryption scheme to ensure zero-trust servers while maintaining scalability and offline delivery?
💡 Hint
Consider client-side encryption keys and how to manage key distribution without server access to plaintext.
Key Decision
Implement client-side end-to-end encryption with keys managed on clients and use a secure key management service (e.g., HashiCorp Vault) for key exchange metadata, ensuring servers never see plaintext.
Interviewer Expectation
Mid-level: Mentions end-to-end encryption. Senior: Details client-side key management, zero-trust architecture, and how offline delivery is supported without server access to keys.
⚠️Common Wrong Answer
Encrypting messages only in transit (TLS) but storing plaintext on servers - violates zero-trust and exposes data at scale.
Step 4
Designing the Offline Delivery and Sync Mechanism
How will the system guarantee offline message delivery with at-least-once semantics and efficient client sync after reconnect?
💡 Hint
Think about cursor-based sync, message acknowledgments, and how to avoid duplicate deliveries at scale.
Key Decision
Use cursor-based sync with Cassandra range queries on conversation_id and Kafka offsets for message replay; clients send ACKs to update delivery state stored in Redis with TTL for quick lookup.
Interviewer Expectation
Mid-level: Describes cursor-based sync. Senior: Explains how Kafka offsets and Cassandra range scans combine for efficient replay, and how Redis TTL presence supports offline detection and deduplication.
⚠️Common Wrong Answer
HTTP polling for offline sync - causes massive reconnect storms at 50M+ DAU and collapses under load.
Step 5
API Design for Scalability and Reliability
Sketch 3 core API endpoints that support sending messages, syncing offline messages, and acknowledging receipt, ensuring they scale and meet latency constraints.
💡 Hint
Consider idempotency, batching, and statelessness in your API design.
Key Decision
Design POST /messages/send with idempotency keys, GET /messages/sync with cursor parameters for incremental fetch, and POST /messages/ack for delivery confirmation; all endpoints stateless and backed by scalable Kafka and Cassandra layers.
Interviewer Expectation
Mid-level: Names endpoints and basic functionality. Senior: Explains idempotency keys to prevent duplicates, cursor-based incremental sync, and how stateless APIs enable horizontal scaling.
⚠️Common Wrong Answer
Using synchronous blocking APIs without idempotency - leads to retries causing duplicates and poor scalability.
🔌
API Design Challenge
Sketch 3 core API endpoints for the messaging platform that support scalability and offline delivery.
Hints
  • How does the client send a message securely and efficiently?
  • How does the client synchronize messages after being offline?
  • How does the recipient acknowledge message receipt to ensure delivery guarantees?
📊
Scoring Rubric
▼ Show
Must Have - 60%
  • Use of Kafka with conversation_id partition key to ensure ordering and scalability
  • Use of Cassandra with conversation_id partition key for efficient message storage and sync
  • Client-side end-to-end encryption ensuring zero-trust server architecture
  • Cursor-based sync mechanism for offline delivery with at-least-once semantics
Good to Have - 30%
  • Redis TTL presence for offline detection and delivery state caching
  • Stateless, idempotent REST APIs designed for horizontal scaling
  • Geo-replication strategies for data durability and latency optimization
Bonus - 10%
  • Detailed failure mode analysis of Kafka partition rebalancing impact on ordering
  • Use of asynchronous workers for message processing to handle 1000x scale
Anti-patterns to Avoid
HTTP polling at 50M DAU: 50 million reconnects per minute just to check for messages - collapses under load and causes system instability
✅
Model Answer
▼ Reveal

The system uses Apache Kafka with conversation_id partitioning to ensure ordered, scalable messaging. Apache Cassandra stores encrypted messages with conversation_id as partition key for efficient range queries and geo-replication. Client-side end-to-end encryption with keys managed on clients and secure key exchange via HashiCorp Vault ensures zero-trust. Offline delivery uses cursor-based sync combining Kafka offsets and Cassandra scans, with Redis caching delivery state and presence with TTL. APIs are stateless REST endpoints with idempotency keys and cursor parameters for scalability and low latency.

Key Decisions
  • Use Kafka with conversation_id as partition key to preserve message order and enable linear scalability
  • Use Cassandra with conversation_id partition key for high write throughput and efficient cursor-based sync
  • Implement client-side end-to-end encryption with keys never exposed to servers
  • Use cursor-based sync combining Kafka offsets and Cassandra range scans for offline message replay
  • Cache delivery acknowledgments and presence in Redis with TTL for quick lookup and deduplication
  • Design stateless REST APIs with idempotency keys and cursor parameters to support horizontal scaling and prevent duplicates
  • Apache Kafka with conversation_id as partition key for ordered, scalable messaging
  • Apache Cassandra with conversation_id partition key for high write throughput and efficient range queries
  • Client-side end-to-end encryption with secure key management (HashiCorp Vault) for zero-trust
  • Redis with TTL-based presence detection for offline status and delivery state caching
  • Cursor-based sync combining Kafka offsets and Cassandra range scans for offline message replay
  • Stateless REST APIs with idempotency keys and cursor parameters for scalable client interaction
  • Geo-replication of Cassandra clusters for data durability and low latency across regions
  • Use of asynchronous message processing workers consuming Kafka partitions for horizontal scaling

Practice

(1/5)
1. A mobile app requires tailored API responses per device type to reduce payload size and improve UX. Which component best handles this requirement in a microservices architecture?
easy
A. Use a shared database view that filters data per device type
B. Use a CDN to cache different payloads per device type
C. Let each microservice handle device-specific logic internally
D. Implement a Backend-for-Frontend (BFF) layer that adapts responses per client

Solution

  1. Step 1: Identify the role of BFF

    The BFF layer acts as a tailored API gateway that adapts responses specifically for each client type, reducing payload and improving UX.

  2. Step 2: Eliminate other options

    Use a shared database view that filters data per device type is incorrect because database views are not designed for client-specific adaptation and increase coupling. Let each microservice handle device-specific logic internally is wrong as microservices should remain generic and not handle client-specific logic. Use a CDN to cache different payloads per device type is incorrect because CDNs cache static content and cannot dynamically tailor API responses per device.

  3. Final Answer:

    Option D -> Option D

  4. Quick Check:

    BFF adapts APIs per client type for optimized payloads [check]
Hint: BFF adapts APIs per client type for optimized payloads [check]
Common Mistakes:
2. In the adaptive streaming request flow, at step 4 the CDN edge server checks the client's current bandwidth estimate. What does the CDN do if the bandwidth is below the threshold for HD quality?
easy
A. It immediately switches the stream to the lowest bitrate available without buffering
B. It pauses the stream and waits for the client to buffer more data
C. It serves the next video chunk at a lower bitrate from its cache if available
D. It requests the origin server to send a new manifest with lower bitrate options

Solution

  1. Step 1: Understand CDN adaptive streaming role at step 4

    The CDN edge server uses bandwidth info to decide streaming quality dynamically.

  2. Step 2: Analyze adaptive streaming behavior on low bandwidth

    Switching immediately to lowest bitrate (It immediately switches the stream to the lowest bitrate available without buffering) can cause playback stalls. Requesting new manifest (It requests the origin server to send a new manifest with lower bitrate options) is unnecessary at CDN edge. Serving lower bitrate chunk from cache (It serves the next video chunk at a lower bitrate from its cache if available) requires prior caching. The CDN typically pauses streaming to allow client buffering (It pauses the stream and waits for the client to buffer more data) to avoid playback interruption.

  3. Final Answer:

    Option B -> Option B

  4. Quick Check:

    CDN pauses streaming to buffer on low bandwidth [check]
Hint: Pause to buffer before switching bitrate [check]
Common Mistakes:
3. At 10x scale, which component in the real-time leaderboard system is most likely to fail first, and why?
hard
A. The Redis cluster because sorted sets require in-memory storage and cannot shard scores efficiently
B. The API Gateway because it cannot handle the surge in client connections
C. The Ranking Worker because it cannot process the increased event stream fast enough
D. The database because it cannot handle the increased write throughput

Solution

  1. Step 1: Analyze Redis Sorted Sets scaling limits

    Sorted Sets require in-memory storage and have limited sharding options, making Redis cluster memory the bottleneck.

  2. Step 2: Compare other components

    Databases can be scaled horizontally; Ranking Workers can be parallelized; API Gateways scale with load balancers.

  3. Step 3: Identify Redis cluster as first failure point

    Memory pressure and inability to shard sorted sets efficiently cause Redis cluster to fail first under 10x load.

  4. Final Answer:

    Option A -> Option A

  5. Quick Check:

    Redis Sorted Sets memory and sharding limits cause first failure at scale [check]
Hint: Redis Sorted Sets memory limits break first at scale [check]
Common Mistakes:
4. At 10x traffic scale, which component is most likely to fail first in a latency monitoring system that tracks P50, P95, and P99 latencies, and why?
hard
A. The data storage, because it cannot handle the increased volume of raw latency data
B. The aggregation service, because computing percentiles in real-time requires heavy CPU and memory resources
C. The alerting system, because it cannot process the increased number of alerts generated
D. The client instrumentation, because it cannot send telemetry fast enough

Solution

  1. Step 1: Identify bottleneck in percentile computation

    Computing P50, P95, and P99 latencies in real-time requires sorting or approximation algorithms that are CPU and memory intensive.

  2. Step 2: Why aggregation service fails first

    At 10x scale, the aggregation service faces exponential resource demands, making it the first to fail before storage or alerting.

  3. Final Answer:

    Option B -> Option B

  4. Quick Check:

    Real-time percentile computation scales poorly -> aggregation service fails first [check]
Hint: Percentile computation is CPU-heavy; aggregation fails first [OK]
Common Mistakes:
5. At 10x scale, which component in the SLO error budget tracking system is most likely to become the bottleneck first, and why?
hard
A. The metrics aggregation service, because it maintains stateful counters and cannot shard error budget calculations easily
B. The dashboard visualization layer, because rendering 10x more data overwhelms client browsers
C. The raw event ingestion pipeline, because Kafka partitions can be increased indefinitely without limit
D. The alerting system, because it can batch alerts and scale horizontally without state

Solution

  1. Step 1: Identify stateful components that limit horizontal scaling

    The metrics aggregation service maintains counters and error budget state, which are hard to shard without losing accuracy.

  2. Step 2: Evaluate other components' scalability

    Dashboards (A) are client-side and less critical; Kafka (C) scales well with partitions; alerting (D) can batch and scale horizontally.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Stateful aggregation -> sharding limits -> bottleneck at scale [check]
Hint: Stateful aggregation limits sharding, causing first bottleneck at scale [OK]
Common Mistakes:
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard