Bird
Raised Fist0

Scalability - Designing for 10x, 100x, 1000x Traffic Growth from Day One

HLDmediumInterview freq: high
Choose your preparation mode4 modes available
ArchitectureTraceChallenge
🏗️
Scalability - Designing for 10x, 100x, 1000x Traffic Growth from Day One
Designing systems that gracefully handle exponential traffic growth from launch
mediumNon-Functional Requirements FrameworkFreq: high
⚙️
Functional Requirements
  • Support seamless scaling of read and write traffic by 10x, 100x, and 1000x without downtime
  • Maintain consistent low latency (e.g., 100ms P99) under peak load
  • Provide data durability and availability with 99.99% uptime SLA
  • Enable horizontal scaling of components with minimal manual intervention
  • Support real-time monitoring and alerting for performance and failures
📊
Non-Functional Requirements
  • Handle up to 2 billion daily active users (DAU)
  • Sustain peak write throughput of approximately 925K writes/sec
  • Ensure 100ms P99 latency for critical user-facing APIs
  • Maintain 99.99% availability across all services
  • Data retention for at least 30 days with replication factor of 3
❓
Clarifying Questions to Ask
  • What is the expected user growth timeline for 10x, 100x, and 1000x traffic?
  • Are there specific peak traffic patterns or daily/weekly seasonality?
  • What is the acceptable latency SLA for read and write operations?
  • Is eventual consistency acceptable or is strong consistency required?
  • What is the data retention policy and replication requirements?
  • Are there any constraints on technology stack or cloud/on-premise deployment?
Out of Scope
✗ Designing specific business logic or domain models
✗ Security and compliance requirements beyond availability and durability
✗ Detailed UI/UX considerations
✗ Multi-region geo-distribution and disaster recovery
📌
Assumptions
  • 2 billion DAU
  • Each user generates 40 messages per day
  • Average message size is 100 bytes
  • Replication factor of 3 for durability
⚡
QPS Calculation
write qps2,000,000,000 × 40 / 86,400 ≈ 925,926 writes/sec
peak write qpsAssuming 2× peak factor → 1,851,852 writes/sec
read qpsAssuming 5 reads per write → 925,926 × 5 = 4,629,630 reads/sec
💾
Storage Estimate
daily storage2,000,000,000 × 40 × 100 bytes = 8,000,000,000,000 bytes = 8 TB/day
retention 30 days8 TB × 30 = 240 TB
with replication240 TB × 3 = 720 TB total storage
🖥️
Server Estimate
write pathKafka cluster with 100 brokers (each handles ~20K writes/sec)
read pathCassandra cluster with 200 nodes (each handles ~23K reads/sec)
🏗️
Architecture Diagram
Architecture Diagram
          +-------------+
          |   Clients   |
          +------+------+
                 |
          +------+-------+
          |  API Gateway |
          +------+-------+
                 |
          +------+-------+
          |  Kafka Cluster|
          +------+-------+
                 |
          +------+-------+
          |  Processing  |
          |  Workers     |
          +------+-------+
                 |
          +------+-------+
          | Cassandra DB |
          +------+-------+
                 |
          +------+-------+
          |  Redis Cache |
          +--------------+
🧩
Components
API Gateway
Handles client requests, performs authentication, rate limiting, and routes to backend
Why thisEnvoy proxy supports high throughput, dynamic routing, and observability at scale
Why notNginx lacks native service mesh integration and dynamic config for 1M+ QPS
Kafka Cluster
Durable, distributed write log to buffer and decouple write traffic from processing
Why thisKafka scales linearly with partition count and supports 1M+ writes/sec with replication
Why notRabbitMQ does not scale linearly and has higher latency under heavy load
Processing Workers
Consume Kafka messages, perform business logic, and write to storage
Why thisKubernetes-managed stateless pods enable elastic scaling and fault tolerance
Why notMonolithic services lack elasticity and increase blast radius on failure
Cassandra DB
Distributed NoSQL store for high write throughput and horizontal scaling
Why thisCassandra supports linear write scalability, tunable consistency, and multi-DC replication
Why notPostgreSQL requires complex manual sharding and does not scale writes linearly
Redis Cache
In-memory cache for hot data to reduce read latency and backend load
Why thisRedis TTL natively expires stale data without cron jobs, supports sub-millisecond lookup
Why notMemcached lacks persistence and advanced eviction policies needed at scale
🔄
Data Flow
1
Clients → API Gateway
Send write/read requests
API Gateway handles auth and rate limiting
2
API Gateway → Kafka Cluster
Publish write messages asynchronously
Decouples client writes from processing
3
Processing Workers → Kafka Cluster
Consume messages for processing
Workers scale horizontally to handle load
4
Processing Workers → Cassandra DB
Write processed data
Cassandra handles high write throughput
5
Clients → API Gateway
Send read requests
Reads first check Redis cache
6
API Gateway → Redis Cache
Query cache for hot data
Sub-millisecond reads reduce DB load
7
API Gateway → Cassandra DB
Fallback read if cache miss
Ensures data availability
🗄️
messages
PK: user_id
Stores user messages partitioned by user_id and clustered by timestamp for time-ordered queries
🗄️
user_profiles
PK: user_id
Stores user profile data partitioned by user_id and clustered by last update time
🔌
POST /messages
Request
{
  "user_id": "string",
  "message": "string",
  "timestamp": "ISO8601 string"
}
Response
{
  "status": "success|failure",
  "message_id": "string"
}
Endpoint to submit a new message; writes are async via Kafka
🔌
GET /messages
Request
{
  "user_id": "string",
  "start_time": "ISO8601 string",
  "end_time": "ISO8601 string",
  "limit": "integer"
}
Response
{
  "messages": "array of message objects"
}
Fetches messages for a user in a time range, reads from cache or DB
🔌
GET /user/profile
Request
{
  "user_id": "string"
}
Response
{
  "profile": "user profile object"
}
Fetches user profile data with low latency

Practice

(1/5)
1. A monitoring system must detect when a metric exceeds a fixed limit for 5 consecutive minutes and trigger an alert. Which component is best suited to implement this logic?
easy
A. Threshold-based alerting component that evaluates metric values against static limits
B. Event ingestion pipeline that collects raw metric data without evaluation
C. Composite alert processor that combines multiple alert conditions into one
D. Anomaly detection engine using machine learning to identify unusual patterns

Solution

  1. Step 1: Identify alerting logic type

    The requirement is to detect when a metric exceeds a fixed limit for a duration, which is a classic threshold alert scenario.

  2. Step 2: Match component to logic

    Threshold-based alerting components are designed to evaluate metrics against static limits over time windows, unlike anomaly detection which looks for statistical deviations.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Threshold alerting matches fixed limit checks over time [check]
Hint: Fixed limit over time -> threshold alert component [check]
Common Mistakes:
2. In a code repository system, which component is best suited to efficiently store and retrieve file diffs to minimize storage space while allowing fast access for merges?
easy
A. Use a relational database with BLOB columns to store entire file versions
B. Use a distributed cache like Redis to store diffs temporarily for quick access
C. Use a content-addressable storage system that stores objects by hash and reuses identical content
D. Use a simple file system directory structure with full copies of each file version

Solution

  1. Step 1: Identify storage needs for diffs

    Storing diffs efficiently requires deduplication and immutability; content-addressable storage (CAS) stores objects by their hash, enabling reuse of identical content and saving space.

  2. Step 2: Evaluate other options

    Relational DB with BLOBs stores full versions, wasting space; Redis is volatile and not durable for persistent storage; file system copies waste space and slow for merges.

  3. Final Answer:

    Option C -> Option C

  4. Quick Check:

    CAS stores objects by hash -> deduplication and fast retrieval [check]
Hint: Content-addressable storage deduplicates by hash [check]
Common Mistakes:
3. In the trace propagation flow, at step 4 the downstream Service B receives an HTTP request with trace headers. What does Service B's instrumentation do at this step?
easy
A. Starts a new root trace ignoring incoming trace headers
B. Sends trace data directly to the trace storage without processing
C. Drops trace headers to reduce overhead and logs locally only
D. Extracts trace context from headers and starts a child span linked to the parent

Solution

  1. Step 1: Understand instrumentation behavior on incoming requests

    At step 4, Service B's instrumentation extracts the trace context from incoming HTTP headers and starts a new span as a child of the parent span to maintain trace continuity.

  2. Step 2: Eliminate incorrect behaviors

    Starting a new root trace breaks trace linkage. Sending data directly to storage without processing is not the instrumentation's role. Dropping headers loses trace continuity.

  3. Final Answer:

    Option D -> Option D

  4. Quick Check:

    Service B extracts context and starts child span [check]
Hint: Extract incoming context, start child span to continue trace [OK]
Common Mistakes:
4. You must design a notification system that requires immediate user feedback on success or failure of each notification. Which communication approach is best suited for this requirement?
medium
A. Use synchronous HTTP calls with client-side timeouts and retries
B. Use fire-and-forget UDP packets for low latency
C. Use batch processing with delayed acknowledgment
D. Use an asynchronous message queue with eventual consistency

Solution

  1. Step 1: Identify requirement for immediate feedback

    Immediate success/failure feedback requires synchronous communication to confirm delivery.

  2. Step 2: Evaluate synchronous HTTP calls

    Synchronous HTTP with timeouts and retries provides immediate response and error handling.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Synchronous calls provide immediate feedback required by the system [check]
Hint: Immediate feedback needs synchronous calls with timeouts [check]
Common Mistakes:
5. At 10x scale, which component in a cache stampede prevention system is most likely to fail first, and why?
hard
A. Cache servers, because they run out of memory storing more keys
B. Backend database, because it cannot handle the increased read load from cache misses
C. Distributed lock service, because lock contention and coordination overhead grow exponentially
D. Load balancers, because they cannot distribute requests evenly at high concurrency

Solution

  1. Step 1: Identify bottlenecks at 10x scale

    Backend and cache scale horizontally; load balancers scale well.

  2. Step 2: Analyze distributed lock service

    Lock service faces increased contention and coordination overhead, causing latency and failures.

  3. Final Answer:

    Option C -> Option C

  4. Quick Check:

    Distributed lock service bottlenecks first due to coordination overhead [check]
Hint: Lock service bottlenecks first due to exponential coordination overhead [OK]
Common Mistakes:
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard