Bird
Raised Fist0

Vertical Scaling vs Horizontal Scaling - When Each Breaks and What to Do

HLDmediumInterview freq: common
Choose your preparation mode4 modes available
ArchitectureTraceChallenge
🎯
Vertical Scaling vs Horizontal Scaling - When Each Breaks and What to Do
45 min · senior
senior

You are designing the backend infrastructure for a startup with 50 million daily active users (DAU) that requires sub-200ms P99 latency for user requests. The system must handle encrypted data at rest and in transit, support offline delivery of notifications, and maintain high availability. The engineering team is debating vertical scaling (beefier machines) versus horizontal scaling (more machines). You must design a system that balances these approaches, understands their failure modes at scale, and proposes mitigations.

Constraints
  • End-to-end encryption required for all user data
  • Offline delivery support with eventual consistency
  • P99 latency under 200ms for all user-facing APIs
  • System must handle 50M DAU with peak QPS of ~1M
  • High availability with zero single points of failure
📋
Guided Steps
1/5
Step 1
Choosing the Scaling Strategy for Compute Layer
Given 50M DAU and sub-200ms P99 latency, should you vertically scale a few powerful servers or horizontally scale many commodity servers for your compute layer? Why?
💡 Hint
Consider failure domains and cost-effectiveness at 1M QPS peak load.
Key Decision
Use Kubernetes on AWS EC2 M5 instances with autoscaling groups - horizontal scaling allows graceful degradation and cost-effective scaling beyond single machine CPU/memory limits.
Interviewer Expectation
Mid: Names horizontal scaling with autoscaling groups. Senior: Explains vertical scaling bottlenecks like CPU saturation, memory limits, and single points of failure at 1M QPS, plus how horizontal scaling enables load distribution and fault isolation.
⚠️Common Wrong Answer
Relying solely on vertical scaling with beefy servers - leads to single points of failure and expensive hardware upgrades that don’t scale linearly beyond ~64 cores.
Step 2
Data Storage: Vertical vs Horizontal Scaling
For encrypted user data storage, should you use a vertically scaled relational database or a horizontally scalable NoSQL store? What breaks at scale?
💡 Hint
Think about write throughput, encryption overhead, and query patterns for 50M users.
Key Decision
Use Cassandra with encryption at rest enabled - horizontally scalable with linear write throughput, supports partition keys for user data locality, and handles encryption overhead efficiently.
Interviewer Expectation
Mid: Names Cassandra for horizontal scaling. Senior: Explains why vertical RDBMS breaks due to write bottlenecks and encryption CPU overhead, and how Cassandra’s LSM-tree and partitioning enable linear scaling and efficient range queries.
⚠️Common Wrong Answer
Using a single vertically scaled PostgreSQL instance - breaks under write load and encryption CPU overhead, causing high latency and downtime.
Step 3
Handling Offline Delivery and Eventual Consistency
How do you design offline delivery of notifications with eventual consistency while maintaining P99 < 200ms for online users?
💡 Hint
Consider message queues and caching layers that support retries and offline sync.
Key Decision
Use Kafka for durable message queueing with partition keys per user, combined with Redis as a cache for recent notifications - Kafka ensures ordered delivery and retries, Redis enables low-latency reads for online users.
Interviewer Expectation
Mid: Names Kafka and Redis. Senior: Explains Kafka partitioning by user ID to maintain order, how Redis TTL caching reduces latency, and how offline clients sync by replaying Kafka offsets.
⚠️Common Wrong Answer
Using HTTP polling for offline delivery - causes massive reconnect storms at 50M DAU, collapsing the system under load.
Step 4
Mitigating Failure Modes in Vertical and Horizontal Scaling
What are the key failure modes when vertical scaling hits hardware limits or horizontal scaling hits network partitions? How do you mitigate them?
💡 Hint
Think about cascading failures, single points of failure, and consistency trade-offs.
Key Decision
Implement circuit breakers and bulkheads in microservices, use multi-AZ deployments with Cassandra and Kafka replication, and employ client-side exponential backoff and idempotency keys to handle retries safely.
Interviewer Expectation
Mid: Mentions multi-AZ and retries. Senior: Details cascading failure sequences from vertical scaling saturation (CPU/memory exhaustion), network partitions causing split-brain in Cassandra, and how circuit breakers prevent cascading failures.
⚠️Common Wrong Answer
Ignoring failure modes and relying on auto-restart - leads to cascading failures and data inconsistency under load spikes.
Step 5
API Design Under Scaling Constraints
Design 3 core API endpoints that reflect your scaling choices and constraints, including encryption, offline delivery, and latency requirements.
💡 Hint
Consider endpoints for sending encrypted data, syncing offline updates, and acknowledging processing.
Key Decision
POST /sendEncryptedData with client-side encryption, GET /syncUpdates with cursor-based pagination from Kafka offsets, and POST /acknowledgeDelivery with idempotency keys to ensure exactly-once processing.
Interviewer Expectation
Mid: Defines basic endpoints. Senior: Explains encryption at client, cursor-based sync to avoid full data fetch, and idempotency to handle retries without duplication.
⚠️Common Wrong Answer
Designing synchronous blocking endpoints without pagination or idempotency - causes high latency and duplicate processing under load.
🔌
API Design Challenge
Sketch 3 core API endpoints that illustrate how clients interact with the system under vertical and horizontal scaling constraints.
Hints
  • How does the client send encrypted data to the backend?
  • How does the client sync offline data and receive updates?
  • How does the system acknowledge successful processing to the client?
📊
Scoring Rubric
▼ Show
Must Have - 60%
  • Clear explanation of vertical scaling limits at 1M QPS and why horizontal scaling is needed
  • Use of Cassandra with partition keys for encrypted user data storage at scale
  • Kafka for ordered durable messaging supporting offline delivery
  • Mitigation of failure modes with circuit breakers and multi-AZ replication
  • API design reflecting encryption, offline sync, and idempotency
Good to Have - 30%
  • Redis caching layer to reduce latency for online users
  • Client-side encryption details and key management considerations
  • Autoscaling groups and Kubernetes orchestration for horizontal scaling
  • Discussion of network partitions and split-brain scenarios
Bonus - 10%
  • Detailed failure sequence tracing for vertical scaling saturation
  • Explanations of trade-offs between consistency and availability in network partitions
Anti-patterns to Avoid
✅
Model Answer
▼ Reveal

This design balances vertical and horizontal scaling for a backend serving 50M DAU with sub-200ms P99 latency. Horizontal scaling with Kubernetes on AWS EC2 M5 instances enables cost-effective, fault-isolated compute. Cassandra provides horizontally scalable encrypted data storage with partition keys for user locality. Kafka ensures ordered, durable messaging for offline notification delivery, complemented by Redis caching for low-latency reads. Failure modes from vertical scaling saturation and network partitions are mitigated with circuit breakers, bulkheads, multi-AZ replication, and client-side retry strategies with idempotency. APIs are designed with client-side encryption, cursor-based pagination, and idempotency keys to meet latency and consistency requirements.

Key Decisions
  • Adopt horizontal scaling for compute using Kubernetes on AWS EC2 M5 instances to handle 1M QPS with fault isolation
  • Use Cassandra with encryption at rest and partition keys on user_id to achieve linear write scalability and efficient queries
  • Implement Kafka with user_id partition keys to guarantee ordered, durable message delivery supporting offline sync
  • Deploy Redis cache with TTL to reduce latency for online users accessing recent notifications
  • Apply circuit breakers and bulkheads in microservices to isolate failures and prevent cascading outages
  • Use multi-AZ replication for Cassandra and Kafka to ensure high availability and fault tolerance
  • Design APIs with client-side encryption, cursor-based pagination from Kafka offsets, and idempotency keys to handle retries and offline sync safely
  • Kubernetes on AWS EC2 M5 instances for horizontal compute scaling with autoscaling groups
  • Cassandra with encryption at rest and partition key on user_id for linear write scalability
  • Kafka with user_id partition keys for ordered, durable message delivery supporting offline sync
  • Redis cache with TTL for low-latency reads of recent notifications for online users
  • Client-side encryption to meet end-to-end encryption requirements
  • Circuit breakers and bulkheads in microservices to isolate failures and prevent cascading
  • Multi-AZ deployment for Cassandra and Kafka to ensure high availability and fault tolerance
  • API design with cursor-based pagination and idempotency keys to handle retries and offline sync

Practice

(1/5)
1. A system must match riders with nearby drivers in under 2 seconds while handling millions of concurrent requests. Which component is best suited to efficiently index and query driver locations in real-time?
easy
A. An in-memory spatial index like an R-tree or geohash-based Redis module
B. A relational database with geospatial extensions for location queries
C. A distributed log system like Kafka to stream location updates
D. A batch processing system like Hadoop to periodically update driver locations

Solution

  1. Step 1: Identify location query requirements

    Real-time matching requires low-latency spatial queries on dynamic driver locations.

  2. Step 2: Evaluate component suitability

    Relational DBs (A) have high latency for frequent updates; Kafka (C) streams data but doesn't index locations; Hadoop (D) is batch-oriented and too slow. In-memory spatial indexes (B) provide fast nearest-neighbor queries and frequent updates.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    In-memory spatial index enables fast real-time location queries [check]
Hint: Real-time location needs fast spatial index, not batch or logs [check]
Common Mistakes:
2. In a system where a client writes a user profile update to DynamoDB, then a cache invalidation message is sent to Redis at step 4, what happens if the cache invalidation fails at step 4?
easy
A. DynamoDB write is rolled back automatically to maintain consistency
B. Redis automatically retries invalidation and blocks client until success
C. The cache remains stale until TTL expires or manual refresh
D. Client receives an error and retries the entire update operation

Solution

  1. Step 1: Trace cache invalidation at step 4

    Cache invalidation is asynchronous and best-effort; failure does not rollback DB writes.

  2. Step 2: Understand failure impact on cache state

    If invalidation fails, Redis cache remains stale until TTL expiry or next explicit refresh.

  3. Final Answer:

    Option C -> Option C

  4. Quick Check:

    Failed cache invalidation -> stale cache until TTL or manual refresh [check]
Hint: Failed cache invalidation -> stale cache, DB write persists [check]
Common Mistakes:
3. For ensuring exactly-once payment processing in a distributed payment system, which architecture is best suited given high throughput and fault tolerance requirements?
medium
A. Use an event-driven architecture with Kafka for event sourcing and idempotent consumers
B. Use a synchronous two-phase commit protocol across all services to guarantee atomicity
C. Use a centralized relational database with serializable transactions for all payment states
D. Use client-side retries with unique request IDs and eventual consistency in backend

Solution

  1. Step 1: Analyze architecture trade-offs

    Synchronous two-phase commit (Use a synchronous two-phase commit protocol across all services to guarantee atomicity) blocks and reduces throughput; centralized DB (Use a centralized relational database with serializable transactions for all payment states) limits scalability; client-side retries (Use client-side retries with unique request IDs and eventual consistency in backend) risk inconsistent states.

  2. Step 2: Evaluate event-driven approach

    Kafka event sourcing with idempotent consumers supports high throughput, fault tolerance, and exactly-once processing semantics.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Kafka event sourcing -> scalable, fault-tolerant, idempotent processing [check]
Hint: Event sourcing with idempotent consumers scales best [check]
Common Mistakes:
4. At 10x scale, which component in the faceted search system is most likely to fail first and why?
hard
A. The Query Coordinator, because it cannot merge partial results fast enough under high concurrency
B. The indexing pipeline, because it cannot keep up with the volume of document updates
C. The shard-level cache, because memory pressure causes frequent evictions and cache misses spike
D. The shard nodes, because inverted index lookups become CPU-bound and cannot handle query volume

Solution

  1. Step 1: Analyze bottlenecks at 10x query scale

    Shard nodes perform CPU-intensive inverted index lookups; query volume increase causes CPU saturation first.

  2. Step 2: Compare other components

    Query Coordinator merges are parallelizable; cache evictions increase but cache can be scaled; indexing pipeline is async and can be scaled horizontally.

  3. Final Answer:

    Option D -> Option D

  4. Quick Check:

    Shard nodes CPU-bound -> first failure point at scale [check]
Hint: Shard nodes CPU-bound first at 10x scale [check]
Common Mistakes:
5. At 10x scale, which component in a NoSQL cluster handling user sessions fails first and why?
hard
A. The load balancer because it cannot distribute requests evenly at scale
B. The session management layer because it maintains stateful connections that can't be sharded easily
C. The storage nodes because disk I/O saturates under high write volume
D. The network switches because they cannot handle increased traffic volume

Solution

  1. Step 1: Identify storage node bottlenecks at high write volume

    Disk I/O can saturate under 10x writes, limiting storage node throughput.

  2. Step 2: Compare other components' scalability

    Session management can shard state; load balancers and network switches scale with hardware upgrades.

  3. Final Answer:

    Option C -> Option C

  4. Quick Check:

    Disk I/O saturation -> storage node bottleneck at 10x scale [check]
Hint: Disk I/O saturates first at 10x writes [check]
Common Mistakes:
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard