Bird
Raised Fist0

Vertical Scaling vs Horizontal Scaling - When Each Breaks and What to Do

HLDmediumInterview freq: common
Choose your preparation mode4 modes available
ArchitectureTraceChallenge
🏗️
Vertical Scaling vs Horizontal Scaling - When Each Breaks and What to Do
Understanding the trade-offs and failure modes of vertical and horizontal scaling in distributed systems
mediumCore Distributed Systems ConceptsFreq: common
⚙️
Functional Requirements
  • Support increasing system capacity to handle growing user load
  • Maintain low latency (sub-100ms P99) for user requests under peak load
  • Ensure high availability (99.99% uptime) during scaling operations
  • Provide mechanisms to detect and recover from scaling-related failures
  • Allow seamless capacity expansion without significant downtime
  • Support both read-heavy and write-heavy workloads
📊
Non-Functional Requirements
  • System must handle up to 2 billion daily active users (DAU)
  • Peak write throughput up to 1 million writes per second
  • Read throughput up to 10 million reads per second
  • End-to-end request latency P99 under 100 milliseconds
  • System availability SLA of 99.99% (less than ~52 minutes downtime per year)
❓
Clarifying Questions to Ask
  • What is the expected growth rate of user traffic and data volume?
  • Are there specific latency or availability SLAs to meet during scaling?
  • Is the workload read-heavy, write-heavy, or balanced?
  • What is the data consistency requirement (strong, eventual, causal)?
  • Are there existing infrastructure constraints (e.g., cloud provider, hardware)?
  • What is the acceptable downtime or performance degradation during scaling?
Out of Scope
✗ Designing specific application business logic or domain models
✗ Security and authentication mechanisms
✗ Detailed network infrastructure design (e.g., CDN, load balancers)
✗ Data consistency models beyond eventual consistency assumptions
📌
Assumptions
  • 2 billion DAU
  • 40 messages per user per day
  • 100 bytes per message
  • Peak traffic concentrated in 6 hours daily (25% of daily writes)
⚡
QPS Calculation
write qps2B × 40 / 86400 ≈ 925,926 writes/sec average
peak write qps925,926 × 4 (peak factor) ≈ 3.7 million writes/sec
read qpsAssuming 10 reads per write, 3.7M × 10 = 37 million reads/sec peak
💾
Storage Estimate
daily storage2B × 40 × 100 bytes = 8 TB/day raw
retention days30
total storage8 TB × 30 = 240 TB
with replication240 TB × 3 (replication factor) = 720 TB total
🖥️
Server Estimate
write db nodesAssuming Cassandra node handles 50K writes/sec, need 3.7M / 50K = 74 nodes
read cache nodesAssuming Redis node handles 100K reads/sec, need 37M / 100K = 370 nodes
🏗️
Architecture Diagram
Architecture Diagram
          +----------------------+
          |      Load Balancer    |
          +----------+-----------+
                     |
          +----------+-----------+
          |                      |
   +------+-------+      +-------+-------+
   | Vertical     |      | Horizontal    |
   | Scaling      |      | Scaling       |
   | (Single Big  |      | (Distributed  |
   | Server/Node) |      | Cluster)      |
   +------+-------+      +-------+-------+
          |                      |
   +------+-------+      +-------+-------+
   | Database     |      | Cassandra     |
   | (Vertical    |      | Cluster       |
   | Scaling DB)  |      |               |
   +--------------+      +---------------+
          |                      |
   +------+-------+      +-------+-------+
   | Cache Layer  |      | Redis Cluster |
   +--------------+      +---------------+
          |                      |
   +------+-------+      +-------+-------+
   | Application  |      | Application   |
   | Servers      |      | Servers       |
   +--------------+      +---------------+
🧩
Components
Vertical Scaling DB
Single powerful database server scaled up with more CPU, RAM, and SSD
Why thisUsing a vertically scaled PostgreSQL or MySQL instance simplifies consistency and management by avoiding distributed complexity; suitable for moderate scale
Why notCassandra rejected because it requires horizontal sharding and eventual consistency, adding complexity at moderate scale
Cassandra Cluster
Horizontally scalable distributed NoSQL database for write-heavy workloads
Why thisApache Cassandra provides linear write scalability with tunable consistency; handles millions of writes/sec by adding nodes
Why notPostgreSQL rejected due to complex manual sharding and B-tree write bottlenecks at >925K writes/sec
Redis Cluster
Distributed in-memory cache to serve high read QPS with low latency
Why thisRedis Cluster supports horizontal scaling with sharding and replication; sub-millisecond reads reduce DB load
Why notMemcached rejected because it lacks built-in clustering and persistence, limiting scale and reliability
Load Balancer
Distributes incoming requests to application servers and scaling tiers
Why thisUsing NGINX or HAProxy supports health checks and routing policies to balance load and isolate failures
Why notDNS round-robin rejected due to lack of health awareness and uneven load distribution
Application Servers
Process user requests, apply business logic, and interact with DB/cache layers
Why thisStateless design enables horizontal scaling by adding/removing servers dynamically; implemented with stateless microservices frameworks like Spring Boot or Node.js
Why notStateful monolith rejected as it limits horizontal scaling and complicates failover
🔄
Data Flow
1
User → Load Balancer
Send request
Load balancer routes request based on load and health
2
Load Balancer → Application Servers
Forward request
Application servers are stateless and horizontally scaled
3
Application Servers → Cache Layer (Redis Cluster)
Check cache for data
Cache hit reduces DB load and latency
4
Application Servers → Database (Vertical or Cassandra Cluster)
Read/write data if cache miss or write operation
Vertical DB used for moderate scale; Cassandra for high scale
5
Database → Cache Layer
Update cache asynchronously
Ensures cache consistency with eventual consistency
6
Application Servers → User
Return response
Response latency depends on cache hit and DB performance
🗄️
UserMessages
PK: user_id
user_idmessage_idmessage_contentmessage_timestamp
Stores messages per user ordered by timestamp for efficient retrieval
🗄️
UserProfiles
PK: user_id
user_idnameemailcreated_at
Stores user profile information
🔌
POST
/messages/send
Request
{
  "user_id": "string",
  "message_content": "string"
}
Response
{
  "status": "string",
  "message_id": "string"
}
API to send a message for a user
🔌
GET
/messages/fetch
Request
{
  "user_id": "string",
  "limit": "int"
}
Response
{
  "messages": "array of message objects"
}
API to fetch recent messages for a user
🔌
GET
/users/profile
Request
{
  "user_id": "string"
}
Response
{
  "user_profile": "object"
}
API to fetch user profile information

Practice

(1/5)
1. In a chat system, at step 4 the Delivery Worker checks Bob's presence TTL in Redis before delivering a message. What happens if Bob's presence key has expired at this step?
easy
A. The Delivery Worker delivers the message immediately to Bob's device
B. The Delivery Worker queues the message for offline delivery and marks it as undelivered
C. The Delivery Worker deletes the message assuming Bob is offline permanently
D. The Delivery Worker retries checking Bob's presence key indefinitely

Solution

  1. Step 1: Trace Delivery Worker decision at Step 4 -- Bob's presence TTL

    If Bob's presence key expired, it means Bob is offline. The worker should queue the message for offline delivery and mark it undelivered.

  2. Step 2: Eliminate wrong states

    Delivering immediately (The Delivery Worker delivers the message immediately to Bob's device) ignores offline status. Deleting message (The Delivery Worker deletes the message assuming Bob is offline permanently) causes data loss. Retrying indefinitely (The Delivery Worker retries checking Bob's presence key indefinitely) blocks delivery pipeline.

  3. Final Answer:

    Option B -> Option B

  4. Quick Check:

    Expired presence -> queue for offline delivery [check]
Hint: Expired presence means offline; queue message for later [check]
Common Mistakes:
2. In a consistent hashing system, at Step 4 the request router receives a key and queries the hash ring to find the responsible shard. What happens if the shard mapped by the hash ring is currently down?
easy
A. The router retries the same shard until it recovers before forwarding the request
B. The router uses the next clockwise shard on the hash ring as a fallback for the key
C. The router returns an error immediately to the client indicating shard unavailability
D. The router hashes the key again with a different hash function to find another shard

Solution

  1. Step 1: Trace request router behavior on shard failure

    Retrying the same shard (The router retries the same shard until it recovers before forwarding the request) blocks requests; returning error (The router returns an error immediately to the client indicating shard unavailability) reduces availability; next clockwise shard (The router uses the next clockwise shard on the hash ring as a fallback for the key) is correct only if the ring is intact and shard failure is handled by virtual nodes.

  2. Step 2: Understand fallback hashing

    Rehashing with a different function (The router hashes the key again with a different hash function to find another shard) is a common fallback to find an alternate shard when the primary is down, ensuring availability without blocking or errors.

  3. Final Answer:

    Option D -> Option D

  4. Quick Check:

    Fallback hashing finds alternate shard on failure [check]
Hint: Fallback hashing finds alternate shard if primary is down [OK]
Common Mistakes:
3. If the prefix hash map component fails and becomes unavailable, what is the immediate impact on the autocomplete system's behavior?
medium
A. The system returns no results for prefix queries until the prefix hash map recovers
B. The Trie component automatically handles prefix queries without any degradation in performance
C. Autocomplete queries fall back to Elasticsearch but with increased latency and less precise prefix matching
D. User input is cached locally on the client until the prefix hash map is restored

Solution

  1. Step 1: Understand component roles

    The prefix hash map is an optimization for O(1) prefix lookups, but the Trie is the primary data structure for prefix matching.

  2. Step 2: Trace failure impact

    If the prefix hash map fails, the system falls back to the Trie for prefix queries, which still works but may have slightly higher latency.

  3. Final Answer:

    Option B -> Option B

  4. Quick Check:

    Trie fallback ensures prefix queries continue despite prefix hash map failure [check]
Hint: Trie fallback keeps prefix search alive if prefix hash fails [check]
Common Mistakes:
4. At 10x scale, which component in a cache stampede prevention system is most likely to fail first, and why?
hard
A. Cache servers, because they run out of memory storing more keys
B. Backend database, because it cannot handle the increased read load from cache misses
C. Distributed lock service, because lock contention and coordination overhead grow exponentially
D. Load balancers, because they cannot distribute requests evenly at high concurrency

Solution

  1. Step 1: Identify bottlenecks at 10x scale

    Backend and cache scale horizontally; load balancers scale well.

  2. Step 2: Analyze distributed lock service

    Lock service faces increased contention and coordination overhead, causing latency and failures.

  3. Final Answer:

    Option C -> Option C

  4. Quick Check:

    Distributed lock service bottlenecks first due to coordination overhead [check]
Hint: Lock service bottlenecks first due to exponential coordination overhead [OK]
Common Mistakes:
5. At 10x scale, which component of a distributed Count-Min Sketch system is most likely to fail first and why?
hard
A. The network bandwidth saturates due to large sketch state synchronization
B. The sketch arrays in memory overflow due to fixed size limits
C. The hash functions become slower, increasing latency linearly
D. The centralized coordinator bottlenecks due to serializing all update requests

Solution

  1. Step 1: Identify bottleneck in distributed updates

    Centralized coordinator serializes updates to maintain consistency, limiting throughput.

  2. Step 2: Analyze other components

    Network bandwidth and hash function speed scale better; sketch arrays have fixed size but don't overflow.

  3. Step 3: Understand scale impact

    At 10x scale, coordinator becomes overwhelmed, causing update latency and failures.

  4. Final Answer:

    Option D -> Option D

  5. Quick Check:

    Central coordinator serialization bottlenecks updates at scale [check]
Hint: Central coordinator serializes updates, bottlenecking at scale [check]
Common Mistakes:
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard