Why scalability handles growing traffic in HLD - Scalability Evidence

Scalability Analysis - Why scalability handles growing traffic

Growth Table: What Changes as Traffic Grows

Users/Traffic	Requests per Second	Server Load	Database Load	Network Bandwidth	Storage Needs
100 users	~10-50 RPS	Single server handles easily	Single DB instance sufficient	Low bandwidth usage	Minimal storage
10,000 users	~1,000-5,000 RPS	Multiple app servers needed	DB nearing capacity, may need read replicas	Moderate bandwidth usage	Growing storage, backups needed
1,000,000 users	~100,000 RPS	Horizontal scaling essential	DB sharding or distributed DB needed	High bandwidth, CDN recommended	Large storage, tiered storage useful
100,000,000 users	~10,000,000 RPS	Massive cluster of servers	Multiple shards, distributed DB clusters	Very high bandwidth, global CDN	Petabytes of storage, archival systems

First Bottleneck: What Breaks First and Why

At low to medium traffic, the database is usually the first bottleneck. This is because it handles all data reads and writes, and has limited query throughput (typically 5,000-10,000 queries per second for a single instance). As traffic grows, the DB CPU, memory, and disk I/O get saturated first.

At higher traffic, application servers can become CPU or memory bottlenecks due to processing many concurrent requests. Network bandwidth can also become a bottleneck when serving large amounts of data or media.

Scaling Solutions

• Database: Use read replicas to spread read load, connection pooling to reduce overhead, and sharding to split data across multiple DB instances.
• Application Servers: Add more servers horizontally behind load balancers to distribute traffic evenly.
• Caching: Use in-memory caches like Redis or Memcached to reduce DB load for frequent queries.
• Content Delivery Network (CDN): Offload static content delivery to edge servers closer to users, reducing bandwidth and latency.
• Storage: Use tiered storage and archival for old data to reduce expensive fast storage usage.

Back-of-Envelope Cost Analysis

• At 10,000 users generating ~5,000 RPS, a single DB instance near max capacity (5,000-10,000 QPS) may need read replicas.
• Each server can handle ~1,000-5,000 concurrent connections; so 2-5 app servers needed at this scale.
• Network bandwidth at 1 Gbps (~125 MB/s) can handle roughly 10,000-20,000 requests per second depending on payload size.
• Storage grows with data; for example, 1 million users might generate terabytes of data requiring distributed storage.

Interview Tip: Structuring Scalability Discussion

Start by understanding current traffic and system limits. Identify the first bottleneck clearly (usually DB). Discuss how traffic growth affects each component. Propose targeted solutions like caching, read replicas, horizontal scaling. Mention trade-offs and cost implications. Use real numbers to show understanding.

Self Check Question

Question: Your database handles 1000 QPS. Traffic grows 10x to 10,000 QPS. What do you do first and why?

Answer: The first step is to add read replicas to distribute read queries and reduce load on the primary database. This helps handle increased read traffic without immediate costly sharding or redesign.

Key Result

The database is usually the first bottleneck as traffic grows; scaling it with read replicas, caching, and sharding is key to handling increasing user requests efficiently.

Practice

1. What does scalability in system design primarily help with?

easy

A. Handling more users and traffic smoothly

B. Reducing the number of servers

C. Making the system slower

D. Limiting the system to one user

Solution

1. Step 1: Understand the meaning of scalability

   Scalability means the system can grow to handle more users or traffic without problems.

2. Step 2: Identify the main goal of scalability

   The main goal is to manage increasing traffic smoothly, not to reduce servers or slow down.

3. Final Answer:

   Handling more users and traffic smoothly -> Option A

4. Quick Check:

   Scalability = Handling more traffic [OK]

Hint: Scalability means growing capacity to handle more users [OK]

Common Mistakes:

• Thinking scalability reduces servers
• Confusing scalability with slowing system
• Believing scalability limits users

2. Which option correctly describes vertical scaling?

easy

A. Upgrading the hardware of a single machine

B. Distributing traffic using load balancers

C. Adding more machines to the system

D. Reducing the number of users

Solution

1. Step 1: Define vertical scaling

   Vertical scaling means improving one machine's hardware, like adding CPU or memory.

2. Step 2: Differentiate from other options

   Adding machines is horizontal scaling; load balancers distribute traffic; reducing users is unrelated.

3. Final Answer:

   Upgrading the hardware of a single machine -> Option A

4. Quick Check:

   Vertical scaling = Upgrade one machine [OK]

Hint: Vertical scaling upgrades one machine's hardware [OK]

Common Mistakes:

• Confusing vertical with horizontal scaling
• Thinking vertical scaling adds machines
• Mixing load balancers with scaling types

3. Consider a system with 3 servers behind a load balancer. If traffic doubles, what happens when horizontal scaling is applied?

medium

A. Remove servers to reduce complexity

B. Upgrade the CPU of one server only

C. Add more servers to share the load

D. Ignore traffic increase and keep same setup

Solution

1. Step 1: Understand horizontal scaling

   Horizontal scaling means adding more servers to handle increased traffic.

2. Step 2: Apply to the scenario

   With doubled traffic, adding servers helps share load; upgrading one server is vertical scaling.

3. Final Answer:

   Add more servers to share the load -> Option C

4. Quick Check:

   Horizontal scaling = Add servers [OK]

Hint: Horizontal scaling means adding servers [OK]

Common Mistakes:

• Choosing to upgrade one server (vertical scaling)
• Removing servers when traffic grows
• Ignoring traffic increase

4. A system uses a load balancer but still crashes under high traffic. What is the likely issue?

medium

A. Too many servers are added unnecessarily

B. Load balancer is not distributing traffic evenly

C. Vertical scaling was applied correctly

D. Traffic is decreasing, causing overload

Solution

1. Step 1: Analyze load balancer role

   Load balancers distribute traffic evenly to servers to prevent overload.

2. Step 2: Identify cause of crash

   If system crashes despite load balancer, it may not be distributing traffic properly, causing some servers to overload.

3. Final Answer:

   Load balancer is not distributing traffic evenly -> Option B

4. Quick Check:

   Uneven load distribution causes crashes [OK]

Hint: Check if load balancer spreads traffic evenly [OK]

Common Mistakes:

• Assuming too many servers cause crash
• Thinking vertical scaling always fixes crashes
• Believing traffic decrease causes overload

5. A web app faces sudden traffic spikes. Which combination best ensures smooth scaling?

hard

A. Only upgrade one server's CPU and RAM

B. Limit user access to reduce traffic

C. Remove load balancers and rely on a single powerful server

D. Use horizontal scaling with load balancers and auto-scaling groups

Solution

1. Step 1: Identify best scaling approach for spikes

   Horizontal scaling with load balancers allows adding/removing servers dynamically to handle spikes.

2. Step 2: Evaluate other options

   Upgrading one server limits capacity; removing load balancers causes bottlenecks; limiting users reduces usability.

3. Final Answer:

   Use horizontal scaling with load balancers and auto-scaling groups -> Option D

4. Quick Check:

   Horizontal scaling + load balancers = smooth spikes [OK]

Hint: Combine horizontal scaling and load balancers for spikes [OK]

Common Mistakes:

• Relying only on vertical scaling
• Removing load balancers causing bottlenecks
• Limiting users instead of scaling