Why distributed patterns solve common challenges in HLD - Scalability Evidence

Scalability Analysis - Why distributed patterns solve common challenges

Growth Table: What Changes as Users and Data Grow

Scale	Users	Data Volume	Traffic	System Behavior
Small	100 users	Low (MBs)	Low (few QPS)	Single server handles all; simple design
Medium	10,000 users	Moderate (GBs)	Moderate (hundreds QPS)	Server CPU/memory stressed; DB load increases
Large	1,000,000 users	High (TBs)	High (thousands QPS)	Single server insufficient; DB bottleneck; latency rises
Very Large	100,000,000 users	Very High (PBs)	Very High (hundreds of thousands QPS)	Need distributed systems; partitioning; multi-region

First Bottleneck: Why Single Systems Break

At small scale, a single server and database can handle all requests. As users grow, the database becomes the first bottleneck because it can only process a limited number of queries per second (usually up to 5,000-10,000 QPS for a single instance). CPU and memory on the application server also get stressed as traffic increases. Network bandwidth and storage limits appear at very large scale.

Distributed patterns solve these by spreading load across many machines, avoiding single points of failure and scaling horizontally.

Scaling Solutions Using Distributed Patterns

• Horizontal Scaling: Add more servers to share load, improving capacity and fault tolerance.
• Load Balancing: Distribute incoming requests evenly to prevent overload on any one server.
• Database Replication: Use read replicas to handle read-heavy traffic, reducing load on primary DB.
• Sharding: Split data across multiple databases by key ranges or hashes to handle large data volumes.
• Caching: Store frequent data in fast memory (e.g., Redis) to reduce database hits.
• Message Queues: Decouple components and smooth traffic spikes by asynchronous processing.
• CDNs: Cache static content closer to users to reduce bandwidth and latency.
• Multi-region Deployment: Place servers near users to reduce latency and improve availability.

Back-of-Envelope Cost Analysis

Example for 1 million users with 1 request per second each:

• Requests per second: 1,000,000 QPS (too high for single DB)
• Database capacity: Single DB ~10,000 QPS -> Need ~100 DB shards or replicas
• Network bandwidth: 1 Gbps = 125 MB/s; 1M QPS with 1 KB payload = ~1 GB/s -> Need multiple network links
• Storage: TBs of data requiring distributed storage solutions
• Servers: Hundreds of app servers behind load balancers

Interview Tip: Structuring Scalability Discussion

Start by describing current system limits at small scale. Identify the first bottleneck as traffic grows. Explain why that component breaks (e.g., DB QPS limit). Then propose distributed solutions step-by-step, explaining how each solves a specific problem. Use real numbers to justify your choices. Finally, mention trade-offs and monitoring needs.

Self-Check Question

Question: Your database handles 1000 QPS. Traffic grows 10x to 10,000 QPS. What do you do first and why?

Answer: The first step is to add read replicas to offload read queries from the primary database. This reduces load and increases read capacity. If writes also increase, consider sharding data or scaling the database vertically or horizontally.

Key Result

Distributed patterns help systems grow by spreading load and data across many machines, avoiding single points of failure and bottlenecks as user count and traffic increase.

Practice

1. Which of the following best explains why distributed patterns are used in system design?

easy

A. They split work across machines to improve speed and reliability.

B. They reduce the number of users a system can handle.

C. They make systems more complex without benefits.

D. They centralize data to a single machine for simplicity.

Solution

1. Step 1: Understand the purpose of distributed patterns

   Distributed patterns divide tasks among multiple machines to improve performance and fault tolerance.

2. Step 2: Compare options with this understanding

   Only They split work across machines to improve speed and reliability. correctly states the benefit of splitting work to improve speed and reliability.

3. Final Answer:

   They split work across machines to improve speed and reliability. -> Option A

4. Quick Check:

   Distributed patterns improve speed and reliability = A [OK]

Hint: Distributed means spreading work to improve speed and reliability [OK]

Common Mistakes:

• Thinking distributed means fewer users can be handled
• Assuming distributed patterns add complexity without benefits
• Believing data is centralized in distributed systems

2. Which of the following is a correct example of a distributed pattern used to balance user requests?

easy

A. Load balancing

B. Single-threading

C. Monolithic deployment

D. Local caching only

Solution

1. Step 1: Identify patterns that distribute user requests

   Load balancing distributes incoming requests across multiple servers to avoid overload.

2. Step 2: Eliminate incorrect options

   Single-threading and monolithic deployment do not distribute load; local caching helps speed but not load distribution.

3. Final Answer:

   Load balancing -> Option A

4. Quick Check:

   Load balancing distributes requests = B [OK]

Hint: Load balancing spreads requests evenly across servers [OK]

Common Mistakes:

• Confusing single-threading with load distribution
• Thinking monolithic means distributed
• Assuming caching balances load

3. Consider a system using sharding to split a database into parts. What is the main benefit of this approach?

medium

A. It reduces the total data stored.

B. It duplicates all data on every server.

C. It centralizes data for easier management.

D. It improves query speed by parallelizing data access.

Solution

1. Step 1: Understand sharding in distributed systems

   Sharding splits data into smaller parts stored on different servers to allow parallel access.

2. Step 2: Analyze options based on sharding benefits

   Sharding does not reduce total data or centralize it; it improves speed by parallel queries.

3. Final Answer:

   It improves query speed by parallelizing data access. -> Option D

4. Quick Check:

   Sharding speeds queries by splitting data = D [OK]

Hint: Sharding splits data to speed up queries by parallel access [OK]

Common Mistakes:

• Thinking sharding reduces total data stored
• Believing sharding centralizes data
• Confusing sharding with replication

4. A system uses replication to copy data across servers. If one server fails, users still experience downtime. What is the likely problem?

medium

A. Replication slows down the system.

B. Replication duplicates data too many times.

C. Replication is not configured for failover.

D. Replication centralizes data on one server.

Solution

1. Step 1: Understand replication and failover

   Replication copies data to multiple servers to provide backup if one fails, but failover must be configured to switch users automatically.

2. Step 2: Identify why downtime occurs despite replication

   If users face downtime, failover is likely missing or misconfigured, so traffic doesn't switch to healthy servers.

3. Final Answer:

   Replication is not configured for failover. -> Option C

4. Quick Check:

   Failover missing causes downtime despite replication = C [OK]

Hint: Replication needs failover setup to avoid downtime [OK]

Common Mistakes:

• Assuming replication alone prevents downtime
• Thinking too many copies cause downtime
• Believing replication centralizes data

5. You design a global e-commerce platform expecting millions of users. Which combination of distributed patterns best solves challenges of speed, reliability, and data consistency?

hard

A. Single server for all data, caching only, no replication.

B. Load balancing for requests, replication for reliability, sharding for data scaling.

C. Replication only without load balancing or sharding.

D. Sharding only without replication or load balancing.

Solution

1. Step 1: Identify challenges in a global platform

   Speed requires spreading requests (load balancing), reliability needs data copies (replication), and scaling needs data partitioning (sharding).

2. Step 2: Match patterns to challenges

   Load balancing for requests, replication for reliability, sharding for data scaling. combines all three patterns to address speed, reliability, and scaling effectively.

3. Step 3: Eliminate incomplete options

   Options B, C, and D miss one or more key patterns, risking bottlenecks or failures.

4. Final Answer:

   Load balancing for requests, replication for reliability, sharding for data scaling. -> Option B

5. Quick Check:

   Combine load balancing, replication, sharding = A [OK]

Hint: Use load balancing, replication, and sharding together for big systems [OK]

Common Mistakes:

• Relying on single server for millions of users
• Using only one pattern and ignoring others
• Confusing replication with sharding roles