Reverse proxy concept in HLD - Scalability & System Analysis

Scalability Analysis - Reverse proxy concept

Growth Table: Reverse Proxy Scaling

Users	Requests per Second	Reverse Proxy Load	Network Traffic	Backend Impact
100	~10-50	Single reverse proxy server handles all requests easily	Low bandwidth, minimal latency	Backend servers lightly loaded
10,000	~1,000-5,000	Single reverse proxy may reach CPU/network limits	Moderate bandwidth, some latency possible	Backend servers under moderate load
1,000,000	~100,000-500,000	Single reverse proxy insufficient; need load balancing and multiple proxies	High bandwidth, potential network bottlenecks	Backend servers require scaling and caching
100,000,000	~10,000,000+	Distributed reverse proxy clusters with global load balancing	Very high bandwidth; CDN integration essential	Backend heavily sharded and cached

First Bottleneck

The reverse proxy server's CPU and network bandwidth become the first bottleneck as traffic grows beyond a few thousand requests per second. A single proxy can only handle limited concurrent connections and throughput before latency increases and requests queue up.

Scaling Solutions

• Horizontal scaling: Add multiple reverse proxy servers behind a load balancer to distribute incoming traffic.
• Load balancing: Use DNS round-robin or dedicated load balancers to spread requests evenly.
• Caching: Implement caching at the reverse proxy to reduce backend load and speed up responses.
• CDN integration: Offload static content delivery to Content Delivery Networks to reduce proxy and backend load.
• Connection pooling and keep-alive: Optimize connections between proxy and backend to reduce overhead.

Back-of-Envelope Cost Analysis

At 10,000 users generating ~5,000 requests/sec, a single reverse proxy server with 1 Gbps network and 8 CPU cores can handle the load.

At 1 million users (~500,000 requests/sec), you need ~100 reverse proxy servers (assuming 5,000 req/sec per server) and ~125 Gbps network bandwidth.

Storage is minimal at the proxy layer but caching requires memory (e.g., 64GB RAM per proxy for effective caching).

Interview Tip

Start by explaining the role of a reverse proxy in handling client requests and protecting backend servers. Then discuss how it scales with traffic growth, identifying bottlenecks like CPU and bandwidth. Finally, propose clear scaling solutions such as horizontal scaling, caching, and CDN integration, justifying each step with realistic capacity numbers.

Self Check

Your reverse proxy handles 1,000 requests per second. Traffic grows 10x to 10,000 requests per second. What do you do first?

Answer: Add more reverse proxy servers behind a load balancer to distribute the increased traffic and prevent CPU/network bottlenecks on a single proxy.

Key Result

A single reverse proxy server handles low to moderate traffic, but CPU and network bandwidth become bottlenecks as traffic grows. Horizontal scaling with multiple proxies and caching are key to scaling effectively.

Practice

1. What is the primary role of a reverse proxy in a web system?

easy

A. It stores data permanently like a database.

B. It receives client requests and forwards them to backend servers.

C. It acts as a client to send requests to other servers.

D. It directly serves static files without contacting servers.

Solution

1. Step 1: Understand the role of reverse proxy

   A reverse proxy acts as an intermediary between clients and backend servers, receiving requests from clients.

2. Step 2: Identify what it does with requests

   It forwards these requests to the appropriate backend servers for processing, then returns the response to the client.

3. Final Answer:

   It receives client requests and forwards them to backend servers. -> Option B

4. Quick Check:

   Reverse proxy role = Forward requests [OK]

Hint: Reverse proxy forwards client requests to servers [OK]

Common Mistakes:

• Confusing reverse proxy with database
• Thinking reverse proxy acts as a client
• Assuming it serves files directly without servers

2. Which of the following is a correct function of a reverse proxy?

easy

A. Load balancing incoming client requests across servers.

B. Storing user session data permanently.

C. Generating dynamic web pages on the server.

D. Directly connecting clients to databases.

Solution

1. Step 1: Identify reverse proxy functions

   Reverse proxies distribute incoming client requests to multiple backend servers to balance load and improve performance.

2. Step 2: Eliminate incorrect options

   Storing session data, generating pages, or direct DB connections are not reverse proxy tasks.

3. Final Answer:

   Load balancing incoming client requests across servers. -> Option A

4. Quick Check:

   Reverse proxy function = Load balancing [OK]

Hint: Reverse proxy balances load, not stores data [OK]

Common Mistakes:

• Confusing reverse proxy with application server
• Thinking it stores session data
• Assuming it generates web pages

3. Consider a system where a reverse proxy receives 1000 requests per second and forwards them evenly to 4 backend servers. How many requests per second does each backend server receive?

medium

A. 4 requests per second

B. 4000 requests per second

C. 250 requests per second

D. 1000 requests per second

Solution

1. Step 1: Understand load distribution

   The reverse proxy evenly splits 1000 requests among 4 servers, so each gets an equal share.

2. Step 2: Calculate requests per server

   1000 requests ÷ 4 servers = 250 requests per server per second.

3. Final Answer:

   250 requests per second -> Option C

4. Quick Check:

   1000 ÷ 4 = 250 [OK]

Hint: Divide total requests by number of servers [OK]

Common Mistakes:

• Multiplying instead of dividing
• Confusing total with per server requests
• Ignoring even distribution

4. A reverse proxy is configured to cache responses but clients report stale data. What is the most likely cause?

medium

A. The cache expiration time is set too long.

B. The reverse proxy is not forwarding requests.

C. Backend servers are down.

D. Clients are bypassing the reverse proxy.

Solution

1. Step 1: Analyze caching behavior

   If clients get stale data, it means cached responses are not refreshed frequently enough.

2. Step 2: Identify cause of stale cache

   Setting cache expiration too long causes the reverse proxy to serve old data instead of fresh responses.

3. Final Answer:

   The cache expiration time is set too long. -> Option A

4. Quick Check:

   Long cache time = stale data [OK]

Hint: Check cache expiration for stale data issues [OK]

Common Mistakes:

• Assuming backend servers are down without checking
• Thinking reverse proxy stops forwarding requests
• Ignoring cache settings

5. You want to design a scalable web system using a reverse proxy that handles SSL termination, load balancing, and caching. Which of the following is the best architectural choice?

hard

A. Place the reverse proxy behind backend servers to handle load balancing.

B. Let each backend server handle SSL and caching individually without a reverse proxy.

C. Use a reverse proxy only for caching, and let clients connect directly to backend servers for SSL.

D. Use a reverse proxy in front of multiple backend servers, configure SSL termination and caching on it.

Solution

1. Step 1: Understand reverse proxy capabilities

   A reverse proxy can handle SSL termination, load balancing, and caching centrally, simplifying backend servers.

2. Step 2: Evaluate architectural options

   Placing the reverse proxy in front of backend servers allows it to manage SSL and cache efficiently, improving scalability and security.

3. Final Answer:

   Use a reverse proxy in front of multiple backend servers, configure SSL termination and caching on it. -> Option D

4. Quick Check:

   Reverse proxy front = SSL, load balance, cache [OK]

Hint: Put reverse proxy in front for SSL and load balance [OK]

Common Mistakes:

• Letting backend servers handle SSL separately
• Placing reverse proxy behind servers
• Splitting SSL and caching between clients and servers