Reverse proxy concept in HLD - System Design Exercise

Design: Reverse Proxy System

Design focuses on the reverse proxy component between clients and backend servers. Does not cover backend server internals or client applications.

Functional Requirements

FR1: Accept client requests and forward them to appropriate backend servers

FR2: Hide backend server details from clients

FR3: Distribute incoming traffic to multiple backend servers for load balancing

FR4: Cache responses to improve performance

FR5: Provide security features like filtering and SSL termination

FR6: Handle failures of backend servers gracefully

Non-Functional Requirements

NFR1: Must handle 10,000 concurrent client connections

NFR2: API response latency p99 should be under 150ms

NFR3: Availability target of 99.9% uptime

NFR4: Support HTTPS connections from clients

NFR5: Support backend servers running HTTP

Think Before You Design

Questions to Ask

❓ Question 1

❓ Question 2

❓ Question 3

❓ Question 4

❓ Question 5

❓ Question 6

Key Components

Listener to accept client connections

Load balancer to distribute requests

Cache layer for storing responses

Health checker for backend servers

SSL termination module

Logging and monitoring system

Design Patterns

Load balancing patterns

Caching strategies (write-through, write-back)

Circuit breaker for backend failures

SSL termination and offloading

Reverse proxy as a gateway pattern

Reference Architecture

Client
  |
  v
Reverse Proxy
  |-- Listener (HTTPS)
  |-- SSL Termination
  |-- Load Balancer
  |-- Cache
  |-- Health Checker
  |-- Security Filters
  |
  v
Backend Servers (HTTP)

Components

Listener

Nginx or HAProxy

Accepts incoming client connections over HTTPS

SSL Termination

OpenSSL library integrated in proxy

Decrypts HTTPS requests to HTTP for backend communication

Load Balancer

Round robin or least connections algorithm

Distributes requests evenly across backend servers

Cache

In-memory cache like Redis or built-in proxy cache

Stores frequently requested responses to reduce backend load

Health Checker

Periodic HTTP checks

Monitors backend server availability and removes unhealthy servers

Security Filters

IP filtering, rate limiting modules

Protects backend servers from malicious or excessive requests

Request Flow

1. Client sends HTTPS request to reverse proxy listener

2. SSL Termination decrypts request to HTTP

3. Load Balancer selects a healthy backend server

4. If cached response exists, return it directly to client

5. Otherwise, forward request to backend server

6. Backend server processes and sends response to proxy

7. Cache stores response if eligible

8. Reverse proxy encrypts response and sends back to client

Database Schema

Not applicable as reverse proxy typically does not store persistent data. Cache is in-memory or external store like Redis.

Scaling Discussion

Bottlenecks

Single reverse proxy instance limits concurrent connections

Cache size and eviction policy may limit hit rate

Load balancer algorithm may cause uneven load

SSL termination CPU overhead under high traffic

Health checker frequency impacts detection speed

Solutions

Deploy multiple reverse proxy instances behind a DNS load balancer

Use distributed cache with sharding and eviction policies

Implement adaptive load balancing based on server load

Use hardware SSL accelerators or offload SSL to dedicated devices

Tune health check intervals and use asynchronous checks

Interview Tips

Time: Spend 10 minutes understanding requirements and clarifying scope, 20 minutes designing components and data flow, 10 minutes discussing scaling and trade-offs, 5 minutes summarizing.

Explain how reverse proxy improves security and scalability

Discuss SSL termination benefits and challenges

Describe load balancing strategies and cache usage

Highlight failure handling with health checks

Mention scaling techniques and bottleneck mitigation

Practice

1. What is the primary role of a reverse proxy in a web system?

easy

A. It stores data permanently like a database.

B. It receives client requests and forwards them to backend servers.

C. It acts as a client to send requests to other servers.

D. It directly serves static files without contacting servers.

Solution

1. Step 1: Understand the role of reverse proxy

   A reverse proxy acts as an intermediary between clients and backend servers, receiving requests from clients.

2. Step 2: Identify what it does with requests

   It forwards these requests to the appropriate backend servers for processing, then returns the response to the client.

3. Final Answer:

   It receives client requests and forwards them to backend servers. -> Option B

4. Quick Check:

   Reverse proxy role = Forward requests [OK]

Hint: Reverse proxy forwards client requests to servers [OK]

Common Mistakes:

• Confusing reverse proxy with database
• Thinking reverse proxy acts as a client
• Assuming it serves files directly without servers

2. Which of the following is a correct function of a reverse proxy?

easy

A. Load balancing incoming client requests across servers.

B. Storing user session data permanently.

C. Generating dynamic web pages on the server.

D. Directly connecting clients to databases.

Solution

1. Step 1: Identify reverse proxy functions

   Reverse proxies distribute incoming client requests to multiple backend servers to balance load and improve performance.

2. Step 2: Eliminate incorrect options

   Storing session data, generating pages, or direct DB connections are not reverse proxy tasks.

3. Final Answer:

   Load balancing incoming client requests across servers. -> Option A

4. Quick Check:

   Reverse proxy function = Load balancing [OK]

Hint: Reverse proxy balances load, not stores data [OK]

Common Mistakes:

• Confusing reverse proxy with application server
• Thinking it stores session data
• Assuming it generates web pages

3. Consider a system where a reverse proxy receives 1000 requests per second and forwards them evenly to 4 backend servers. How many requests per second does each backend server receive?

medium

A. 4 requests per second

B. 4000 requests per second

C. 250 requests per second

D. 1000 requests per second

Solution

1. Step 1: Understand load distribution

   The reverse proxy evenly splits 1000 requests among 4 servers, so each gets an equal share.

2. Step 2: Calculate requests per server

   1000 requests ÷ 4 servers = 250 requests per server per second.

3. Final Answer:

   250 requests per second -> Option C

4. Quick Check:

   1000 ÷ 4 = 250 [OK]

Hint: Divide total requests by number of servers [OK]

Common Mistakes:

• Multiplying instead of dividing
• Confusing total with per server requests
• Ignoring even distribution

4. A reverse proxy is configured to cache responses but clients report stale data. What is the most likely cause?

medium

A. The cache expiration time is set too long.

B. The reverse proxy is not forwarding requests.

C. Backend servers are down.

D. Clients are bypassing the reverse proxy.

Solution

1. Step 1: Analyze caching behavior

   If clients get stale data, it means cached responses are not refreshed frequently enough.

2. Step 2: Identify cause of stale cache

   Setting cache expiration too long causes the reverse proxy to serve old data instead of fresh responses.

3. Final Answer:

   The cache expiration time is set too long. -> Option A

4. Quick Check:

   Long cache time = stale data [OK]

Hint: Check cache expiration for stale data issues [OK]

Common Mistakes:

• Assuming backend servers are down without checking
• Thinking reverse proxy stops forwarding requests
• Ignoring cache settings

5. You want to design a scalable web system using a reverse proxy that handles SSL termination, load balancing, and caching. Which of the following is the best architectural choice?

hard

A. Place the reverse proxy behind backend servers to handle load balancing.

B. Let each backend server handle SSL and caching individually without a reverse proxy.

C. Use a reverse proxy only for caching, and let clients connect directly to backend servers for SSL.

D. Use a reverse proxy in front of multiple backend servers, configure SSL termination and caching on it.

Solution

1. Step 1: Understand reverse proxy capabilities

   A reverse proxy can handle SSL termination, load balancing, and caching centrally, simplifying backend servers.

2. Step 2: Evaluate architectural options

   Placing the reverse proxy in front of backend servers allows it to manage SSL and cache efficiently, improving scalability and security.

3. Final Answer:

   Use a reverse proxy in front of multiple backend servers, configure SSL termination and caching on it. -> Option D

4. Quick Check:

   Reverse proxy front = SSL, load balance, cache [OK]

Hint: Put reverse proxy in front for SSL and load balance [OK]

Common Mistakes:

• Letting backend servers handle SSL separately
• Placing reverse proxy behind servers
• Splitting SSL and caching between clients and servers