Bird
Raised Fist0

Load Balancer vs API Gateway vs Reverse Proxy - Key Differences

HLDmediumInterview freq: common
Choose your preparation mode4 modes available
ArchitectureTraceChallenge
🏗️
Load Balancer vs API Gateway vs Reverse Proxy - Key Differences
Understanding the distinct roles, scale challenges, and trade-offs between Load Balancers, API Gateways, and Reverse Proxies
mediumLoad BalancersFreq: common
⚙️
Functional Requirements
  • Distribute incoming client requests efficiently across multiple backend servers (Load Balancer).
  • Provide a single entry point for clients with request routing, authentication, rate limiting, and protocol translation (API Gateway).
  • Forward client requests to backend servers while optionally caching responses and handling SSL termination (Reverse Proxy).
  • Support health checks and failover to maintain high availability of backend services.
  • Enable protocol-level features such as HTTP/2 multiplexing and WebSocket proxying where applicable.
📊
Non-Functional Requirements
  • Handle peak traffic of at least 1 million requests per second (RPS) with <100ms P99 latency.
  • Achieve 99.99% uptime SLA for request routing components.
  • Support TLS termination with <5ms added latency per request.
  • Scale horizontally to support 10x traffic growth without downtime.
  • Provide observability with <1s delay for metrics and logs aggregation.
❓
Clarifying Questions to Ask
  • What is the expected peak and average request rate per second?
  • Are there specific protocols to support (HTTP/1.1, HTTP/2, gRPC, WebSocket)?
  • What authentication and authorization mechanisms are required at the gateway?
  • Is caching required at the reverse proxy layer, and what cache invalidation strategy?
  • What SLAs are expected for latency and availability?
  • Should the system support multi-region or global load balancing?
Out of Scope
✗ Designing backend application logic or microservices themselves.
✗ Implementing client-side SDKs or mobile app integration.
✗ Detailed security policies beyond authentication and rate limiting (e.g., WAF).
✗ Data storage or database design for backend services.
📌
Assumptions
  • User base: 100 million daily active users (DAU).
  • Average requests per user per day: 20 requests.
  • Average request size: 500 bytes.
  • Read to write ratio: 4:1 (mostly GET requests).
⚡
QPS Calculation
write qps100M × 20 / 86400 ≈ 23,148 requests/sec
peak write qpsAssuming 5× peak factor: 23,148 × 5 = 115,740 requests/sec
read qps4 × write_qps = 4 × 23,148 = 92,592 requests/sec
peak read qps92,592 × 5 = 462,960 requests/sec
💾
Storage Estimate
daily storage bytes100M users × 20 req × 500 bytes = 1,000,000,000,000 bytes = 1 TB/day
retention days7
total storage1 TB/day × 7 days = 7 TB
with replication7 TB × 3 replicas = 21 TB total storage
🖥️
Server Estimate
load balancerAssuming each NGINX instance handles 20,000 RPS, need 115,740 / 20,000 ≈ 6 instances at peak
api gatewayAssuming Kong Gateway handles 10,000 RPS with auth and rate limiting, need 115,740 / 10,000 ≈ 12 instances
🏗️
Architecture Diagram
Architecture Diagram
Client
  |
  v
Load Balancer (NGINX or HAProxy)
  |
  v
API Gateway (Kong or AWS API Gateway)
  |
  v
Reverse Proxy (NGINX or Envoy)
  |
  v
Backend Services (Microservices Cluster)
  |
  v
Databases / Storage
🧩
Components
Load Balancer
Distributes incoming client requests evenly across multiple API Gateway instances to prevent overload.
Why thisNGINX supports high-performance Layer 4 and Layer 7 load balancing with health checks and SSL termination at scale.
Why notAWS ELB is cloud-specific and less customizable; hardware load balancers like F5 are costly and less flexible.
API Gateway
Acts as a single entry point for clients, handling authentication, authorization, rate limiting, and request routing to microservices.
Why thisKong Gateway supports plugin-based extensibility, OAuth2, JWT auth, and scales horizontally with consistent configuration management.
Why notCustom-built gateways lack maturity and require significant maintenance; NGINX lacks built-in API management features.
Reverse Proxy
Forwards requests from API Gateway to backend services, optionally caching responses and handling protocol upgrades like WebSocket.
Why thisEnvoy Proxy supports advanced routing, retries, circuit breaking, and observability with low latency.
Why notSimple NGINX reverse proxy lacks dynamic service discovery and advanced resilience features.
Backend Services
Microservices that implement business logic, accessed via API Gateway and Reverse Proxy.
Why thisContainerized microservices orchestrated by Kubernetes enable independent scaling and deployment.
Why notMonolithic services reduce scalability and fault isolation.
Databases / Storage
Persist application data with replication and backup.
Why thisCassandra provides linear write scalability and high availability for large-scale data ingestion.
Why notRelational databases like PostgreSQL require complex sharding at this scale and have write bottlenecks.
🔄
Data Flow
1
Client → Load Balancer
Client sends HTTP/HTTPS request to Load Balancer IP
Load Balancer terminates TLS and performs health checks on API Gateway instances
2
Load Balancer → API Gateway
Load Balancer forwards request to a healthy API Gateway instance
Load Balancer uses round-robin or least connections algorithm
3
API Gateway → Reverse Proxy
API Gateway authenticates request, applies rate limiting, then routes to Reverse Proxy
API Gateway enforces security policies and request transformations
4
Reverse Proxy → Backend Services
Reverse Proxy forwards request to appropriate backend microservice
Reverse Proxy handles retries, caching, and protocol upgrades
5
Backend Services → Databases / Storage
Backend services read/write data as needed
Data persistence with replication ensures durability
6
Backend Services → Reverse Proxy
Backend services send response back to Reverse Proxy
Reverse Proxy may cache response if configured
7
Reverse Proxy → API Gateway
Reverse Proxy forwards response to API Gateway
API Gateway may apply response transformations before sending to client

Practice

(1/5)
1. A distributed database system needs to evenly distribute data across 100 shards and handle dynamic addition/removal of shards with minimal data movement. Which component is most appropriate to manage shard assignment?
easy
A. Use a modulo-based hash function on the primary key to assign shards directly
B. Implement consistent hashing with virtual nodes to map keys to shards
C. Assign shards based on a centralized lookup table updated on every shard change
D. Use range-based partitioning with fixed key ranges per shard

Solution

  1. Step 1: Understand shard assignment requirements

    Modulo hashing (Use a modulo-based hash function on the primary key to assign shards directly) causes massive remapping when shards change; centralized lookup (Assign shards based on a centralized lookup table updated on every shard change) creates a bottleneck and single point of failure; range partitioning (Use range-based partitioning with fixed key ranges per shard) can cause hotspots and uneven load.

  2. Step 2: Evaluate consistent hashing benefits

    Consistent hashing with virtual nodes (Implement consistent hashing with virtual nodes to map keys to shards) evenly distributes keys and minimizes remapping when shards are added or removed, making it ideal for dynamic shard management.

  3. Final Answer:

    Option B -> Option B

  4. Quick Check:

    Consistent hashing minimizes shard remapping on topology changes [check]
Hint: Consistent hashing minimizes data movement on shard changes [OK]
Common Mistakes:
2. A database needs to support range queries on a large numeric column with frequent inserts and updates. Which indexing method best supports efficient range scans while maintaining balanced tree structure?
easy
A. Composite index on multiple columns, to speed up multi-column equality queries
B. Hash index, because it provides constant-time lookups for any key
C. B-Tree index, because it maintains sorted order and supports efficient range queries
D. Bitmap index, because it compresses well for high-cardinality numeric data

Solution

  1. Step 1: Identify index type for range queries

    B-Tree indexes maintain sorted order of keys, enabling efficient range scans by traversing leaf nodes sequentially.

  2. Step 2: Eliminate other options

    Hash indexes do not preserve order, so range queries require scanning all buckets. Composite indexes optimize multi-column equality but don't inherently improve single-column range scans. Bitmap indexes are efficient for low-cardinality columns, not large numeric ranges.

  3. Final Answer:

    Option C -> Option C

  4. Quick Check:

    B-Tree supports ordered traversal for range queries [check]
Hint: B-Tree keeps keys sorted for range queries [check]
Common Mistakes:
3. For mobile clients with intermittent connectivity, which message sync strategy is best to minimize data usage while ensuring eventual consistency?
medium
A. Use delta-based sync with version vectors to fetch only changed messages on reconnect
B. Use WebSocket streaming for real-time sync regardless of connectivity
C. Use full message history sync on every reconnect to ensure consistency
D. Use polling every 5 seconds to fetch new messages

Solution

  1. Step 1: Evaluate sync strategies for intermittent mobile connectivity

    WebSocket streaming fails on unstable networks; polling wastes bandwidth; full sync is expensive.

  2. Step 2: Delta-based sync with version vectors minimizes data and ensures eventual consistency

    Only changed messages are fetched, reducing data usage and handling offline periods gracefully.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Delta sync + version vectors -> efficient, consistent mobile sync [check]
Hint: Delta sync with version vectors minimizes mobile data use [OK]
Common Mistakes:
4. At 10x scale, which component of the Snowflake ID generation system is most likely to become the bottleneck first, and why?
hard
A. The timestamp source, because clock synchronization across machines becomes impossible at scale.
B. The network bandwidth, because ID generation requires frequent coordination between machines.
C. The sequence number, because generating more than its max IDs per millisecond causes blocking delays.
D. The machine ID allocation, because limited machine ID bits cause ID collisions as machines increase.

Solution

  1. Step 1: Analyze timestamp and network constraints

    Timestamp synchronization (A) is challenging but tolerable with NTP; network coordination (D) is minimal as IDs are generated locally.

  2. Step 2: Evaluate sequence number limits

    Sequence number (B) limits per millisecond can cause blocking but can be mitigated by load balancing or sharding.

  3. Step 3: Identify machine ID allocation bottleneck

    Machine ID bits are fixed (e.g., 10 bits), limiting the number of unique machines. At 10x scale, running out of unique machine IDs causes collisions or requires complex reallocation.

  4. Final Answer:

    Option D -> Option D

  5. Quick Check:

    Fixed machine ID bits limit scale -> collisions or reconfiguration needed [check]
Hint: Fixed machine ID bits limit scale before sequence or timestamp issues [OK]
Common Mistakes:
5. At 10x traffic scale, which component in a CDN-backed architecture is most likely to fail first and why?
hard
A. Global load balancers, as they cannot distribute traffic evenly at extreme scale
B. Origin servers, due to inability to handle increased cache miss requests
C. CDN edge servers, because limited cache storage causes frequent origin fetches
D. DNS resolvers, since they become overwhelmed by increased domain resolution requests

Solution

  1. Step 1: Analyze CDN edge server cache storage limits

    At 10x scale, edge caches may evict content more frequently, increasing origin fetches.

  2. Step 2: Recognize increased origin fetches overload origin servers but edge cache misses spike first

    Edge servers become bottlenecks due to cache capacity and eviction policies before origin overload.

  3. Final Answer:

    Option C -> Option C

  4. Quick Check:

    Edge cache storage limits cause first failure at scale, not origin or load balancers [check]
Hint: Edge cache storage limits cause origin fetch spikes first at scale [OK]
Common Mistakes:
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard