Layer 4 vs Layer 7 load balancing in HLD - Design Approaches Compared

Design: Load Balancing System

Design focuses on the load balancer components and their differences at Layer 4 and Layer 7. Out of scope are backend server implementations and client-side details.

Functional Requirements

FR1: Distribute incoming client requests efficiently across multiple backend servers

FR2: Support both Layer 4 (Transport Layer) and Layer 7 (Application Layer) load balancing

FR3: Ensure high availability and fault tolerance

FR4: Provide low latency request routing

FR5: Support health checks for backend servers

FR6: Allow session persistence (sticky sessions) when needed

Non-Functional Requirements

NFR1: Handle up to 50,000 concurrent connections

NFR2: API response latency p99 under 100ms

NFR3: Availability target of 99.9% uptime

NFR4: Support both TCP and HTTP/HTTPS protocols

NFR5: Scalable to add more backend servers without downtime

Think Before You Design

Questions to Ask

❓ Question 1

❓ Question 2

❓ Question 3

❓ Question 4

❓ Question 5

❓ Question 6

Key Components

Load balancer (Layer 4 and Layer 7)

Health check service

Backend server pool

Session persistence store

Monitoring and logging system

Design Patterns

Round-robin and least connections load balancing

Sticky sessions using cookies or IP hashing

SSL termination and passthrough

Content-based routing (URL, headers)

Failover and retry mechanisms

Reference Architecture

Client
  |
  | TCP/HTTP Requests
  v
+-------------------+
|   Load Balancer   |
|  +-------------+  |
|  | Layer 4 LB  |  |---> Backend Servers (TCP level routing)
|  +-------------+  |
|                   |
|  +-------------+  |
|  | Layer 7 LB  |  |---> Backend Servers (HTTP routing, content-based)
|  +-------------+  |
+-------------------+
       |
       v
  Health Checks
       |
       v
Backend Server Pool

Components

Layer 4 Load Balancer

IP Hashing, TCP Proxy

Routes traffic based on IP and port without inspecting application data, fast and low latency

Layer 7 Load Balancer

HTTP Proxy, Reverse Proxy (e.g., NGINX, Envoy)

Inspects HTTP headers and content to route requests based on URL, cookies, or headers

Health Check Service

Periodic TCP/HTTP probes

Monitors backend server health to avoid routing to unhealthy servers

Backend Server Pool

Application servers

Handles actual client requests

Session Persistence Store

In-memory store (e.g., Redis) or sticky session mechanism

Maintains session affinity for clients requiring sticky sessions

Request Flow

1. Client sends request to load balancer IP and port

2. Layer 4 load balancer routes request based on IP and port without inspecting payload

3. If Layer 7 load balancing is enabled, request is forwarded to Layer 7 proxy

4. Layer 7 load balancer inspects HTTP headers and content to decide backend server

5. Load balancer performs health check to ensure backend server is healthy

6. Request is forwarded to selected backend server

7. Backend server processes request and sends response back through load balancer

8. Load balancer forwards response to client

9. Session persistence is maintained if configured, using cookies or IP hashing

Database Schema

Not applicable as this design focuses on load balancing components and routing logic rather than persistent data storage.

Scaling Discussion

Bottlenecks

Load balancer CPU and memory limits under high connection rates

Latency increase due to deep packet inspection at Layer 7

Single point of failure if load balancer is not redundant

Session persistence store becoming a bottleneck

Health check frequency causing overhead

Solutions

Use multiple load balancer instances with DNS or anycast for redundancy

Offload SSL termination to dedicated hardware or use optimized libraries

Implement horizontal scaling for Layer 7 proxies with consistent hashing

Use distributed in-memory stores for session persistence with replication

Tune health check intervals and use adaptive health checks

Interview Tips

Time: Spend 10 minutes clarifying requirements and constraints, 20 minutes designing the architecture and explaining Layer 4 vs Layer 7 differences, 10 minutes discussing scaling and trade-offs, 5 minutes for questions.

Explain the difference between Layer 4 and Layer 7 load balancing clearly

Discuss trade-offs: speed vs flexibility

Highlight how session persistence is handled differently

Mention health checks and fault tolerance

Discuss scaling strategies and avoiding single points of failure

Practice

1. Which layer does Layer 4 load balancing operate on?

easy

A. Network layer using routing tables

B. Application layer using request content

C. Transport layer using IP and port

D. Physical layer using hardware signals

Solution

1. Step 1: Understand OSI layers relevant to load balancing

   Layer 4 corresponds to the transport layer, which handles IP addresses and ports.

2. Step 2: Match Layer 4 load balancing function

   Layer 4 load balancing uses IP and port information to distribute traffic without inspecting content.

3. Final Answer:

   Transport layer using IP and port -> Option C

4. Quick Check:

   Layer 4 = Transport layer [OK]

Hint: Layer 4 uses IP and port, Layer 7 uses content [OK]

Common Mistakes:

• Confusing Layer 4 with Layer 7
• Thinking Layer 4 inspects request content
• Mixing network and transport layers

2. Which of the following is a correct characteristic of Layer 7 load balancing?

easy

A. Routes traffic based on IP address only

B. Does not inspect HTTP headers

C. Is faster than Layer 4 load balancing

D. Operates at the application layer inspecting request content

Solution

1. Step 1: Identify Layer 7 load balancing features

   Layer 7 load balancing works at the application layer and inspects request content like HTTP headers.

2. Step 2: Evaluate options

   Operates at the application layer inspecting request content correctly states Layer 7 inspects request content; others are incorrect or misleading.

3. Final Answer:

   Operates at the application layer inspecting request content -> Option D

4. Quick Check:

   Layer 7 = Application layer content inspection [OK]

Hint: Layer 7 inspects content, Layer 4 does not [OK]

Common Mistakes:

• Assuming Layer 7 ignores HTTP headers
• Believing Layer 7 is faster than Layer 4
• Confusing IP-based routing with content-based routing

3. Given a load balancer that routes requests based on URL paths, which layer is it operating at?

medium

A. Layer 2

B. Layer 7

C. Layer 4

D. Layer 3

Solution

1. Step 1: Understand URL path inspection

   URL paths are part of the application data in HTTP requests, which Layer 7 can inspect.

2. Step 2: Match to OSI layer

   Layer 7 is the application layer that can route based on URL paths, unlike lower layers.

3. Final Answer:

   Layer 7 -> Option B

4. Quick Check:

   URL path routing = Layer 7 [OK]

Hint: URL path routing means Layer 7 [OK]

Common Mistakes:

• Choosing Layer 4 for URL-based routing
• Confusing network layers with application layer
• Ignoring request content in routing decisions

4. A system uses a load balancer that inspects HTTP headers but is running slower than expected. What is a likely cause?

medium

A. Layer 7 load balancing adds processing overhead due to content inspection

B. Using Layer 4 load balancing instead of Layer 7

C. Layer 4 load balancing inspects too many headers

D. Layer 7 load balancing does not support IP-based routing

Solution

1. Step 1: Identify load balancer type and performance impact

   Layer 7 load balancing inspects request content like HTTP headers, which adds processing overhead.

2. Step 2: Analyze options for cause of slowness

   Layer 7 load balancing adds processing overhead due to content inspection correctly explains that Layer 7's content inspection slows processing; others are incorrect.

3. Final Answer:

   Layer 7 load balancing adds processing overhead due to content inspection -> Option A

4. Quick Check:

   Content inspection causes slower Layer 7 [OK]

Hint: Content inspection slows Layer 7 load balancers [OK]

Common Mistakes:

• Blaming Layer 4 for inspecting headers
• Ignoring processing overhead of Layer 7
• Assuming Layer 7 cannot route by IP

5. You need to design a scalable web service that routes requests based on user authentication tokens in HTTP headers and also requires high throughput. Which load balancing approach is best?

hard

A. Layer 7 load balancing to inspect tokens despite some overhead

B. Layer 4 load balancing for speed, ignoring tokens

C. Use DNS round-robin instead of load balancers

D. Combine Layer 2 switching with Layer 3 routing

Solution

1. Step 1: Identify requirement to inspect authentication tokens

   Authentication tokens are in HTTP headers, requiring application-level inspection.

2. Step 2: Choose load balancing approach balancing inspection and throughput

   Layer 7 load balancing inspects headers and supports smart routing, suitable despite overhead.

3. Step 3: Evaluate other options

   Layer 4 cannot inspect tokens; DNS round-robin lacks smart routing; Layer 2/3 unrelated to content inspection.

4. Final Answer:

   Layer 7 load balancing to inspect tokens despite some overhead -> Option A

5. Quick Check:

   Token inspection needs Layer 7 [OK]

Hint: Inspect tokens? Use Layer 7 despite overhead [OK]

Common Mistakes:

• Choosing Layer 4 ignoring token inspection need
• Using DNS round-robin for smart routing
• Confusing network layers with application logic