Bird
Raised Fist0

Layer 4 vs Layer 7 Load Balancing - Transport vs Application Level

HLDmediumInterview freq: common
Choose your preparation mode4 modes available
ArchitectureTraceChallenge
🏗️
Layer 4 vs Layer 7 Load Balancing - Transport vs Application Level
Designing scalable load balancing strategies at transport and application layers for high throughput systems
mediumLoad BalancersFreq: common
⚙️
Functional Requirements
  • Distribute incoming client TCP/UDP connections efficiently across backend servers
  • Support session persistence (sticky sessions) when required by application
  • Perform health checks on backend servers to avoid routing to unhealthy nodes
  • Enable advanced routing based on HTTP headers, cookies, or URL paths for Layer 7
  • Provide SSL/TLS termination and offloading at the load balancer
  • Support dynamic scaling of backend pools without downtime
📊
Non-Functional Requirements
  • Handle peak traffic of at least 1 million requests per second (RPS) with <100ms P99 latency
  • Achieve 99.99% availability for the load balancing layer
  • Support up to 2 billion daily active users (DAU) generating 40 requests per user per day
  • Ensure failover within 500ms on backend node failure
  • Maintain throughput with <1% packet loss at peak load
❓
Clarifying Questions to Ask
  • What protocols must the load balancer support? TCP, UDP, HTTP/HTTPS?
  • Is SSL/TLS termination required at the load balancer or backend?
  • Are sticky sessions mandatory or optional?
  • What is the expected peak QPS and average request size?
  • Is advanced Layer 7 routing (e.g., path-based, header-based) required?
  • What are the failure recovery SLAs for backend health checks?
Out of Scope
✗ Design of backend application servers or microservices
✗ Detailed SSL certificate management lifecycle
✗ DNS-level load balancing or global traffic management
✗ Client-side retry or fallback mechanisms
📌
Assumptions
  • 2 billion DAU
  • 40 requests per user per day
  • Average request size 100 bytes
  • Peak traffic is 3x average traffic
⚡
QPS Calculation
write qps2B × 40 / 86400 ≈ 925,926 requests/sec average
peak write qps925,926 × 3 ≈ 2,777,778 requests/sec peak
read qpsAssuming 1:1 read to write ratio, read_qps ≈ 2,777,778 requests/sec peak
💾
Storage Estimate
daily storage2,777,778 req/sec × 100 bytes × 86400 sec ≈ 24 TB/day
retention days7
total storage24 TB × 7 days = 168 TB raw
with replication168 TB × 3 replicas = 504 TB total
🖥️
Server Estimate
layer4 lbAssuming each Layer 4 LB can handle 100K connections/sec, need ~28 servers
layer7 lbAssuming each Layer 7 LB can handle 50K requests/sec due to parsing overhead, need ~56 servers
🏗️
Architecture Diagram
Architecture Diagram
          +----------------------+
          |      DNS Resolver     |
          +----------+-----------+
                     |
          +----------v-----------+
          |  Layer 4 Load Balancer|
          +----------+-----------+
                     |
          +----------v-----------+
          |  Layer 7 Load Balancer|
          +----------+-----------+
                     |
          +----------v-----------+
          |  Backend Application |
          |       Servers        |
          +----------------------+
🧩
Components
DNS Resolver
Resolves client domain names to IP addresses of Layer 4 load balancers
Why thisDNS provides geographic load distribution and fault tolerance by resolving to multiple Layer 4 LB IPs
Why notDirect IP usage lacks geographic failover and load distribution capabilities
Layer 4 Load Balancer (e.g., AWS NLB, HAProxy in TCP mode)
Performs transport-level load balancing by routing TCP/UDP connections based on IP and port
Why thisAWS NLB and HAProxy TCP mode offer high throughput with low latency, efficiently handling millions of connections
Why notLayer 7 LB at this scale would add parsing overhead and increase latency
Layer 7 Load Balancer (e.g., NGINX, Envoy)
Performs application-level load balancing with HTTP header, cookie, and path-based routing
Why thisNGINX and Envoy enable advanced routing, SSL termination, and session persistence required by modern web apps
Why notLayer 4 LB cannot inspect HTTP headers or perform content-based routing
Health Check Service
Continuously monitors backend server health and updates load balancers to avoid unhealthy nodes
Why thisActive health checks using tools like Consul or custom probes reduce failed requests and improve availability
Why notPassive health checks delay failure detection and increase error rates
Backend Application Servers
Serve business logic and respond to client requests routed by load balancers
Why thisStateless or stateful servers behind load balancers handle actual request processing
Why notDirect client access to backend servers would not scale or provide fault tolerance
🔄
Data Flow
1
Client → DNS Resolver
Client resolves domain to Layer 4 LB IP
Enables geographic load distribution and failover
2
Client → Layer 4 Load Balancer
Client initiates TCP connection to Layer 4 LB
Layer 4 LB routes connection based on IP and port
3
Layer 4 Load Balancer → Layer 7 Load Balancer
Forward TCP connection to Layer 7 LB for application-level routing
Layer 7 LB inspects HTTP headers for routing decisions
4
Layer 7 Load Balancer → Backend Application Server
Route HTTP request based on path, headers, or cookies
Supports sticky sessions and SSL termination
5
Health Check Service → Load Balancers
Send health status updates to load balancers
Load balancers avoid routing to unhealthy backends
6
Backend Application Server → Client
Respond to client request via Layer 7 and Layer 4 LBs
Response flows back through same path

Practice

(1/5)
1. In the request flow for creating a pull request, at step 5 the Merge Worker attempts to apply the patch to the target branch. What happens if a merge conflict is detected at this step?
easy
A. The Merge Worker marks the pull request as conflicted and notifies the user to resolve manually
B. The Merge Worker retries the merge indefinitely until no conflicts remain
C. The Merge Worker discards the pull request and deletes the source branch
D. The Merge Worker automatically resolves conflicts using the latest commit's changes

Solution

  1. Step 1: Trace Merge Worker behavior at Step 5

    When a merge conflict occurs, automatic resolution is unsafe; the system flags the PR as conflicted and requires manual user intervention.

  2. Step 2: Eliminate incorrect options

    Automatic conflict resolution risks data loss; discarding PR or deleting branches is destructive; infinite retries cause resource exhaustion.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Merge conflicts -> PR marked conflicted, user notified [check]
Hint: Conflicts block merge until user resolves [check]
Common Mistakes:
2. You need to handle 1 million product updates per minute in the catalog without blocking user searches. Which component best handles this high write throughput while ensuring eventual consistency?
easy
A. Use a distributed log system like Kafka to buffer and asynchronously process updates
B. Use an in-memory cache like Redis as the primary data store for product updates
C. Use a NoSQL key-value store with synchronous writes for every update
D. Use a relational database with immediate consistency for all writes

Solution

  1. Step 1: Identify write throughput handling

    Relational DBs (A) and synchronous NoSQL writes (C) block on writes, limiting throughput. Redis (D) lacks durability guarantees for primary storage.

  2. Step 2: Recognize Kafka's role

    Kafka buffers high write volume asynchronously, decoupling writes from reads and enabling eventual consistency.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Kafka buffers writes asynchronously -> high throughput, eventual consistency [check]
Hint: Buffer writes asynchronously with Kafka for high throughput [check]
Common Mistakes:
3. A video upload service must handle millions of concurrent uploads and ensure durability before transcoding. Which component best handles temporary storage and durability of raw uploads before processing?
easy
A. Use an in-memory cache like Redis to store raw uploads temporarily
B. Use a CDN edge cache to hold raw uploads before transcoding
C. Use a distributed object storage system like Amazon S3 for raw video storage
D. Store raw uploads directly in a relational database for strong consistency

Solution

  1. Step 1: Identify durability and scale needs for raw uploads

    Raw uploads require durable, scalable storage that can handle large binary files and high concurrency. Redis (Use an in-memory cache like Redis to store raw uploads temporarily) is in-memory and not durable for large files. Relational DBs (Store raw uploads directly in a relational database for strong consistency) are inefficient for large blobs and scale poorly. CDN edge caches (Use a CDN edge cache to hold raw uploads before transcoding) are for delivery, not upload storage.

  2. Step 2: Recognize object storage suitability

    Distributed object storage like S3 provides durability, scalability, and cost-effective storage for large files, making it ideal for raw uploads before transcoding.

  3. Final Answer:

    Option C -> Option C

  4. Quick Check:

    Object storage ensures durable, scalable raw upload storage [check]
Hint: Object storage for durable large file uploads [check]
Common Mistakes:
4. In a sliding window rate limiter, at step 3 the system records a new request timestamp. What happens internally to the stored counters at this step?
easy
A. Old timestamps outside the sliding window are removed before adding the new timestamp
B. The leaky bucket drains tokens at a fixed rate before accepting the request
C. The token bucket refills tokens based on elapsed time before decrementing
D. The fixed window counter resets to zero if the current minute changed

Solution

  1. Step 1: Understand sliding window counter mechanics

    Sliding window counters maintain timestamps of recent requests and remove those outside the window to keep an accurate count.

  2. Step 2: At step 3, old timestamps beyond the window are purged before adding the new request timestamp

    This ensures the count reflects only requests within the sliding window duration.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Sliding window removes expired timestamps before counting new requests [check]
Hint: Sliding window purges old timestamps before counting new ones [check]
Common Mistakes:
5. If the API Gateway's authentication service crashes, what is the immediate failure mode observed by clients attempting to access protected endpoints?
medium
A. Clients receive a 500 Internal Server Error due to gateway crash
B. Clients receive a 401 Unauthorized response because authentication cannot be verified
C. Requests bypass authentication and reach backend services unverified
D. Requests queue indefinitely at the gateway until authentication service recovers

Solution

  1. Step 1: Trace failure impact of authentication service crash

    Without the authentication service, the gateway cannot verify credentials and may default to allowing requests through if no fail-closed mechanism is implemented.

  2. Step 2: Evaluate other options

    Clients receive a 500 Internal Server Error due to gateway crash is unlikely as gateway remains up. Clients receive a 401 Unauthorized response because authentication cannot be verified requires fail-closed design, which is not always implemented. Requests queue indefinitely at the gateway until authentication service recovers causes indefinite blocking, which is rare.

  3. Final Answer:

    Option C -> Option C

  4. Quick Check:

    Auth service down -> requests pass unverified if no fail-closed [check]
Hint: Auth crash -> requests may bypass if no fail-closed design [check]
Common Mistakes:
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard