Bird
Raised Fist0

Layer 4 vs Layer 7 Load Balancing - Transport vs Application Level

HLDmediumInterview freq: common
Choose your preparation mode4 modes available
ArchitectureTraceChallenge
🎯
Layer 4 vs Layer 7 Load Balancing - Transport vs Application Level
45 min · senior
senior

You are designing the load balancing layer for a 50 million daily active user (DAU) startup that provides a latency-sensitive web and mobile application. The system must handle encrypted traffic end-to-end, maintain P99 latency under 200ms, and support sticky sessions for user affinity. The startup expects traffic spikes during peak hours and requires seamless failover without dropping active connections. The engineering team is debating between Layer 4 (transport-level) and Layer 7 (application-level) load balancing approaches.

Constraints
  • All traffic must be encrypted end-to-end (TLS termination considerations)
  • P99 latency must remain under 200ms including load balancing overhead
  • Support sticky sessions for user affinity to backend instances
  • Handle traffic spikes up to 3x baseline load without degradation
  • Seamless failover with zero dropped active connections
  • Support health checks without impacting production traffic
📋
Guided Steps
1/5
Step 1
Choosing Between Layer 4 and Layer 7 Load Balancing
Given the scenario, which load balancing layer would you choose and why?
💡 Hint
Consider the trade-offs between protocol awareness, latency overhead, and TLS termination.
Key Decision
Use Layer 7 load balancing with Envoy Proxy for application-aware routing and TLS termination to enable sticky sessions and fine-grained traffic control.
Interviewer Expectation
Mid-level: Names Layer 7 load balancer like Envoy or NGINX. Senior: Explains TLS termination impact on latency and how Layer 7 enables cookie-based sticky sessions and path-based routing.
⚠️Common Wrong Answer
Choosing Layer 4 load balancing with IP hash for sticky sessions - fails to support TLS termination and application-aware routing, leading to limited flexibility and inability to inspect encrypted traffic.
Step 2
Handling TLS Encryption and Termination
How will you handle TLS encryption and termination to meet end-to-end encryption and latency constraints?
💡 Hint
Think about where TLS is terminated and how to maintain encryption between load balancer and backend.
Key Decision
Terminate TLS at Envoy Layer 7 load balancer and re-encrypt traffic to backend using mutual TLS to maintain end-to-end encryption and enable application-level routing.
Interviewer Expectation
Mid-level: Mentions TLS termination at load balancer. Senior: Details mutual TLS between load balancer and backend, explaining how this preserves encryption and supports observability.
⚠️Common Wrong Answer
Terminate TLS only at load balancer without re-encryption - breaks end-to-end encryption requirement and exposes backend traffic unencrypted.
Step 3
Ensuring Sticky Sessions and User Affinity
How will you implement sticky sessions to maintain user affinity with backend instances?
💡 Hint
Consider cookie-based vs IP-hash approaches and their impact on scalability and failover.
Key Decision
Use Layer 7 cookie-based sticky sessions managed by Envoy to bind users to backend instances, enabling session persistence and easier failover handling.
Interviewer Expectation
Mid-level: Suggests IP hash or cookie-based affinity. Senior: Explains why cookie-based affinity is preferred at Layer 7 for better load distribution and failover resilience.
⚠️Common Wrong Answer
Using IP hash for sticky sessions - causes uneven load distribution and fails during client IP changes or proxy scenarios.
Step 4
Scaling and Handling Traffic Spikes
How will your load balancing design handle 3x traffic spikes without degrading latency or dropping connections?
💡 Hint
Think about autoscaling backend pools and load balancer horizontal scaling strategies.
Key Decision
Deploy Envoy in a horizontally scalable service mesh with autoscaling backend pools behind Kubernetes Horizontal Pod Autoscaler, combined with consistent hashing to minimize connection churn.
Interviewer Expectation
Mid-level: Mentions autoscaling backend instances. Senior: Explains horizontal scaling of Envoy proxies, consistent hashing to reduce connection churn, and circuit breakers to protect backends.
⚠️Common Wrong Answer
Static backend pool sizing - leads to overload and increased latency during spikes.
Step 5
Health Checks and Failover Strategy
How will you implement health checks and failover to ensure zero dropped active connections?
💡 Hint
Consider active and passive health checks and connection draining mechanisms.
Key Decision
Use Envoy’s active and passive health checks with connection draining and graceful failover to reroute traffic without dropping active connections.
Interviewer Expectation
Mid-level: Describes health checks. Senior: Details active/passive health checks, connection draining, and how Envoy gracefully removes unhealthy hosts without connection drops.
⚠️Common Wrong Answer
No connection draining - causes dropped connections during failover.
🔌
API Design Challenge
Sketch 3 core API endpoints for the load balancer management plane to configure and monitor load balancing policies.
Hints
  • How does the client configure backend pools and routing rules?
  • How does the client query real-time health and metrics of backend servers?
  • How does the client update sticky session policies or TLS settings dynamically?
📊
Scoring Rubric
▼ Show
Must Have - 60%
  • Clear choice and justification of Layer 4 vs Layer 7 load balancing with real tech
  • Detailed TLS termination and re-encryption strategy preserving end-to-end encryption
  • Sticky session implementation with explanation of cookie-based affinity benefits
  • Scalable architecture to handle 3x traffic spikes without latency degradation
  • Health check and failover design ensuring zero dropped active connections
Good to Have - 30%
  • Use of consistent hashing to reduce connection churn
  • Autoscaling load balancer and backend pools with Kubernetes HPA
  • Dynamic management plane APIs for configuration and monitoring
  • Integration with observability tools like Prometheus
Bonus - 10%
  • Discussion of circuit breakers and rate limiting at Layer 7
  • Consideration of multi-region failover and geo-load balancing
Anti-patterns to Avoid
Terminating TLS only at load balancer without backend re-encryption - breaks end-to-end encryption and security compliance, exposing backend traffic to potential interception or tampering.
Using IP hash for sticky sessions - causes uneven load distribution and fails with client IP changes, leading to poor scalability and session loss.
Static backend pool sizing - leads to overload and increased latency during traffic spikes, causing degraded user experience.
No connection draining during failover - causes dropped active connections, resulting in poor availability and user disruption.
✅
Model Answer
▼ Reveal

For a 50M DAU latency-sensitive application requiring end-to-end encryption and sticky sessions, Layer 7 load balancing with Envoy Proxy is optimal. TLS is terminated at Envoy and re-encrypted to backends using mutual TLS to preserve encryption. Cookie-based sticky sessions ensure user affinity with better failover handling. The system scales horizontally with Kubernetes HPA and uses consistent hashing to reduce connection churn. Health checks are active and passive with connection draining to avoid dropped connections. Management APIs allow dynamic configuration and monitoring, integrated with Prometheus and Grafana for observability.

Key Decisions
  • Choose Envoy Proxy for Layer 7 load balancing to enable application-aware routing and TLS termination.
  • Implement mutual TLS between Envoy and backend to maintain end-to-end encryption.
  • Use cookie-based sticky sessions managed by Envoy for reliable user affinity and failover resilience.
  • Scale Envoy proxies horizontally and backend pools with Kubernetes HPA to handle 3x traffic spikes.
  • Apply consistent hashing to minimize connection churn during scaling and failover.
  • Use Envoy’s active and passive health checks with connection draining to ensure zero dropped connections.
  • Expose management APIs to dynamically configure backend pools, routing rules, sticky session policies, and TLS settings.
  • Integrate with Prometheus and Grafana for real-time health monitoring and alerting.
  • Envoy Proxy as Layer 7 load balancer for application-aware routing and TLS termination
  • Mutual TLS between Envoy and backend services to maintain end-to-end encryption
  • Cookie-based sticky sessions managed by Envoy for user affinity and session persistence
  • Horizontal scaling of Envoy proxies with Kubernetes Horizontal Pod Autoscaler for traffic spikes
  • Consistent hashing to minimize connection churn during scaling and failover
  • Active and passive health checks with connection draining to ensure zero dropped connections
  • Use of gRPC or REST APIs for management plane to configure backend pools, routing, and TLS settings dynamically
  • Integration with Prometheus and Grafana for real-time monitoring and alerting of load balancer health

Practice

(1/5)
1. In the analytics pipeline, at step 4 the Flink job reads from Kafka and applies windowed aggregations. What is the state of the Flink job's checkpoint when a failure occurs immediately after step 4?
easy
A. The checkpoint includes all processed events up to the failure, ensuring exactly-once semantics
B. The checkpoint is incomplete, so Flink will reprocess some events causing duplicates
C. The checkpoint is empty because Flink does not checkpoint at this step
D. The checkpoint only contains metadata, not the actual aggregation state

Solution

  1. Step 1: Understand Flink's checkpointing mechanism

    Flink performs distributed snapshots capturing the full operator state including windowed aggregations at step 4.

  2. Step 2: Reason about failure impact

    Since checkpoint includes all processed events and state, Flink can restore exactly-once semantics after failure.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Flink checkpoints capture full operator state for exactly-once recovery [check]
Hint: Flink checkpoints save full operator state atomically [check]
Common Mistakes:
2. A web crawler must avoid overloading a single domain by spacing requests appropriately. Which component is primarily responsible for enforcing this constraint?
easy
A. Politeness Policy Manager, which schedules requests respecting domain-specific delays
B. Scheduler, which balances load across multiple crawler instances
C. Duplicate Detection module, which filters repeated URLs to reduce load
D. URL Frontier, which stores URLs to be crawled next without timing control

Solution

  1. Step 1: Identify the role of Politeness Policy Manager

    This component enforces delays between requests to the same domain to prevent overloading servers.

  2. Step 2: Eliminate other options

    Duplicate Detection filters URLs but does not control timing; Scheduler balances load but does not enforce domain-specific delays; URL Frontier queues URLs but lacks timing logic.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Politeness Policy controls request timing per domain [check]
Hint: Politeness Policy controls request timing per domain [OK]
Common Mistakes:
3. A globally distributed key-value store must serve low-latency reads even during network partitions. Which architectural choice best supports this requirement?
easy
A. Use asynchronous replication with local reads allowed during partitions
B. Use synchronous replication across all data centers to ensure consistency
C. Use a single primary data center with read replicas in others
D. Use a quorum-based consensus protocol for all reads and writes

Solution

  1. Step 1: Identify PACELC trade-off

    During partitions (P), latency (L) matters; asynchronous replication allows local reads with low latency.

  2. Step 2: Evaluate options for latency under partition

    Synchronous replication (A) and quorum (D) increase latency; single primary (C) risks unavailability if primary is unreachable.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Asynchronous replication -> low latency reads during partitions [check]
Hint: Async replication favors latency over consistency under partitions [check]
Common Mistakes:
4. A web application requires users to authenticate with multi-factor authentication (MFA) before accessing sensitive data. Which component is primarily responsible for enforcing MFA in this scenario?
easy
A. The Database, because it stores user credentials and MFA tokens
B. The API Gateway, because it routes requests and can reject unauthorized traffic
C. The Identity Provider (IdP), because it manages user credentials and enforces MFA policies
D. The Load Balancer, because it distributes authentication requests across servers

Solution

  1. Step 1: Identify the role of the Identity Provider

    The IdP manages user credentials and enforces authentication policies including MFA.

  2. Step 2: Eliminate other components

    The API Gateway routes traffic but does not enforce MFA; the Database stores data but does not enforce policies; the Load Balancer distributes requests but does not handle authentication.

  3. Final Answer:

    Option C -> Option C

  4. Quick Check:

    IdP enforces MFA policies, not routing or storage components [check]
Hint: MFA enforcement belongs to the Identity Provider [check]
Common Mistakes:
5. Without the retention policy enforcement component in a time-series database, what exact failure occurs?
medium
A. Queries return stale data because old data is never deleted
B. Storage fills up causing ingestion to block and new data to be lost
C. Clients receive inconsistent query results due to missing indexes
D. Data is corrupted due to overlapping retention windows

Solution

  1. Step 1: Understand retention policy role

    Retention policies delete or downsample old data to manage storage and maintain data consistency.

  2. Step 2: Analyze failure without retention enforcement

    Without enforcement, overlapping retention windows cause data duplication or corruption during compaction/downsampling. Storage filling (B) is secondary and delayed. Stale queries (A) and missing indexes (D) are unrelated.

  3. Final Answer:

    Option D -> Option D

  4. Quick Check:

    Retention enforcement prevents data corruption from overlapping windows [check]
Hint: Retention policies prevent data overlap and corruption [check]
Common Mistakes:
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard