Bird
Raised Fist0

Global Server Load Balancing - Routing Traffic Across Data Centers

HLDmediumInterview freq: common
Choose your preparation mode4 modes available
ArchitectureTraceChallenge
🏗️
Global Server Load Balancing - Routing Traffic Across Data Centers
Design a system to route user traffic globally across multiple data centers for low latency and high availability
mediumLoad BalancersFreq: common
⚙️
Functional Requirements
  • Route user traffic to the nearest or lowest-latency data center globally
  • Automatically failover traffic to healthy data centers during outages
  • Balance load across data centers based on capacity and health metrics
  • Support geo-based routing policies (e.g., country, region)
  • Provide real-time health monitoring of data centers and endpoints
  • Allow weighted traffic distribution for gradual rollouts or A/B testing
📊
Non-Functional Requirements
  • Support 2 billion daily active users globally
  • Achieve 99.99% availability for routing decisions
  • Maintain end-to-end routing latency under 100ms P99
  • Handle peak write QPS of at least 1 million routing updates/sec
  • Scale to 100+ global data centers with dynamic capacity changes
❓
Clarifying Questions to Ask
  • What is the expected traffic distribution across regions and data centers?
  • Are routing decisions made per user session, per request, or per IP?
  • What is the acceptable failover time when a data center goes down?
  • Is DNS-based routing acceptable or do we require anycast or IP-level routing?
  • How frequently do data center health and capacity metrics update?
  • Are there any regulatory or compliance constraints for routing traffic?
Out of Scope
✗ Designing the internal load balancing within each data center
✗ Detailed DNS server implementation or recursive resolver design
✗ Application-level request handling beyond routing decisions
✗ Security mechanisms like DDoS mitigation or authentication
📌
Assumptions
  • 2 billion daily active users (DAU)
  • Each user generates 20 requests per day requiring routing decisions
  • Average routing decision payload size is 200 bytes
  • Retention period for routing logs and metrics is 30 days
⚡
QPS Calculation
write qps2B users × 20 requests/day ÷ 86400 sec ≈ 462,963 writes/sec
peak write qpsAssuming 2× peak factor: 462,963 × 2 = 925,926 writes/sec
read qpsReads for health checks and routing queries estimated at 10× writes: 4,629,630 reads/sec
💾
Storage Estimate
daily storage462,963 writes/sec × 200 bytes × 86400 sec ≈ 8 TB/day
retention storage8 TB/day × 30 days = 240 TB
with replicationAssuming 3x replication for durability: 240 TB × 3 = 720 TB total
🖥️
Server Estimate
routing decision serviceAssuming 10K QPS per server: 925,926 ÷ 10,000 ≈ 93 servers
health monitoring serviceAssuming 5K QPS per server: 4,629,630 ÷ 5,000 ≈ 926 servers
🏗️
Architecture Diagram
Architecture Diagram
          +----------------------+
          |  Global DNS Resolver  |
          +----------+-----------+
                     |
          +----------v-----------+
          | Global Load Balancer |
          |  (Traffic Router)    |
          +----------+-----------+
                     |
    +----------------+----------------+
    |                                 |
 +--v---+                       +-----v----+
 | DC 1 |                       | DC N     |
 +------+                       +----------+
    |                                 |
 +--v---+                       +-----v----+
 | App  |                       | App      |
 +------+                       +----------+
                     ^
                     |
          +----------+-----------+
          | Health & Metrics Svc |
          +----------------------+
                     ^
                     |
          +----------+-----------+
          | Capacity & Config DB |
          +----------------------+
🧩
Components
Global DNS Resolver
Resolves user DNS queries to IPs of global load balancers based on geo-location
Why thisUses authoritative DNS with geo-DNS features (e.g., NS1, AWS Route53) to direct users to nearest LB
Why notAnycast IP routing rejected due to complex BGP management and slower failover times
Global Load Balancer (Traffic Router)
Makes real-time routing decisions directing traffic to optimal data center endpoints
Why thisEnvoy proxy with xDS API supports dynamic config updates and low-latency routing decisions
Why notHardware load balancers rejected due to lack of global scale and dynamic config flexibility
Health & Metrics Service
Continuously monitors data center and endpoint health, latency, and capacity metrics
Why thisPrometheus + custom exporters provide scalable, real-time health data with alerting
Why notPolling via SNMP rejected due to latency and lack of fine-grained metrics
Capacity & Configuration Database
Stores data center capacity, routing policies, and configuration metadata
Why thisApache Cassandra chosen for linear write scalability and multi-region replication
Why notPostgreSQL rejected due to complex manual sharding and write bottlenecks at 925K writes/sec
Data Center Application Servers
Serve user requests after routing; endpoints for global load balancer
Why thisStateless microservices deployed in Kubernetes clusters for easy scaling and failover
Why notMonolithic servers rejected due to poor scalability and deployment agility
🔄
Data Flow
1
User → Global DNS Resolver
User DNS query for service domain
DNS resolver uses geo-DNS to return IP of nearest global load balancer
2
User → Global Load Balancer
User sends request to resolved IP
Global LB receives request and prepares to route
3
Global Load Balancer → Capacity & Configuration DB
Fetch routing policies and data center capacity info
Ensures routing decisions respect capacity and configured weights
4
Global Load Balancer → Health & Metrics Service
Query latest health and latency metrics for data centers
Avoids routing to unhealthy or high-latency data centers
5
Global Load Balancer → Data Center Application Servers
Forward user request to selected data center endpoint
Routing decision based on lowest latency, health, and capacity
6
Health & Metrics Service → Capacity & Configuration DB
Update health status and metrics periodically
Keeps routing data fresh for accurate routing decisions

Practice

(1/5)
1. In the indexing pipeline, at step 3, the Indexer receives a batch of parsed documents. What state does the Indexer maintain immediately after processing this batch?
easy
A. The Indexer updates the inverted index with new terms but keeps the batch metadata for consistency checks
B. The Indexer sends the batch to the PageRank component before updating the index
C. The Indexer marks the batch as indexed and immediately deletes the raw documents
D. The Indexer queues the batch for re-parsing if term frequency thresholds are not met

Solution

  1. Step 1: Trace Indexer behavior at step 3

    The Indexer updates the inverted index with terms from the batch and retains batch metadata for consistency and fault tolerance.

  2. Step 2: Eliminate incorrect states

    Deleting raw documents immediately risks data loss. PageRank runs after indexing, not before. Re-parsing is triggered by errors, not term frequency.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Indexer updates index and keeps metadata for consistency [check]
Hint: Indexer updates index, keeps metadata for consistency [check]
Common Mistakes:
2. Without the distributed metadata service, what exactly breaks in the blob store's object retrieval process?
medium
A. Object retrieval requests block indefinitely waiting for metadata resolution
B. Clients receive errors because object locations and versions cannot be resolved
C. Clients can still retrieve blobs but with stale metadata versions
D. Blobs are retrieved but data integrity checks fail silently

Solution

  1. Step 1: Trace retrieval dependency on metadata service

    The metadata service provides object location and version info; without it, retrieval cannot resolve where to fetch data.

  2. Step 2: Eliminate other options

    Stale metadata retrieval is impossible without metadata service; blocking indefinitely is unlikely as client errors occur; silent integrity failures require data access, which fails first.

  3. Final Answer:

    Option B -> Option B

  4. Quick Check:

    Metadata service absence causes unresolved object location errors [check]
Hint: No metadata service -> no object location resolution -> retrieval errors [check]
Common Mistakes:
3. At 10x scale, which component in a WebRTC video calling system is most likely to become the bottleneck first, and why?
hard
A. Media Servers, because transcoding CPU load grows linearly with concurrent calls
B. Signaling Servers, because they maintain stateful WebSocket connections and cannot easily shard sessions
C. STUN/TURN Servers, because NAT traversal requests increase exponentially with users
D. Client devices, because battery and CPU limits prevent scaling user concurrency

Solution

  1. Step 1: Analyze component scalability at 10x load

    Media Servers scale with CPU but can be horizontally scaled; STUN/TURN scale well with stateless UDP; clients are outside system control.

  2. Step 2: Identify Signaling Server bottleneck

    Signaling Servers hold stateful WebSocket connections per user, making sharding and scaling challenging.

  3. Final Answer:

    Option B -> Option B

  4. Quick Check:

    Stateful WebSocket connections limit Signaling Server horizontal scaling [check]
Hint: Stateful WebSocket limits Signaling Server scaling first [check]
Common Mistakes:
4. At 10x current scale, which component in TikTok's architecture is most likely to fail first and why?
hard
A. The WebSocket servers maintaining live connections, since stateful connections cannot be easily sharded
B. The relational database storing user profiles, due to write amplification from frequent updates
C. The Feed Ranking Service, as it requires low-latency access to large user history data causing memory bottlenecks
D. The video transcoding cluster, because CPU-intensive jobs cannot be parallelized beyond a limit

Solution

  1. Step 1: Analyze stateful connection scaling

    WebSocket servers (D) hold state per connection; scaling requires complex connection migration or sticky sessions.

  2. Step 2: Compare other bottlenecks

    Relational DB (A) can be sharded; transcoding (B) parallelizes; Feed Ranking (C) uses caching and sharding.

  3. Step 3: Identify first failure point

    WebSocket servers hit connection limits first due to stateful nature and limited horizontal scaling.

  4. Final Answer:

    Option A -> Option A

  5. Quick Check:

    Stateful WebSocket servers bottleneck first due to connection affinity [check]
Hint: Stateful WebSocket servers limit horizontal scaling first [OK]
Common Mistakes:
5. At 10x scale, which Elasticsearch component is most likely to become the bottleneck first, and why?
hard
A. Ingest Nodes, because preprocessing pipelines cannot keep up with write throughput
B. Master Node, because cluster state updates grow exponentially with shard count
C. Data Nodes, because shard replication traffic saturates network bandwidth
D. Coordinating Node, because it must merge results from an increasing number of shards

Solution

  1. Step 1: Analyze scaling impact on Coordinating Node

    Coordinating Nodes merge and sort results from all shards; with 10x shards, merging overhead grows significantly.

  2. Step 2: Compare other components

    Master Node scales with cluster state but less impacted than query merging; Data Nodes handle replication but network can be scaled; Ingest Nodes can be scaled horizontally.

  3. Final Answer:

    Option D -> Option D

  4. Quick Check:

    Coordinating Node merging overhead grows fastest at scale [check]
Hint: Coordinating Node merges explode with shard count increase [check]
Common Mistakes:
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard