Load balancing algorithms (round robin, least connections) in HLD - System Design Guide

Problem Statement

When a single server handles all incoming requests, it quickly becomes overwhelmed, causing slow responses and potential crashes. Without distributing traffic evenly, some servers stay idle while others get overloaded, leading to poor resource use and unstable service.

Solution

Load balancing algorithms distribute incoming requests across multiple servers to prevent overload. Round robin assigns requests in a fixed order, cycling through servers evenly. Least connections sends requests to the server with the fewest active connections, balancing load dynamically based on current usage.

Architecture

Load

Load

↓

Server1

This diagram shows a load balancer distributing incoming requests to multiple servers using an algorithm like round robin or least connections.

Trade-offs

✓ Pros

→

Round robin is simple to implement and ensures even distribution when servers have similar capacity.

→

Least connections adapts to server load dynamically, improving performance under uneven request durations.

→

Both algorithms improve system availability by preventing any single server from becoming a bottleneck.

✗ Cons

→

Round robin does not consider server load or capacity differences, which can cause uneven performance if servers vary.

→

Least connections requires tracking active connections, adding overhead and complexity to the load balancer.

→

Neither algorithm handles server failures inherently; additional health checks are needed.

Use round robin when servers have similar capacity and request processing times are uniform. Use least connections when request durations vary significantly or server capacities differ.

Avoid round robin when servers have different performance or request loads vary widely. Avoid least connections in very high throughput systems where tracking connections adds unacceptable overhead.

Real World Examples

Netflix

Netflix uses least connections load balancing to route streaming requests to edge servers with the fewest active streams, ensuring smooth playback.

Amazon

Amazon employs round robin load balancing for distributing API requests evenly across identical backend servers to maintain consistent response times.

Uber

Uber uses least connections to balance ride request traffic dynamically among servers, adapting to fluctuating demand and server load.

Alternatives

Consistent Hashing

Routes requests based on a hash of client identifiers to ensure the same client hits the same server, improving cache locality.

Use when: Choose when session stickiness or cache affinity is critical, such as in user-specific data caching.

Weighted Round Robin

Extends round robin by assigning weights to servers based on capacity, sending more requests to stronger servers.

Use when: Choose when servers have different capacities but request durations are similar.

Summary

Load balancing algorithms distribute incoming requests to prevent server overload and improve availability.

Round robin cycles through servers evenly without considering load, suitable for uniform servers and requests.

Least connections routes to the server with the fewest active connections, adapting to dynamic load differences.

Practice

1. Which load balancing algorithm sends requests to servers in a fixed, repeating order without considering current load?

easy

A. Round Robin

B. Least Connections

C. Random Selection

D. IP Hash

Solution

1. Step 1: Understand Round Robin behavior

   Round Robin cycles through servers one by one in order, ignoring current load or connections.

2. Step 2: Compare with other algorithms

   Least Connections considers active connections, Random Selection picks randomly, IP Hash uses client IP. Only Round Robin ignores load.

3. Final Answer:

   Round Robin -> Option A

4. Quick Check:

   Fixed order without load check = Round Robin [OK]

Hint: Round Robin cycles evenly, ignores load [OK]

Common Mistakes:

• Confusing Least Connections with Round Robin
• Thinking Random Selection is fixed order
• Assuming IP Hash ignores load

2. Which of the following is the correct pseudocode snippet for implementing Round Robin load balancing?

easy

A. server = server with minimum active connections

B. server = hash(client_ip) % total_servers

C. server = random.choice(servers)

D. server = servers[(current_index + 1) % total_servers]

Solution

1. Step 1: Identify Round Robin logic

   Round Robin selects the next server by incrementing an index and wrapping around using modulo.

2. Step 2: Match pseudocode to logic

   server = servers[(current_index + 1) % total_servers] uses (current_index + 1) % total_servers, which correctly cycles through servers.

3. Final Answer:

   server = servers[(current_index + 1) % total_servers] -> Option D

4. Quick Check:

   Modulo cycling index = Round Robin [OK]

Hint: Round Robin uses modulo to cycle servers [OK]

Common Mistakes:

• Choosing minimum connections for Round Robin
• Using random selection syntax
• Using hash function for Round Robin

3. Given 3 servers with active connections: Server A=5, Server B=2, Server C=4, which server will Least Connections algorithm select for the next request?

medium

A. Server B

B. Server C

C. Server A

D. Server A or Server C (tie)

Solution

1. Step 1: Identify server with fewest active connections

   Server B has 2 connections, fewer than A (5) and C (4).

2. Step 2: Apply Least Connections selection

   Least Connections chooses the server with the minimum active connections, so Server B is selected.

3. Final Answer:

   Server B -> Option A

4. Quick Check:

   Fewest connections = Server B [OK]

Hint: Least Connections picks server with minimum active connections [OK]

Common Mistakes:

• Picking server with most connections
• Choosing Round Robin instead of Least Connections
• Confusing ties with multiple servers

4. A load balancer using Least Connections algorithm always sends requests to Server A, even though Server B has fewer active connections. What is the most likely cause?

medium

A. Round Robin algorithm is being used instead

B. The connection count for Server B is not updated correctly

C. Server A has a lower IP address

D. The load balancer is using random selection

Solution

1. Step 1: Analyze behavior mismatch

   Least Connections should pick server with fewer active connections, but always picks Server A.

2. Step 2: Identify likely cause

   If Server B's connection count is stale or incorrect, the balancer wrongly thinks Server A has fewer connections.

3. Final Answer:

   The connection count for Server B is not updated correctly -> Option B

4. Quick Check:

   Incorrect connection tracking causes wrong Least Connections choice [OK]

Hint: Check if connection counts update properly for Least Connections [OK]

Common Mistakes:

• Assuming algorithm is Round Robin without checking
• Thinking IP address affects Least Connections
• Ignoring connection count accuracy

5. You have 4 servers with varying capacities and active connections: Server A (capacity 10, connections 5), Server B (capacity 20, connections 15), Server C (capacity 15, connections 10), Server D (capacity 10, connections 2). Which server should a capacity-aware Least Connections algorithm select to balance load best?

hard

A. Server C

B. Server B

C. Server D

D. Server A

Solution

1. Step 1: Calculate load ratio (connections/capacity) for each server

   Server A: 5/10 = 0.5, Server B: 15/20 = 0.75, Server C: 10/15 ≈ 0.67, Server D: 2/10 = 0.2

2. Step 2: Select server with lowest load ratio

   Server D has the lowest ratio (0.2), so it is selected in capacity-aware Least Connections.

3. Final Answer:

   Server D -> Option C

4. Quick Check:

   Lowest load ratio = Server D [OK]

Hint: Consider connections relative to capacity, not just counts [OK]

Common Mistakes:

• Picking server with fewest connections ignoring capacity
• Ignoring capacity differences
• Choosing server with lowest absolute connections only