Load balancing algorithms (round robin, least connections) in HLD - Deep Dive

Overview - Load balancing algorithms (round robin, least connections)

What is it?

Load balancing algorithms are methods used to distribute incoming network or application traffic across multiple servers. Two common types are round robin, which cycles through servers in order, and least connections, which sends traffic to the server with the fewest active connections. These algorithms help ensure no single server is overwhelmed, improving performance and reliability. They are essential in systems that handle many users or requests simultaneously.

Why it matters

Without load balancing algorithms, some servers could become overloaded while others sit idle, causing slow responses or crashes. This would lead to poor user experience and unreliable services. Load balancing algorithms solve this by spreading work evenly or smartly, so systems stay fast and available even under heavy use. They make websites, apps, and services scalable and resilient.

Where it fits

Before learning load balancing algorithms, you should understand basic networking, servers, and client-server communication. After this, you can explore advanced load balancing techniques, health checks, and auto-scaling. This topic fits into the broader study of system design, especially in building scalable and fault-tolerant architectures.

Mental Model

Core Idea

Load balancing algorithms decide how to share incoming work fairly or efficiently among servers to keep systems fast and stable.

Think of it like...

Imagine a cashier line at a grocery store. Round robin is like sending each new customer to the next cashier in order, while least connections is like sending the customer to the cashier with the shortest line.

┌───────────────┐
│ Incoming      │
│ Requests      │
└──────┬────────┘
       │
       ▼
┌───────────────┐
│ Load Balancer │
│ Algorithm     │
└──────┬────────┘
       │
 ┌─────┴─────┬─────┬─────┐
 │           │     │     │
 ▼           ▼     ▼     ▼
Server 1   Server 2 Server 3 Server 4
(Round    (Round  (Round  (Round
Robin or  Robin or Robin or
Least     Least   Least   Least
Conn.)    Conn.)  Conn.)  Conn.)

Build-Up - 7 Steps

1

Foundation

What is Load Balancing

Concept: Introduce the basic idea of load balancing and why it is needed.

Load balancing means sharing incoming work or requests among multiple servers. This helps avoid overloading one server while others are idle. It improves speed and reliability of services like websites or apps.

Result

You understand that load balancing is about spreading work to keep systems responsive.

Understanding load balancing is key to building systems that handle many users without slowing down or crashing.

2

Foundation

Basic Server Request Handling

Concept: Explain how servers handle requests and what happens when overloaded.

A server receives requests and processes them one by one or in parallel. If too many requests come at once, the server slows down or fails. Without load balancing, some servers get too busy while others wait.

Result

You see why distributing requests is necessary to keep servers healthy.

Knowing server limits helps appreciate why load balancing algorithms are crucial.

3

Intermediate

Round Robin Algorithm Explained

🤔Before reading on: do you think round robin always balances load perfectly? Commit to yes or no.

Concept: Round robin sends each new request to the next server in a fixed order, cycling through all servers.

Imagine servers numbered 1 to N. The load balancer sends the first request to server 1, second to server 2, and so on. After server N, it starts again at server 1. This is simple and fair if all servers are equal.

Result

Requests are evenly distributed in a repeating sequence.

Understanding round robin shows how simple fairness can be used to share load, but it assumes all servers have equal capacity and request load.

4

Intermediate

Least Connections Algorithm Explained

🤔Before reading on: do you think least connections always picks the fastest server? Commit to yes or no.

Concept: Least connections sends each new request to the server with the fewest active connections, aiming to balance actual load rather than just count requests.

The load balancer tracks how many requests each server is currently handling. It sends new requests to the server with the smallest number of ongoing connections. This helps when servers have different speeds or request times.

Result

Requests go to the least busy server, improving response times under uneven loads.

Knowing least connections helps understand smarter load distribution that adapts to real-time server load.

5

Intermediate

Comparing Round Robin and Least Connections

🤔Before reading on: which algorithm do you think handles uneven server speeds better? Commit to your answer.

Concept: Compare strengths and weaknesses of round robin and least connections algorithms.

Round robin is simple and works well if servers are identical and requests are similar. Least connections adapts to server load differences and varying request times but needs more tracking. Each has tradeoffs in complexity and performance.

Result

You can choose the right algorithm based on system needs.

Understanding tradeoffs helps pick the best load balancing method for different real-world scenarios.

6

Advanced

Handling Server Failures in Load Balancing

🤔Before reading on: do you think load balancers automatically know if a server is down? Commit to yes or no.

Concept: Explain how load balancers detect and avoid sending requests to failed servers.

Load balancers use health checks to test if servers respond correctly. If a server fails, it is temporarily removed from the rotation. Algorithms like round robin or least connections then skip that server until it recovers.

Result

Systems stay reliable by not sending requests to broken servers.

Knowing failure handling is critical to building fault-tolerant load balancing systems.

7

Expert

Scaling Load Balancers and Algorithm Limitations

🤔Before reading on: can a single load balancer become a bottleneck? Commit to yes or no.

Concept: Discuss challenges when load balancers themselves need scaling and algorithm limits in large systems.

In very large systems, one load balancer can become a bottleneck or single point of failure. Techniques like multiple load balancers, consistent hashing, or weighted algorithms help. Also, algorithms may not perfectly balance load due to network delays or uneven request sizes.

Result

You understand real-world complexities beyond basic algorithms.

Knowing these limits prepares you to design scalable, robust load balancing architectures.

Under the Hood

Load balancers maintain a list of backend servers and track their status. For round robin, they keep a pointer to the last server used and move it forward for each request. For least connections, they maintain counters of active connections per server, updating them as requests start and finish. Health checks run periodically to mark servers as healthy or unhealthy. The load balancer routes incoming requests based on the chosen algorithm and current server states.

Why designed this way?

These algorithms were designed to balance simplicity and effectiveness. Round robin is easy to implement and works well when servers and requests are uniform. Least connections addresses uneven load by dynamically adapting to server usage. Alternatives like random or weighted algorithms exist but add complexity. The chosen designs prioritize predictable, fair distribution with minimal overhead.

┌─────────────────────────────┐
│        Load Balancer        │
│ ┌───────────────┐           │
│ │ Server List   │           │
│ │ ┌───────────┐ │           │
│ │ │ Server 1  │ │◄──────────┤
│ │ │ Server 2  │ │◄───┐      │
│ │ │ Server 3  │ │    │      │
│ │ └───────────┘ │    │      │
│ └───────────────┘    │      │
│  ┌───────────────┐   │      │
│  │ Round Robin   │───┘      │
│  │ Pointer to N  │          │
│  └───────────────┘          │
│  ┌───────────────┐          │
│  │ Least Conn.   │          │
│  │ Counters per  │          │
│  │ Server        │          │
│  └───────────────┘          │
└─────────────┬───────────────┘
              │
      ┌───────┴────────┐
      │ Backend Servers │
      └────────────────┘

Myth Busters - 4 Common Misconceptions

Quick: Does round robin always balance load perfectly regardless of request size? Commit yes or no.

Common Belief: Round robin always balances load evenly because it cycles through servers fairly.

Reality: Round robin does not account for differences in request processing time or server capacity, so load can be uneven in practice.

Why it matters: Assuming perfect balance can lead to overloaded servers and poor performance if requests vary in size or servers differ.

Quick: Does least connections always pick the fastest server? Commit yes or no.

Common Belief: Least connections always sends requests to the fastest server because it picks the least busy one.

Reality: Least connections picks the server with fewest active connections, but that server may still be slow or overloaded in other ways.

Why it matters: Relying solely on connection count can cause bottlenecks if server speed or health is not considered.

Quick: Do load balancers automatically detect server failures without extra setup? Commit yes or no.

Common Belief: Load balancers always know if a server is down and stop sending requests to it automatically.

Reality: Load balancers need configured health checks to detect failures; without them, they may send requests to dead servers.

Why it matters: Failing to configure health checks can cause downtime and errors for users.

Quick: Can a single load balancer handle unlimited traffic without becoming a bottleneck? Commit yes or no.

Common Belief: One load balancer can handle any amount of traffic by distributing it efficiently.

Reality: A single load balancer has limits and can become a bottleneck or single point of failure in large systems.

Why it matters: Ignoring this can cause system outages and poor scalability.

Expert Zone

1

Least connections algorithm can cause uneven load if connection durations vary widely, requiring additional weighting or smoothing.

2

Round robin can be combined with weights to favor more powerful servers, improving balance in heterogeneous environments.

3

Health checks must be carefully designed to avoid false positives or negatives, which can disrupt load balancing.

When NOT to use

Avoid round robin when servers have different capacities or request loads; prefer weighted or adaptive algorithms. Least connections may not suit systems with very short-lived connections or where connection count is not a good load indicator. In very large systems, consider consistent hashing or multi-level load balancing instead.

Production Patterns

In production, load balancers often combine algorithms with health checks and weights. Multi-level load balancing uses DNS or global load balancers to distribute traffic across regions, then local load balancers use round robin or least connections. Metrics and monitoring guide dynamic adjustments to algorithms.

Connections

Caching Strategies

Builds-on

Understanding load balancing helps optimize caching by directing requests to servers with cached data, reducing load and latency.

Traffic Signal Control

Similar pattern

Load balancing algorithms resemble traffic signals directing cars to avoid jams, showing how distributed control manages flow in different domains.

Queueing Theory

Builds-on

Load balancing relies on queueing theory principles to predict and manage request wait times and server utilization.

Common Pitfalls

#1 Ignoring server health status in load balancing decisions.

Wrong approach:

Load balancer sends requests to all servers equally without health checks.

Correct approach:

Load balancer performs regular health checks and excludes unhealthy servers from rotation.

Root cause:

Misunderstanding that load balancing alone ensures reliability without monitoring server health.

#2 Using round robin on servers with different capacities.

Wrong approach:

Configure round robin without weights on a mix of small and large servers.

Correct approach:

Use weighted round robin to assign more requests to powerful servers.

Root cause:

Assuming all servers are identical leads to uneven load and poor performance.

#3 Relying on least connections without considering connection duration.

Wrong approach:

Load balancer picks server with fewest connections, ignoring that some connections last much longer.

Correct approach:

Combine least connections with weighted or adaptive algorithms that consider connection duration.

Root cause:

Oversimplifying load measurement causes imbalance and bottlenecks.

Key Takeaways

Load balancing algorithms distribute incoming requests to servers to improve system performance and reliability.

Round robin cycles through servers in order, best for equal servers and similar requests but ignores load differences.

Least connections sends requests to the server with the fewest active connections, adapting to real-time load but requiring tracking.

Health checks are essential to avoid sending traffic to failed servers and maintain system availability.

In large or complex systems, combining algorithms, weights, and multi-level load balancing ensures scalability and fault tolerance.