Why load balancing matters in Azure - Why It Works This Way

Overview - Why load balancing matters

What is it?

Load balancing is a way to spread work evenly across many computers or servers. It helps make sure no single server gets too busy while others are idle. This keeps websites and apps running smoothly and quickly. Load balancing also helps keep services available even if some servers fail.

Why it matters

Without load balancing, some servers would get overwhelmed and slow down or crash, making websites or apps hard to use or unavailable. This can frustrate users and cause lost business. Load balancing ensures a fair share of work, improving speed and reliability that people expect every day.

Where it fits

Before learning load balancing, you should understand basic cloud servers and networking. After this, you can learn about advanced traffic management, auto-scaling, and fault tolerance to build highly reliable cloud systems.

Mental Model

Core Idea

Load balancing is like a traffic cop that directs requests evenly to servers so none get stuck in a jam.

Think of it like...

Imagine a busy restaurant with many waiters. The host seats new guests evenly among waiters so no one waiter is overwhelmed and all guests get good service quickly.

┌───────────────┐
│   Clients     │
└──────┬────────┘
       │ Requests
       ▼
┌───────────────┐
│ Load Balancer │
└──────┬────────┘
       │ Distributes
       ▼
┌──────┴───────┐  ┌──────┴───────┐  ┌──────┴───────┐
│   Server 1   │  │   Server 2   │  │   Server 3   │
└──────────────┘  └──────────────┘  └──────────────┘

Build-Up - 7 Steps

1

Foundation

What is Load Balancing?

Concept: Load balancing means sharing work across multiple servers to avoid overload.

When many people visit a website, their requests go to servers. If one server gets too many requests, it slows down or crashes. Load balancing sends requests to different servers to keep things fast and stable.

Result

Requests are spread out, so no single server is overwhelmed.

Understanding load balancing starts with seeing how spreading work prevents slowdowns and crashes.

2

Foundation

Types of Load Balancers

Concept: There are different ways to balance load, like hardware devices or software services.

Hardware load balancers are physical devices in data centers. Software load balancers run on servers or cloud platforms like Azure. Azure offers services like Azure Load Balancer and Azure Application Gateway to manage traffic.

Result

You know the basic tools used to balance load in cloud environments.

Knowing the types helps choose the right tool for different needs and environments.

3

Intermediate

How Load Balancers Distribute Traffic

🤔Before reading on: do you think load balancers send requests randomly or follow a pattern? Commit to your answer.

Concept: Load balancers use rules to decide which server gets each request.

Common methods include round-robin (each server in turn), least connections (server with fewest active users), and IP hash (based on client IP). These methods help balance load fairly and keep sessions consistent.

Result

Traffic is distributed according to chosen rules, improving performance and reliability.

Understanding distribution methods reveals how load balancers optimize user experience and resource use.

4

Intermediate

Health Probes Keep Services Reliable

🤔Before reading on: do you think load balancers send traffic to all servers regardless of health? Commit to your answer.

Concept: Load balancers check if servers are healthy before sending them traffic.

Azure load balancers use health probes that regularly test servers by sending simple requests. If a server doesn't respond correctly, the load balancer stops sending it traffic until it recovers.

Result

Users avoid broken or slow servers, improving uptime and experience.

Knowing health probes explains how load balancers maintain service availability automatically.

5

Intermediate

Load Balancing and Scalability

Concept: Load balancing works with scaling to handle more users by adding servers.

When demand grows, cloud systems add more servers (scaling out). Load balancers then spread traffic across all servers. This combination lets apps handle big spikes without slowing down.

Result

Systems can grow smoothly to meet user demand.

Seeing load balancing as part of scaling helps understand how cloud apps stay fast under pressure.

6

Advanced

Session Persistence and Sticky Sessions

🤔Before reading on: do you think load balancers always send each user to a different server? Commit to your answer.

Concept: Sometimes users must keep talking to the same server for their session to work properly.

Sticky sessions make the load balancer send all requests from one user to the same server. This is important for apps that store user info locally. Azure Application Gateway supports this feature.

Result

User sessions stay consistent, avoiding errors or lost data.

Understanding session persistence reveals how load balancing adapts to app needs beyond simple traffic spread.

7

Expert

Load Balancer Failover and High Availability

🤔Before reading on: do you think load balancers themselves can fail without backup? Commit to your answer.

Concept: Load balancers must also be reliable and avoid becoming a single point of failure.

Azure uses redundant load balancers and automatic failover to keep traffic flowing even if one load balancer fails. This design ensures continuous service availability.

Result

Load balancing remains effective even during hardware or software failures.

Knowing load balancer failover mechanisms highlights the full reliability chain in cloud infrastructure.

Under the Hood

Load balancers receive incoming network requests and use configured rules to select a backend server. They track server health via probes and maintain connection states if needed. In Azure, load balancers operate at different layers: Azure Load Balancer works at the transport layer (TCP/UDP), while Application Gateway works at the application layer (HTTP/HTTPS), allowing more advanced routing.

Why designed this way?

Load balancing was designed to solve the problem of uneven workload distribution and single points of failure. Early systems had simple round-robin methods, but as apps grew complex, features like health probes and session persistence were added. Azure's layered approach allows flexibility for different app types and scales.

┌───────────────┐
│ Client Request│
└──────┬────────┘
       │
       ▼
┌───────────────┐
│ Load Balancer │
├───────────────┤
│ Health Probes │
│ Distribution  │
│ Rules         │
└──────┬────────┘
       │
       ▼
┌──────┴───────┐  ┌──────┴───────┐  ┌──────┴───────┐
│ Server 1     │  │ Server 2     │  │ Server 3     │
│ (Healthy)    │  │ (Healthy)    │  │ (Unhealthy)  │
└──────────────┘  └──────────────┘  └──────────────┘
       ▲               ▲               ▲
       └───────────────┴───────────────┘
               Health Probe Checks

Myth Busters - 4 Common Misconceptions

Quick: Do load balancers always send traffic equally to all servers, no matter what? Commit to yes or no.

Common Belief: Load balancers just split traffic evenly among all servers all the time.

Reality: Load balancers send traffic based on rules and server health; unhealthy servers get no traffic.

Why it matters: Ignoring health checks can cause users to get errors or slow responses from broken servers.

Quick: Do you think load balancers can fix slow servers by themselves? Commit to yes or no.

Common Belief: Load balancers can speed up slow servers by balancing load.

Reality: Load balancers only distribute traffic; they cannot fix server performance issues.

Why it matters: Relying on load balancing alone can hide underlying problems, leading to poor user experience.

Quick: Do you think session persistence is always on by default? Commit to yes or no.

Common Belief: Load balancers always send users to the same server to keep sessions consistent.

Reality: Session persistence is optional and must be configured; otherwise, users may be sent to different servers.

Why it matters: Not configuring persistence can break apps that rely on session data stored on one server.

Quick: Do you think load balancers themselves never fail? Commit to yes or no.

Common Belief: Load balancers are always reliable and cannot fail.

Reality: Load balancers can fail, so cloud providers build redundancy and failover to keep services running.

Why it matters: Assuming load balancers never fail risks single points of failure and downtime.

Expert Zone

1

Some load balancing algorithms can cause uneven load if server capacities differ; weighting servers helps balance this.

2

Application-layer load balancers can inspect and route traffic based on content, enabling advanced scenarios like A/B testing.

3

Health probes must be carefully designed to avoid false positives or negatives that can cause traffic to be sent to unhealthy servers.

When NOT to use

Load balancing is not suitable for stateful applications that cannot share session data without additional mechanisms. In such cases, consider sticky sessions or distributed caches. Also, for very low traffic or single-server apps, load balancing adds unnecessary complexity.

Production Patterns

In production, Azure load balancers are combined with auto-scaling groups to add or remove servers automatically. Application Gateway is used for web apps needing SSL termination and URL-based routing. Multi-region load balancing distributes traffic globally for disaster recovery and latency optimization.

Connections

Auto-scaling

Load balancing works together with auto-scaling to handle changing traffic by adding or removing servers.

Understanding load balancing helps grasp how cloud systems grow and shrink smoothly to meet demand.

Fault Tolerance

Load balancing contributes to fault tolerance by routing traffic away from failed servers.

Knowing load balancing clarifies how systems stay available despite hardware or software failures.

Traffic Management in Road Systems

Both manage flow to avoid congestion and ensure smooth movement.

Seeing load balancing like road traffic control reveals universal principles of distributing work to prevent jams.

Common Pitfalls

#1 Sending traffic to unhealthy servers causes errors.

Wrong approach:

Configure load balancer without health probes or disable them.
Example: Azure Load Balancer with no health probe setup.

Correct approach:

Configure health probes to regularly check server health.
Example: Azure Load Balancer with TCP health probe on port 80.

Root cause:

Not understanding that load balancers rely on health checks to avoid broken servers.

#2 Not configuring session persistence breaks user sessions.

Wrong approach:

Use load balancer without sticky sessions for apps needing session data.
Example: Azure Application Gateway without cookie-based affinity.

Correct approach:

Enable session persistence (cookie-based affinity) for stateful apps.
Example: Azure Application Gateway with cookie affinity enabled.

Root cause:

Assuming load balancers always keep users on the same server by default.

#3 Using simple round-robin when servers have different capacities causes overload.

Wrong approach:

Configure load balancer with round-robin only, ignoring server power differences.

Correct approach:

Use weighted load balancing to assign more traffic to stronger servers.

Root cause:

Not considering server capacity differences in traffic distribution.

Key Takeaways

Load balancing spreads user requests across multiple servers to keep apps fast and reliable.

It uses rules and health checks to send traffic only to healthy servers, avoiding downtime.

Session persistence is important for apps that need users to stay connected to the same server.

Load balancers themselves must be highly available to prevent becoming a single point of failure.

In cloud systems like Azure, load balancing works closely with scaling and fault tolerance to build resilient applications.