Why load balancing matters in AWS - Why It Works This Way

Overview - Why load balancing matters

What is it?

Load balancing is a way to spread work evenly across many computers or servers. It helps make sure no single server gets too busy while others sit idle. This keeps websites and apps running smoothly and quickly. Load balancing also helps keep services available even if one server stops working.

Why it matters

Without load balancing, some servers would get overwhelmed and slow down or crash, making websites or apps hard to use or unavailable. This can frustrate users and hurt businesses. Load balancing solves this by sharing the work, so everything stays fast and reliable. It also helps handle sudden spikes in traffic without breaking.

Where it fits

Before learning load balancing, you should understand basic servers and how websites or apps run on them. After load balancing, you can learn about auto-scaling, which adds or removes servers automatically based on demand, and about advanced networking concepts like DNS and traffic routing.

Mental Model

Core Idea

Load balancing is like a traffic cop that directs requests evenly to multiple servers to keep everything running smoothly and reliably.

Think of it like...

Imagine a busy restaurant with many customers arriving at once. If all customers go to one waiter, that waiter gets overwhelmed and service slows down. A host at the entrance directs customers evenly to all waiters, so everyone gets served quickly and no waiter is overloaded.

┌───────────────┐
│   Clients     │
└──────┬────────┘
       │ Requests
       ▼
┌───────────────┐
│ Load Balancer │
└──────┬────────┘
       │ Distributes
       ▼
┌──────┬───────┬──────┐
│Server│Server │Server│
│  1   │  2    │  3   │
└──────┴───────┴──────┘

Build-Up - 6 Steps

1

Foundation

What is Load Balancing

Concept: Load balancing means sharing work across multiple servers to avoid overload.

When many users visit a website, their requests need to be handled by servers. If only one server handles all requests, it can get too busy and slow down. Load balancing spreads these requests across several servers so each one handles a fair share.

Result

Servers share the work, so no single server is overwhelmed.

Understanding that load balancing prevents overload is key to keeping services fast and reliable.

2

Foundation

Types of Load Balancers

Concept: There are different ways to balance load, like hardware devices or software services.

Load balancers can be physical devices or software running in the cloud. AWS offers Elastic Load Balancing (ELB) services that automatically distribute traffic. These can work at different levels, like directing traffic based on IP addresses or the content of requests.

Result

You know the basic tools used to balance load in real systems.

Knowing the types helps choose the right load balancing method for your needs.

3

Intermediate

How Load Balancers Distribute Traffic

🤔Before reading on: do you think load balancers send requests randomly or follow a pattern? Commit to your answer.

Concept: Load balancers use rules or algorithms to decide which server gets each request.

Common methods include round-robin (sending requests in order), least connections (sending to the server with fewest active users), or IP hash (sending based on client IP). These methods help balance load fairly and keep sessions consistent.

Result

Traffic is distributed efficiently, improving speed and reliability.

Understanding distribution methods helps optimize performance and user experience.

4

Intermediate

Load Balancing and High Availability

🤔Before reading on: do you think load balancers can help if a server crashes? Commit to your answer.

Concept: Load balancers detect unhealthy servers and stop sending them traffic to keep services available.

Load balancers regularly check if servers respond correctly. If a server fails, the load balancer routes traffic only to healthy servers. This prevents users from seeing errors and keeps the service running smoothly.

Result

Services stay available even if some servers fail.

Knowing this shows how load balancing supports reliability and fault tolerance.

5

Advanced

Scaling with Load Balancers

🤔Before reading on: do you think load balancers can work with servers that come and go dynamically? Commit to your answer.

Concept: Load balancers work with auto-scaling to add or remove servers based on demand.

In cloud environments like AWS, servers can be added or removed automatically. Load balancers update their list of servers to include new ones and exclude removed ones. This dynamic scaling helps handle traffic spikes without manual intervention.

Result

Systems can grow or shrink smoothly to match user demand.

Understanding this reveals how load balancing enables flexible, cost-effective infrastructure.

6

Expert

Load Balancer Internals and Performance

🤔Before reading on: do you think load balancers add noticeable delay to requests? Commit to your answer.

Concept: Load balancers use efficient algorithms and hardware acceleration to minimize delay while managing traffic.

Load balancers operate at network layers and use optimized code or specialized hardware to quickly inspect and forward requests. They maintain connection states and handle SSL encryption without slowing down traffic noticeably. Misconfiguration or overload can cause delays, so tuning is important.

Result

Load balancing adds minimal delay while improving reliability and scalability.

Knowing internal workings helps troubleshoot performance issues and optimize setups.

Under the Hood

Load balancers sit between clients and servers, receiving all incoming requests. They inspect each request and decide which server to forward it to based on configured rules. They keep track of server health by sending regular checks. When a server fails, the load balancer removes it from the pool. They may also handle encryption and session persistence to keep user experience seamless.

Why designed this way?

Load balancing was designed to solve the problem of single points of failure and overloaded servers. Early systems had one server that could crash or slow down under load. Distributing requests improves reliability and performance. The design balances simplicity, speed, and flexibility, allowing many algorithms and health checks to keep systems robust.

┌───────────────┐
│   Clients     │
└──────┬────────┘
       │ Requests
       ▼
┌───────────────┐
│ Load Balancer │
│  - Receives   │
│  - Checks     │
│  - Routes     │
└──────┬────────┘
       │
       ▼
┌──────┬───────┬──────┐
│Server│Server │Server│
│  1   │  2    │  3   │
└──────┴───────┴──────┘

Health Checks: Load Balancer → Servers
Routing Decisions: Load Balancer → Servers

Myth Busters - 4 Common Misconceptions

Quick: Does a load balancer always send requests to the least busy server? Commit to yes or no.

Common Belief: Load balancers always send requests to the least busy server to optimize performance.

Reality: Load balancers use different algorithms; some like round-robin do not consider server load, while others do. The choice depends on the use case.

Why it matters: Assuming all load balancers optimize for load can lead to wrong configurations and unexpected bottlenecks.

Quick: Do load balancers eliminate the need for multiple servers? Commit to yes or no.

Common Belief: Using a load balancer means you only need one server because it manages traffic.

Reality: Load balancers require multiple servers to distribute traffic; they do not replace servers but help manage many servers efficiently.

Why it matters: Thinking a load balancer replaces servers can cause under-provisioning and poor performance.

Quick: Can a load balancer fix a slow application code problem? Commit to yes or no.

Common Belief: Load balancers can fix slow applications by spreading requests around.

Reality: Load balancers distribute traffic but cannot fix slow or inefficient application code running on servers.

Why it matters: Relying on load balancing alone can hide performance issues that need code optimization.

Quick: Does a load balancer add significant delay to user requests? Commit to yes or no.

Common Belief: Load balancers add noticeable delay because they inspect and route every request.

Reality: Modern load balancers are optimized to add minimal delay, often unnoticeable to users.

Why it matters: Fearing delay may prevent using load balancers, missing out on their reliability benefits.

Expert Zone

1

Some load balancers support session persistence, keeping a user connected to the same server for stateful applications, which is critical for user experience.

2

Health checks can be customized to check specific application endpoints, not just server availability, improving failure detection accuracy.

3

Load balancers can operate at different network layers (Layer 4 vs Layer 7), affecting how they inspect and route traffic with trade-offs in complexity and flexibility.

When NOT to use

Load balancing is not suitable for very simple applications with low traffic where a single server suffices. Also, for applications requiring strict data locality or very low latency, direct connections may be better. Alternatives include client-side load balancing or DNS-based load distribution.

Production Patterns

In production, load balancers are combined with auto-scaling groups to handle traffic spikes automatically. They are configured with health checks and SSL termination. Multi-region load balancing is used for disaster recovery. Monitoring and logging on load balancers help detect issues early.

Connections

Auto-scaling

Builds-on

Load balancing works hand-in-hand with auto-scaling to add or remove servers dynamically, ensuring efficient resource use and consistent performance.

DNS (Domain Name System)

Complementary

DNS can distribute traffic at a high level by directing users to different data centers, while load balancers distribute traffic within a data center, together improving global availability.

Traffic Management in Road Networks

Similar pattern

Both load balancing and road traffic management aim to prevent congestion by directing flows evenly, showing how principles of flow control apply across technology and urban planning.

Common Pitfalls

#1 Not configuring health checks, so traffic is sent to failed servers.

Wrong approach:

LoadBalancer:
Servers: [Server1, Server2]
HealthCheck: None

Correct approach:

LoadBalancer:
Servers: [Server1, Server2]
HealthCheck:
Path: /health
Interval: 30s

Root cause:

Assuming servers are always healthy leads to downtime when a server fails but still receives traffic.

#2 Using a single load balancer without redundancy, creating a single point of failure.

Wrong approach:

Deploy one load balancer without backup or failover.

Correct approach:

Deploy multiple load balancers with failover or use managed services that provide high availability.

Root cause:

Not planning for load balancer failure risks total service outage.

#3 Ignoring session persistence for stateful applications, causing user sessions to break.

Wrong approach:

LoadBalancer:
Algorithm: RoundRobin
SessionPersistence: None

Correct approach:

LoadBalancer:
Algorithm: RoundRobin
SessionPersistence: Enabled (sticky sessions)

Root cause:

Not understanding application needs leads to poor user experience with lost sessions.

Key Takeaways

Load balancing spreads user requests across multiple servers to keep services fast and reliable.

It prevents any single server from becoming overwhelmed, which avoids slowdowns and crashes.

Load balancers detect unhealthy servers and stop sending them traffic to maintain availability.

They work closely with auto-scaling to handle changing traffic automatically and efficiently.

Understanding load balancing internals helps optimize performance and troubleshoot real-world issues.