Sticky sessions in HLD - System Design Exercise

Design: Sticky Sessions Load Balancing

Design the load balancing and session management system to support sticky sessions. Out of scope: detailed backend application logic and database design.

Functional Requirements

FR1: Maintain user session affinity to the same backend server during a session

FR2: Support at least 10,000 concurrent users

FR3: Ensure session persistence even if load balancer restarts

FR4: Handle session expiration after 30 minutes of inactivity

FR5: Support failover to another server if the original server is down

Non-Functional Requirements

NFR1: API response latency p99 under 150ms

NFR2: System availability 99.9% uptime (max 8.77 hours downtime per year)

NFR3: Session data size limited to 4KB per user

NFR4: Load balancer must handle 20,000 requests per second

Think Before You Design

Questions to Ask

❓ Question 1

❓ Question 2

❓ Question 3

❓ Question 4

❓ Question 5

Key Components

Load balancer with sticky session support

Session store (in-memory cache like Redis or local server memory)

Backend application servers

Health check service for backend servers

Client (browser or app) managing session cookies

Design Patterns

Session affinity using cookies or IP hashing

Centralized session store for failover

Sticky session with load balancer cookie insertion

Session replication across backend servers

Fallback routing on server failure

Reference Architecture

Client
  |
  | HTTP Request with session cookie
  v
Load Balancer (sticky session enabled)
  |-- Checks session cookie to route to same backend server
  |-- If no cookie, routes to least loaded server and sets cookie
  v
Backend Servers (App Server 1, App Server 2, ...)
  |
  | Store session data in centralized Redis cache
  v
Redis Session Store

Health Check Service monitors backend servers and informs Load Balancer

Components

Load Balancer

Nginx or AWS ALB

Route client requests to backend servers maintaining session affinity using cookies

Backend Application Servers

Node.js / Java / Python servers

Process user requests and read/write session data

Redis Session Store

Redis in-memory cache

Centralized storage of session data for persistence and failover

Health Check Service

Custom or built-in load balancer health checks

Monitor backend server health and update load balancer routing

Client

Web browser or mobile app

Send session cookie with requests to maintain sticky session

Request Flow

1. Client sends HTTP request with session cookie to Load Balancer

2. Load Balancer reads cookie and routes request to the same backend server

3. Backend server processes request and reads/writes session data in Redis

4. If no session cookie, Load Balancer routes to a backend server and sets a session cookie in response

5. Health Check Service monitors backend servers and informs Load Balancer to avoid unhealthy servers

6. If backend server fails, Load Balancer routes requests to another server and client gets a new session cookie

Database Schema

Entity: Session Attributes: session_id (PK), user_id, data (JSON), last_accessed_timestamp Relationships: None (flat key-value store in Redis) Sessions expire after 30 minutes of inactivity using Redis TTL feature.

Scaling Discussion

Bottlenecks

Load balancer CPU and memory limits under high request rate

Redis session store becoming a single point of failure or bottleneck

Backend servers overloaded with session read/write operations

Session cookie size limits affecting client requests

Failover delay when backend server goes down

Solutions

Use multiple load balancers with DNS round-robin or cloud managed load balancers

Deploy Redis in clustered mode with replication and sharding

Cache session data locally on backend servers with periodic sync to Redis

Keep session data minimal and use session IDs referencing server-side data

Implement fast health checks and automatic failover routing in load balancer

Interview Tips

Time: Spend 10 minutes clarifying requirements and constraints, 20 minutes designing architecture and data flow, 10 minutes discussing scaling and trade-offs, 5 minutes summarizing.

Explain why sticky sessions are needed and how session affinity works

Discuss pros and cons of storing session data in cookies vs centralized store

Describe how load balancer routes requests based on session cookie

Highlight failover handling and session persistence strategies

Address scaling challenges and solutions for high concurrency

Practice

1. What is the main purpose of sticky sessions in system design?

easy

A. To keep a user connected to the same server during their session

B. To distribute user requests evenly across all servers

C. To store session data in a shared database

D. To encrypt user data during transmission

Solution

1. Step 1: Understand sticky sessions concept

   Sticky sessions ensure that a user's requests go to the same server to maintain session state.

2. Step 2: Compare with other options

   Options B and C describe load balancing and shared storage, not sticky sessions. To encrypt user data during transmission is about security, unrelated here.

3. Final Answer:

   To keep a user connected to the same server during their session -> Option A

4. Quick Check:

   Sticky sessions = same server connection [OK]

Hint: Sticky sessions keep user-server link constant [OK]

Common Mistakes:

• Confusing sticky sessions with load balancing
• Thinking session data is always shared
• Assuming sticky sessions encrypt data

2. Which method is commonly used to implement sticky sessions in a load balancer?

easy

A. Randomly assigning servers per request

B. Encrypting all session data

C. Caching data in a CDN

D. Using IP hashing to route requests

Solution

1. Step 1: Identify sticky session implementation methods

   IP hashing is a common way to ensure requests from the same user go to the same server.

2. Step 2: Eliminate unrelated options

   Encrypting data and caching in CDN do not control session routing. Random assignment breaks stickiness.

3. Final Answer:

   Using IP hashing to route requests -> Option D

4. Quick Check:

   Sticky sessions use IP hashing [OK]

Hint: IP hashing helps keep sessions sticky [OK]

Common Mistakes:

• Confusing encryption with session routing
• Thinking CDN caching manages sessions
• Assuming random server assignment keeps sessions sticky

3. Consider a load balancer using cookie-based sticky sessions. If a user's first request is routed to Server A, where will the second request go?

medium

A. Server B

B. Server A

C. Random server

D. Load balancer itself

Solution

1. Step 1: Understand cookie-based sticky sessions

   The load balancer sets a cookie to identify the server handling the user's session.

2. Step 2: Follow the cookie routing logic

   Subsequent requests with the cookie go to the same server, Server A in this case.

3. Final Answer:

   Server A -> Option B

4. Quick Check:

   Cookie sticky sessions route to same server [OK]

Hint: Cookie keeps user on original server [OK]

Common Mistakes:

• Assuming requests go to different servers
• Thinking load balancer handles requests directly
• Confusing random routing with sticky sessions

4. A system uses IP hashing for sticky sessions but users report inconsistent session behavior. What is the most likely cause?

medium

A. Users have dynamic IP addresses changing frequently

B. Session data is stored in a shared database

C. Load balancer is not configured for SSL

D. Cookies are disabled in user browsers

Solution

1. Step 1: Analyze IP hashing limitations

   IP hashing depends on stable IP addresses to route requests consistently.

2. Step 2: Identify cause of inconsistency

   Dynamic IPs cause requests to appear from different IPs, breaking stickiness.

3. Final Answer:

   Users have dynamic IP addresses changing frequently -> Option A

4. Quick Check:

   Dynamic IPs break IP hashing sticky sessions [OK]

Hint: Dynamic IPs break IP hashing stickiness [OK]

Common Mistakes:

• Blaming shared database for session routing
• Confusing SSL with session stickiness
• Assuming cookies affect IP hashing

5. You design a web app with multiple servers storing session data locally. To ensure users always reach the same server, which approach is best and why?

hard

A. Use IP hashing but allow fallback to any server if IP changes

B. Use a shared database for session storage and random load balancing

C. Use sticky sessions with cookie-based routing to bind users to servers

D. Disable sticky sessions and rely on client-side session storage

Solution

1. Step 1: Understand session storage setup

   Sessions stored locally require users to connect to the same server to access their data.

2. Step 2: Evaluate approaches for consistent routing

   Cookie-based sticky sessions reliably bind users to servers, ensuring session continuity.

3. Step 3: Compare other options

   Shared DB removes need for stickiness but contradicts local storage. IP hashing can fail with IP changes. Client-side storage shifts session responsibility.

4. Final Answer:

   Use sticky sessions with cookie-based routing to bind users to servers -> Option C

5. Quick Check:

   Local sessions need sticky cookie routing [OK]

Hint: Local sessions need cookie sticky routing [OK]

Common Mistakes:

• Choosing shared DB when sessions are local
• Relying on IP hashing despite IP changes
• Ignoring server session storage needs