Sticky sessions in HLD - System Design Exercise

Design: Sticky Sessions Load Balancing

Design the load balancing and session management system to support sticky sessions. Out of scope: detailed backend application logic and database design.

Functional Requirements

FR1: Maintain user session affinity to the same backend server during a session

FR2: Support at least 10,000 concurrent users

FR3: Ensure session persistence even if load balancer restarts

FR4: Handle session expiration after 30 minutes of inactivity

FR5: Support failover to another server if the original server is down

Non-Functional Requirements

NFR1: API response latency p99 under 150ms

NFR2: System availability 99.9% uptime (max 8.77 hours downtime per year)

NFR3: Session data size limited to 4KB per user

NFR4: Load balancer must handle 20,000 requests per second

Think Before You Design

Questions to Ask

❓ Question 1

❓ Question 2

❓ Question 3

❓ Question 4

❓ Question 5

Key Components

Load balancer with sticky session support

Session store (in-memory cache like Redis or local server memory)

Backend application servers

Health check service for backend servers

Client (browser or app) managing session cookies

Design Patterns

Session affinity using cookies or IP hashing

Centralized session store for failover

Sticky session with load balancer cookie insertion

Session replication across backend servers

Fallback routing on server failure

Reference Architecture

Client
  |
  | HTTP Request with session cookie
  v
Load Balancer (sticky session enabled)
  |-- Checks session cookie to route to same backend server
  |-- If no cookie, routes to least loaded server and sets cookie
  v
Backend Servers (App Server 1, App Server 2, ...)
  |
  | Store session data in centralized Redis cache
  v
Redis Session Store

Health Check Service monitors backend servers and informs Load Balancer

Components

Load Balancer

Nginx or AWS ALB

Route client requests to backend servers maintaining session affinity using cookies

Backend Application Servers

Node.js / Java / Python servers

Process user requests and read/write session data

Redis Session Store

Redis in-memory cache

Centralized storage of session data for persistence and failover

Health Check Service

Custom or built-in load balancer health checks

Monitor backend server health and update load balancer routing

Client

Web browser or mobile app

Send session cookie with requests to maintain sticky session

Request Flow

1. Client sends HTTP request with session cookie to Load Balancer

2. Load Balancer reads cookie and routes request to the same backend server

3. Backend server processes request and reads/writes session data in Redis

4. If no session cookie, Load Balancer routes to a backend server and sets a session cookie in response

5. Health Check Service monitors backend servers and informs Load Balancer to avoid unhealthy servers

6. If backend server fails, Load Balancer routes requests to another server and client gets a new session cookie

Database Schema

Entity: Session Attributes: session_id (PK), user_id, data (JSON), last_accessed_timestamp Relationships: None (flat key-value store in Redis) Sessions expire after 30 minutes of inactivity using Redis TTL feature.

Scaling Discussion

Bottlenecks

Load balancer CPU and memory limits under high request rate

Redis session store becoming a single point of failure or bottleneck

Backend servers overloaded with session read/write operations

Session cookie size limits affecting client requests

Failover delay when backend server goes down

Solutions

Use multiple load balancers with DNS round-robin or cloud managed load balancers

Deploy Redis in clustered mode with replication and sharding

Cache session data locally on backend servers with periodic sync to Redis

Keep session data minimal and use session IDs referencing server-side data

Implement fast health checks and automatic failover routing in load balancer

Interview Tips

Time: Spend 10 minutes clarifying requirements and constraints, 20 minutes designing architecture and data flow, 10 minutes discussing scaling and trade-offs, 5 minutes summarizing.

Explain why sticky sessions are needed and how session affinity works

Discuss pros and cons of storing session data in cookies vs centralized store

Describe how load balancer routes requests based on session cookie

Highlight failover handling and session persistence strategies

Address scaling challenges and solutions for high concurrency