Sticky sessions in HLD - Scalability & System Analysis

Scalability Analysis - Sticky sessions

Growth Table: Sticky Sessions

Users	What Changes?
100 users	Single load balancer with sticky session enabled; one or two app servers; session stored in server memory.
10,000 users	More app servers added; load balancer manages sticky sessions via cookies; session affinity maintained; memory usage grows.
1,000,000 users	Multiple load balancers with consistent hashing or session-aware routing; session replication or centralized session store needed; risk of uneven load due to sticky sessions.
100,000,000 users	Global load balancing with geo-distribution; session data stored in distributed cache or database; sticky sessions cause scaling and failover challenges; need session statelessness or token-based sessions.

First Bottleneck

The first bottleneck is the application server memory and load balancer's ability to maintain session affinity. As users grow, servers become overloaded with session data, and load balancers struggle to route requests correctly, causing uneven load and potential downtime.

Scaling Solutions

• Centralized Session Store: Move sessions to a fast shared cache like Redis to avoid server memory overload and allow any server to handle requests.
• Session Replication: Replicate session data across app servers to handle failover but increases network and memory usage.
• Stateless Sessions: Use tokens (e.g., JWT) so servers do not store session data, removing sticky session dependency.
• Load Balancer Improvements: Use consistent hashing or advanced routing to better distribute load while maintaining session affinity.
• Horizontal Scaling: Add more app servers and load balancers to distribute traffic.
• Geo-Distributed Architecture: Use regional load balancers and session stores to reduce latency and improve availability.

Cost Analysis

At 1M users with 10 requests per user per minute, that is about 166,667 requests per second. A single app server can handle ~5,000 concurrent connections, so at least 334 servers are needed. Session data per user might be ~1KB, so 1M sessions require ~1GB memory in cache. Network bandwidth must support session replication traffic and user requests, potentially hundreds of MB/s. Centralized session stores add operational cost but reduce server memory needs.

Interview Tip

Start by explaining what sticky sessions are and why they are used. Then discuss how they work at small scale and what breaks as users grow. Identify the bottleneck clearly. Propose solutions step-by-step, weighing pros and cons. Mention trade-offs like complexity vs performance. Finally, discuss how to measure and monitor session-related metrics.

Self Check

Your database handles 1000 QPS. Traffic grows 10x. What do you do first?

Answer: Add read replicas and implement caching to reduce database load before scaling vertically or sharding.

Key Result

Sticky sessions work well at small scale but cause uneven load and memory pressure as users grow; moving session data out of app servers and using stateless sessions or centralized stores is key to scaling.

Practice

1. What is the main purpose of sticky sessions in system design?

easy

A. To keep a user connected to the same server during their session

B. To distribute user requests evenly across all servers

C. To store session data in a shared database

D. To encrypt user data during transmission

Solution

1. Step 1: Understand sticky sessions concept

   Sticky sessions ensure that a user's requests go to the same server to maintain session state.

2. Step 2: Compare with other options

   Options B and C describe load balancing and shared storage, not sticky sessions. To encrypt user data during transmission is about security, unrelated here.

3. Final Answer:

   To keep a user connected to the same server during their session -> Option A

4. Quick Check:

   Sticky sessions = same server connection [OK]

Hint: Sticky sessions keep user-server link constant [OK]

Common Mistakes:

• Confusing sticky sessions with load balancing
• Thinking session data is always shared
• Assuming sticky sessions encrypt data

2. Which method is commonly used to implement sticky sessions in a load balancer?

easy

A. Randomly assigning servers per request

B. Encrypting all session data

C. Caching data in a CDN

D. Using IP hashing to route requests

Solution

1. Step 1: Identify sticky session implementation methods

   IP hashing is a common way to ensure requests from the same user go to the same server.

2. Step 2: Eliminate unrelated options

   Encrypting data and caching in CDN do not control session routing. Random assignment breaks stickiness.

3. Final Answer:

   Using IP hashing to route requests -> Option D

4. Quick Check:

   Sticky sessions use IP hashing [OK]

Hint: IP hashing helps keep sessions sticky [OK]

Common Mistakes:

• Confusing encryption with session routing
• Thinking CDN caching manages sessions
• Assuming random server assignment keeps sessions sticky

3. Consider a load balancer using cookie-based sticky sessions. If a user's first request is routed to Server A, where will the second request go?

medium

A. Server B

B. Server A

C. Random server

D. Load balancer itself

Solution

1. Step 1: Understand cookie-based sticky sessions

   The load balancer sets a cookie to identify the server handling the user's session.

2. Step 2: Follow the cookie routing logic

   Subsequent requests with the cookie go to the same server, Server A in this case.

3. Final Answer:

   Server A -> Option B

4. Quick Check:

   Cookie sticky sessions route to same server [OK]

Hint: Cookie keeps user on original server [OK]

Common Mistakes:

• Assuming requests go to different servers
• Thinking load balancer handles requests directly
• Confusing random routing with sticky sessions

4. A system uses IP hashing for sticky sessions but users report inconsistent session behavior. What is the most likely cause?

medium

A. Users have dynamic IP addresses changing frequently

B. Session data is stored in a shared database

C. Load balancer is not configured for SSL

D. Cookies are disabled in user browsers

Solution

1. Step 1: Analyze IP hashing limitations

   IP hashing depends on stable IP addresses to route requests consistently.

2. Step 2: Identify cause of inconsistency

   Dynamic IPs cause requests to appear from different IPs, breaking stickiness.

3. Final Answer:

   Users have dynamic IP addresses changing frequently -> Option A

4. Quick Check:

   Dynamic IPs break IP hashing sticky sessions [OK]

Hint: Dynamic IPs break IP hashing stickiness [OK]

Common Mistakes:

• Blaming shared database for session routing
• Confusing SSL with session stickiness
• Assuming cookies affect IP hashing

5. You design a web app with multiple servers storing session data locally. To ensure users always reach the same server, which approach is best and why?

hard

A. Use IP hashing but allow fallback to any server if IP changes

B. Use a shared database for session storage and random load balancing

C. Use sticky sessions with cookie-based routing to bind users to servers

D. Disable sticky sessions and rely on client-side session storage

Solution

1. Step 1: Understand session storage setup

   Sessions stored locally require users to connect to the same server to access their data.

2. Step 2: Evaluate approaches for consistent routing

   Cookie-based sticky sessions reliably bind users to servers, ensuring session continuity.

3. Step 3: Compare other options

   Shared DB removes need for stickiness but contradicts local storage. IP hashing can fail with IP changes. Client-side storage shifts session responsibility.

4. Final Answer:

   Use sticky sessions with cookie-based routing to bind users to servers -> Option C

5. Quick Check:

   Local sessions need sticky cookie routing [OK]

Hint: Local sessions need cookie sticky routing [OK]

Common Mistakes:

• Choosing shared DB when sessions are local
• Relying on IP hashing despite IP changes
• Ignoring server session storage needs