Consistent hashing in HLD - System Design Guide

Problem Statement

When a distributed system adds or removes servers, a naive hashing method causes most keys to remap to different servers. This leads to massive cache misses or data reshuffling, causing high latency and instability.

Solution

Consistent hashing arranges servers and keys on a circular hash ring. Each key is assigned to the next server clockwise on the ring. When servers join or leave, only a small portion of keys need to move, minimizing disruption and maintaining balance.

Architecture

┌─────────────┐
│   Hash Ring  │
│             │
│  ● Server A │
│       ↑     │
│  ● Server B │
│       ↑     │
│  ● Server C │
│             │
│  ● Key 1 → Server B
│  ● Key 2 → Server C
│  ● Key 3 → Server A
└─────────────┘

This diagram shows servers and keys placed on a circular hash ring. Each key maps to the next server clockwise, illustrating how consistent hashing distributes keys.

Trade-offs

✓ Pros

→

Minimizes key remapping when servers change, reducing cache misses and data movement.

→

Balances load evenly across servers by spreading keys uniformly on the ring.

→

Scales smoothly as servers are added or removed without full system rehashing.

✗ Cons

→

Requires careful implementation of virtual nodes to avoid uneven load distribution.

→

Adds complexity compared to simple modulo hashing.

→

Does not handle heterogeneous server capacities without extra logic.

Use when you have a distributed cache or storage system with frequent server changes and need to minimize data reshuffling, typically at scales above hundreds of servers.

Avoid when the system has very few servers (less than 10) or when server membership rarely changes, as the added complexity is unnecessary.

Real World Examples

Amazon

Amazon DynamoDB uses consistent hashing to distribute data partitions across nodes, enabling smooth scaling and fault tolerance.

Netflix

Netflix uses consistent hashing in its caching layer to ensure user sessions consistently route to the same cache server, reducing re-authentication overhead.

LinkedIn

LinkedIn applies consistent hashing in its distributed key-value stores to handle node failures and additions with minimal data movement.

Alternatives

Modulo Hashing

Assigns keys by key hash modulo number of servers, causing massive remapping when server count changes.

Use when: Choose when server count is fixed and rarely changes, and simplicity is preferred over flexibility.

Rendezvous (Highest Random Weight) Hashing

Assigns keys to the server with the highest hash weight per key, avoiding a ring structure.

Use when: Choose when you want minimal remapping and simpler load balancing without virtual nodes.

Summary

Consistent hashing reduces data reshuffling when servers join or leave a distributed system.

It maps keys and servers on a hash ring, assigning keys to the next server clockwise.

This approach improves scalability and fault tolerance in large distributed caches and storage.

Practice

1. What is the main advantage of consistent hashing in distributed systems?

easy

A. It stores data only on one server

B. It guarantees equal data size on all servers

C. It requires no hashing function

D. It minimizes data movement when servers are added or removed

Solution

1. Step 1: Understand consistent hashing purpose

   Consistent hashing is designed to reduce the amount of data that needs to be moved when servers join or leave the system.

2. Step 2: Compare options with consistent hashing benefits

   Only It minimizes data movement when servers are added or removed correctly states this key advantage. Other options are incorrect or unrelated.

3. Final Answer:

   It minimizes data movement when servers are added or removed -> Option D

4. Quick Check:

   Minimize data movement = D [OK]

Hint: Remember: consistent hashing reduces data reshuffling [OK]

Common Mistakes:

• Thinking consistent hashing balances data perfectly
• Assuming no hashing function is needed
• Believing data is stored on only one server

2. Which of the following is a correct step in implementing consistent hashing?

easy

A. Assign each server multiple points on the hash ring (virtual nodes)

B. Store all data on a single server to avoid hashing

C. Use a linear list instead of a circular ring

D. Hash data keys directly to server IP addresses without hashing servers

Solution

1. Step 1: Recall consistent hashing implementation details

   Consistent hashing uses a circular ring and assigns multiple virtual nodes per server to balance load.

2. Step 2: Evaluate each option

   Assign each server multiple points on the hash ring (virtual nodes) correctly describes virtual nodes. Options A, C, and D contradict consistent hashing principles.

3. Final Answer:

   Assign each server multiple points on the hash ring (virtual nodes) -> Option A

4. Quick Check:

   Virtual nodes = B [OK]

Hint: Virtual nodes improve load balance in consistent hashing [OK]

Common Mistakes:

• Ignoring virtual nodes and using single points
• Using linear lists instead of circular rings
• Skipping hashing of servers

3. Given a consistent hashing ring with servers at hash points 10, 30, and 50, where will the data key with hash 35 be stored?

medium

A. Server at 50

B. Server at 30

C. Server at 10

D. Server at 35

Solution

1. Step 1: Understand consistent hashing key assignment

   Data is assigned to the next server clockwise on the ring after the key's hash.

2. Step 2: Find the next server after hash 35

   Servers are at 10, 30, and 50. After 35 clockwise is 50.

3. Final Answer:

   Server at 50 -> Option A

4. Quick Check:

   Next clockwise server after 35 = 50 [OK]

Hint: Assign data to next clockwise server on ring [OK]

Common Mistakes:

• Assigning data to previous server instead of next
• Choosing server at 30 because it's closest numerically
• Assuming server at 35 exists

4. In a consistent hashing system, a developer notices that when a new server is added, almost all data moves to the new server. What is the likely cause?

medium

A. The data keys are not hashed before assignment

B. The new server has fewer virtual nodes than others

C. The hash function is not distributing servers evenly on the ring

D. The ring is implemented as a linear list

Solution

1. Step 1: Analyze the problem of data movement on server addition

   If almost all data moves to the new server, it means the new server's position dominates the ring.

2. Step 2: Identify cause related to server distribution

   This happens if the hash function places servers unevenly, clustering many keys near the new server.

3. Final Answer:

   The hash function is not distributing servers evenly on the ring -> Option C

4. Quick Check:

   Uneven server distribution = A [OK]

Hint: Check server hash distribution to avoid data skew [OK]

Common Mistakes:

• Blaming virtual nodes count without checking hash function
• Ignoring hashing of data keys
• Confusing ring structure with linear list

5. You are designing a distributed cache using consistent hashing. To handle server failures gracefully, which approach is best?

hard

A. Assign each data key to only one server without replication

B. Use virtual nodes and replicate data on the next N servers clockwise

C. Use a fixed number of servers without adding or removing nodes

D. Hash data keys to servers randomly each time

Solution

1. Step 1: Understand fault tolerance in consistent hashing

   To handle failures, data should be replicated to multiple servers to avoid data loss.

2. Step 2: Identify best practice for replication

   Using virtual nodes improves load balance, and replicating data on next N servers clockwise ensures availability.

3. Final Answer:

   Use virtual nodes and replicate data on the next N servers clockwise -> Option B

4. Quick Check:

   Replication + virtual nodes = A [OK]

Hint: Replicate data on next servers for fault tolerance [OK]

Common Mistakes:

• Not replicating data causing data loss on failure
• Avoiding virtual nodes and causing imbalance
• Randomly hashing keys each time causing inconsistency