Shard key selection in HLD - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

Understanding shard key impact on data distribution

Imagine you have a large database storing user profiles. You want to split this data across multiple servers (shards) to improve performance. Which shard key choice will most evenly distribute the data?

A User's country code (e.g., US, IN, UK)

B User's signup date (YYYY-MM-DD)

C User's unique ID (a random UUID)

D User's last name initial (A-Z)

💡 Hint

Think about which key has the most randomness and uniqueness to spread data evenly.

A random UUID as a shard key spreads data evenly because it has high cardinality and randomness. Country code or last name initial have limited values causing uneven distribution. Signup date can cause hotspots if many users sign up on the same day.

❓ Architecture

intermediate

Choosing shard key for query efficiency

You design a product catalog system where users often search products by category and price range. Which shard key will optimize query speed for these searches?

A Product category

B Product creation timestamp

C Product ID

D Product price

💡 Hint

Consider which shard key aligns with common query filters to reduce cross-shard queries.

Sharding by product category groups similar products together, so queries filtering by category hit fewer shards. Price or creation timestamp are continuous values causing range queries to span many shards. Product ID is unique but unrelated to query patterns.

❓ scaling

advanced

Handling shard key hotspots in a social media app

A social media app uses user ID as the shard key. Suddenly, a celebrity user causes a hotspot with millions of requests to their shard. What is the best architectural approach to reduce this hotspot?

A Implement a cache layer in front of the database for popular user data

B Change shard key to user location to spread load geographically

C Increase the number of shards without changing the shard key

D Use a composite shard key combining user ID and request timestamp

💡 Hint

Think about reducing load on the hotspot shard without major data reshuffling.

Caching popular user data reduces direct database hits on the hotspot shard. Changing shard key or increasing shards alone won't solve the hotspot because the celebrity's data is concentrated. Composite keys add complexity but don't reduce hotspot load effectively.

❓ tradeoff

advanced

Tradeoffs of choosing a monotonically increasing shard key

What is a major downside of using a monotonically increasing value (like timestamp or auto-increment ID) as a shard key?

A It prevents horizontal scaling of the database

B It increases network latency between shards

C It makes data replication impossible

D It causes uneven data distribution leading to shard hotspots

💡 Hint

Consider how new data is assigned to shards over time.

Monotonically increasing shard keys cause new data to be written to the same shard, creating hotspots and uneven load. This reduces write scalability. It does not inherently increase latency, prevent replication, or block scaling.

❓ estimation

expert

Estimating shard count for expected load

You expect 1 billion user records growing by 10 million per month. Each shard can handle 100 million records efficiently. How many shards do you need initially and after 6 months?

A Initially 5 shards; after 6 months 11 shards

B Initially 10 shards; after 6 months 16 shards

C Initially 15 shards; after 6 months 20 shards

D Initially 8 shards; after 6 months 14 shards

💡 Hint

Divide total records by shard capacity to estimate shard count.

Initially: 1,000,000,000 / 100,000,000 = 10 shards.
After 6 months: 1,000,000,000 + (6 * 10,000,000) = 1,060,000,000 records.
1,060,000,000 / 100,000,000 ≈ 10.6 → 11 shards minimally. Option B (10 initial, 16 after) correctly matches the initial count and provides overprovisioning for safety, ongoing growth, and inefficiencies.

Practice

1. What is the main purpose of selecting a shard key in a distributed database system?

easy

A. To compress data to save storage space

B. To encrypt data for security purposes

C. To backup data automatically

D. To split data across servers for better load distribution and faster access

Solution

1. Step 1: Understand shard key role

   The shard key determines how data is divided across multiple servers or shards.

2. Step 2: Identify main benefit

   Splitting data helps balance load and speeds up data retrieval by parallelizing queries.

3. Final Answer:

   To split data across servers for better load distribution and faster access -> Option D

4. Quick Check:

   Shard key purpose = split data load [OK]

Hint: Shard key splits data to spread load and speed up access [OK]

Common Mistakes:

• Confusing shard key with encryption key
• Thinking shard key compresses data
• Assuming shard key is for backup

2. Which of the following is a good characteristic of a shard key?

easy

A. Has very few unique values

B. Has many unique values and distributes data evenly

C. Is always a timestamp

D. Is a large text field

Solution

1. Step 1: Identify shard key traits

   A good shard key should have many unique values to avoid data concentration.

2. Step 2: Understand distribution importance

   Even distribution prevents hotspots and balances load across shards.

3. Final Answer:

   Has many unique values and distributes data evenly -> Option B

4. Quick Check:

   Good shard key = unique + even distribution [OK]

Hint: Choose shard keys with many unique values and even spread [OK]

Common Mistakes:

• Picking keys with few unique values causing hotspots
• Assuming timestamp is always best shard key
• Using large text fields that slow down indexing

3. Consider a user database sharded by country_code which has only 5 unique values. What is the likely problem with this shard key?

medium

A. Data will be evenly distributed across shards

B. It will encrypt user data automatically

C. Some shards will be overloaded due to uneven data distribution

D. Queries will be faster because of fewer shards

Solution

1. Step 1: Analyze unique values count

   Only 5 unique values means few shards and many users per shard.

2. Step 2: Understand load distribution

   Popular countries will have more users, causing some shards to be overloaded.

3. Final Answer:

   Some shards will be overloaded due to uneven data distribution -> Option C

4. Quick Check:

   Few unique keys = uneven shard load [OK]

Hint: Few unique shard keys cause uneven load and hotspots [OK]

Common Mistakes:

• Assuming fewer shards always improve speed
• Thinking shard key encrypts data
• Ignoring uneven data distribution effects

4. A system uses user_id % 10 as a shard key but notices uneven shard sizes. What is the likely cause?

medium

A. User IDs are not uniformly distributed modulo 10

B. Shard key is too unique

C. Shard key is a string instead of number

D. System is encrypting shard keys

Solution

1. Step 1: Understand modulo shard key

   Modulo operation expects uniform distribution of user IDs.

2. Step 2: Identify distribution problem

   If user IDs cluster in certain ranges, modulo results cluster causing uneven shards.

3. Final Answer:

   User IDs are not uniformly distributed modulo 10 -> Option A

4. Quick Check:

   Non-uniform IDs cause uneven modulo shard keys [OK]

Hint: Modulo shard keys need uniform input distribution [OK]

Common Mistakes:

• Assuming modulo always balances shards
• Confusing shard key type with encryption
• Ignoring data distribution patterns

5. You design a sharded system for an e-commerce app. Users query orders mostly by user_id and order_date. Which shard key choice best supports fast queries and balanced load?

hard

A. Shard by a composite key of user_id and order_date

B. Shard by user_id only

C. Shard by order_date only

D. Shard by a random UUID

Solution

1. Step 1: Match shard key to query patterns

   Users query by both user_id and order_date, so shard key should support both.

2. Step 2: Evaluate shard key options

   Sharding by user_id alone may cause hotspots; order_date alone may cause uneven load by time.

3. Step 3: Use composite key benefits

   Composite key balances load and supports queries filtering by either field efficiently.

4. Final Answer:

   Shard by a composite key of user_id and order_date -> Option A

5. Quick Check:

   Composite shard key matches queries and balances load [OK]

Hint: Composite keys balance load and match query patterns best [OK]

Common Mistakes:

• Choosing single field shard keys ignoring query patterns
• Using random UUID causing inefficient queries
• Ignoring load balancing in shard key choice