Why Write sharding in DynamoDB? - Purpose & Use Cases

The Big Idea

What if your app could never slow down no matter how many users write data at once?

The Scenario

Imagine you run a popular online store. Many customers try to add items to their carts at the same time. You try to record all these actions in one single database entry.

But the database slows down and sometimes even crashes because too many writes happen at once on the same spot.

The Problem

Writing all data to one place causes a traffic jam. The database can only handle so many writes per second on one item.

This leads to delays, errors, and unhappy customers who can't add their items quickly.

The Solution

Write sharding splits the data into many smaller parts. Each part handles a portion of the writes.

This spreads out the load so no single spot gets overwhelmed. The database stays fast and reliable even with many users.

Before vs After

✗ Before

UpdateItem CartID=123 Quantity=1

✓ After

UpdateItem CartID=123-Shard1 Quantity=1
UpdateItem CartID=123-Shard2 Quantity=1

What It Enables

Write sharding lets your app handle huge numbers of simultaneous writes smoothly and without errors.

Real Life Example

Big social media apps use write sharding to record likes and comments from millions of users at the same time without slowing down.

Key Takeaways

Writing all data to one place causes slowdowns and errors.

Write sharding splits data to spread out the load.

This keeps databases fast and reliable under heavy use.

Practice

1. What is the main purpose of write sharding in DynamoDB?

easy

A. To backup data automatically to S3

B. To spread write load across multiple partitions by adding shard IDs

C. To encrypt data at rest for security

D. To reduce read latency by caching data locally

Solution

1. Step 1: Understand write sharding concept

   Write sharding adds shard IDs to partition keys to distribute writes evenly.

2. Step 2: Identify the main goal

   This distribution avoids hot partitions and improves write performance.

3. Final Answer:

   To spread write load across multiple partitions by adding shard IDs -> Option B

4. Quick Check:

   Write sharding = spreading writes [OK]

Hint: Write sharding spreads writes by adding shard IDs [OK]

Common Mistakes:

• Confusing write sharding with read caching
• Thinking it encrypts data
• Assuming it handles backups

2. Which of the following is the correct way to implement write sharding in a DynamoDB partition key?

easy

A. Use a fixed partition key without any shard ID

B. Use the sort key as the shard ID

C. Append a random shard ID to the partition key before writing

D. Write all data to a single partition key

Solution

1. Step 1: Recall write sharding method

   Write sharding requires adding a shard ID to the partition key to distribute writes.

2. Step 2: Identify correct implementation

   Appending a random shard ID to the partition key spreads writes across partitions.

3. Final Answer:

   Append a random shard ID to the partition key before writing -> Option C

4. Quick Check:

   Shard ID + partition key = write sharding [OK]

Hint: Add random shard ID to partition key for sharding [OK]

Common Mistakes:

• Not adding shard ID at all
• Using sort key as shard ID incorrectly
• Writing all data to one partition

3. Given the following code snippet for write sharding, what is the expected effect?

shard_id = random.randint(0, 4)
partition_key = f"user#{user_id}#{shard_id}"
dynamodb_table.put_item(Item={"PK": partition_key, "Data": data})

medium

A. Writes are distributed across 5 partitions reducing hot spots

B. All writes go to the same partition causing throttling

C. Writes fail due to invalid partition key format

D. Data is only readable by shard_id 0

Solution

1. Step 1: Analyze shard ID usage

   The shard_id is a random number from 0 to 4, appended to the partition key.

2. Step 2: Understand partition key effect

   This creates 5 different partition keys for the same user, spreading writes across 5 partitions.

3. Final Answer:

   Writes are distributed across 5 partitions reducing hot spots -> Option A

4. Quick Check:

   Random shard ID 0-4 = 5 partitions [OK]

Hint: Random shard ID 0-4 means 5 partitions [OK]

Common Mistakes:

• Assuming all writes go to one partition
• Thinking partition key format is invalid
• Believing data is restricted by shard ID

4. You implemented write sharding by appending shard IDs to partition keys, but you notice some writes still cause throttling. What is the likely cause?

medium

A. Shard ID range is too small causing uneven distribution

B. You forgot to add shard ID to the sort key

C. You used a random shard ID but did not update the partition key

D. DynamoDB does not support write sharding

Solution

1. Step 1: Check shard ID range

   If the shard ID range is too small, writes cluster on few partitions causing throttling.

2. Step 2: Verify correct key usage

   Shard ID must be part of the partition key, not sort key, and must cover enough range.

3. Final Answer:

   Shard ID range is too small causing uneven distribution -> Option A

4. Quick Check:

   Small shard range = hot partitions [OK]

Hint: Use enough shard IDs to spread writes evenly [OK]

Common Mistakes:

• Adding shard ID to sort key instead of partition key
• Not updating partition key with shard ID
• Believing DynamoDB disallows sharding

5. You want to design a write sharding strategy for a DynamoDB table with very high write traffic on user data. Which approach best balances write load and allows efficient reads?

hard

A. Write all data to a single partition key and batch writes

B. Use user ID as partition key without sharding; rely on DynamoDB auto-scaling

C. Add shard ID to sort key and keep partition key as user ID

D. Add a random shard ID (0-9) to the partition key and store user data in each shard; aggregate reads by querying all shards

Solution

1. Step 1: Evaluate write sharding options

   Adding a random shard ID (0-9) to partition key spreads writes across 10 partitions.

2. Step 2: Consider read strategy

   Reads require querying all shards and aggregating results, which is manageable for balanced load.

3. Step 3: Compare other options

   Using no sharding risks throttling; shard ID in sort key does not distribute writes; single partition causes hot spots.

4. Final Answer:

   Add a random shard ID (0-9) to the partition key and store user data in each shard; aggregate reads by querying all shards -> Option D

5. Quick Check:

   Shard ID in partition key + aggregate reads = balanced writes [OK]

Hint: Shard in partition key, aggregate reads from all shards [OK]

Common Mistakes:

• Not sharding partition key causing hot partitions
• Adding shard ID to sort key only
• Ignoring read complexity after sharding