Hot partition prevention in DynamoDB - Mini Project: Build & Apply

Hot Partition Prevention in DynamoDB

📖 Scenario: You are managing a DynamoDB table that stores user activity logs. The table uses UserID as the partition key. You notice that some users generate a lot of activity, causing hot partitions that slow down the database.To prevent this, you want to distribute the load more evenly by adding a suffix to the UserID to create multiple partitions per user.

🎯 Goal: Build a DynamoDB table design that prevents hot partitions by adding a partition key suffix. You will create the initial table schema, define a suffix count, implement logic to assign suffixes to UserID, and finalize the table with the new partition key.

📋 What You'll Learn

Create a DynamoDB table schema with UserID as the partition key and ActivityTimestamp as the sort key.

Define a variable suffix_count to control how many suffixes to use for partition keys.

Implement logic to assign a suffix to each UserID based on a hash modulo suffix_count.

Update the partition key to include the suffix, preventing hot partitions.

💡 Why This Matters

🌍 Real World

Hot partitions in DynamoDB can cause throttling and slow performance. This project shows how to design partition keys to distribute load evenly, which is critical for high-traffic applications like user activity tracking.

💼 Career

Database engineers and backend developers often need to optimize NoSQL database designs to handle uneven data access patterns. Understanding hot partition prevention is essential for building scalable, reliable systems.

1

Create DynamoDB table schema

Create a DynamoDB table schema named ActivityLog with UserID as the partition key and ActivityTimestamp as the sort key.

DynamoDB

# Define the DynamoDB table schema with UserID as partition key and ActivityTimestamp as sort key
# Your code here

Hint

Use KeySchema to define UserID as the partition key (HASH) and ActivityTimestamp as the sort key (RANGE).

2

Define suffix count for partition keys

Define an integer variable called suffix_count and set it to 5. This will control how many suffixes to use for partition keys.

DynamoDB

ActivityLog = {
    'TableName': 'ActivityLog',
    'KeySchema': [
        {'AttributeName': 'UserID', 'KeyType': 'HASH'},
        {'AttributeName': 'ActivityTimestamp', 'KeyType': 'RANGE'}
    ],
    'AttributeDefinitions': [
        {'AttributeName': 'UserID', 'AttributeType': 'S'},
        {'AttributeName': 'ActivityTimestamp', 'AttributeType': 'S'}
    ],
    'ProvisionedThroughput': {
        'ReadCapacityUnits': 5,
        'WriteCapacityUnits': 5
    }
}

# Define suffix_count variable
# Your code here

Hint

Set suffix_count to 5 to create 5 suffixes for partition keys.

3

Implement suffix assignment logic

Write a function called assign_partition_key that takes a user_id string and returns a new partition key by appending a suffix. The suffix is the hash of user_id modulo suffix_count. Use Python's built-in hash() function.

DynamoDB

ActivityLog = {
    'TableName': 'ActivityLog',
    'KeySchema': [
        {'AttributeName': 'UserID', 'KeyType': 'HASH'},
        {'AttributeName': 'ActivityTimestamp', 'KeyType': 'RANGE'}
    ],
    'AttributeDefinitions': [
        {'AttributeName': 'UserID', 'AttributeType': 'S'},
        {'AttributeName': 'ActivityTimestamp', 'AttributeType': 'S'}
    ],
    'ProvisionedThroughput': {
        'ReadCapacityUnits': 5,
        'WriteCapacityUnits': 5
    }
}

suffix_count = 5

def assign_partition_key(user_id):
    # Your code here

Hint

Use hash(user_id) % suffix_count to get the suffix number, then append it to user_id with a # separator.

4

Update table schema with new partition key

Update the ActivityLog table schema to use PartitionKey as the new partition key instead of UserID. The PartitionKey will store the value returned by assign_partition_key. Keep ActivityTimestamp as the sort key.

DynamoDB

ActivityLog = {
    # Your code here
}

suffix_count = 5

def assign_partition_key(user_id):
    suffix = hash(user_id) % suffix_count
    return f"{user_id}#{suffix}"

Hint

Change the partition key attribute name from UserID to PartitionKey in both KeySchema and AttributeDefinitions.

Practice

1. What is the main goal of hot partition prevention in DynamoDB?

easy

A. To spread requests evenly across partitions to avoid slowdowns

B. To increase the size of each partition for more storage

C. To reduce the number of partitions to save costs

D. To backup data more frequently for safety

Solution

1. Step 1: Understand what causes hot partitions

   Hot partitions happen when too many requests target the same partition key, causing slowdowns.

2. Step 2: Identify the goal of prevention

   Prevention aims to spread requests evenly to avoid overloading any single partition.

3. Final Answer:

   To spread requests evenly across partitions to avoid slowdowns -> Option A

4. Quick Check:

   Hot partition prevention = spreading requests evenly [OK]

Hint: Think: avoid overload by spreading requests evenly [OK]

Common Mistakes:

• Confusing prevention with increasing storage size
• Thinking it reduces partitions instead of balancing load
• Assuming it relates to backup frequency

2. Which of the following is a correct way to design a partition key to prevent hot partitions?

easy

A. Use the same timestamp as the partition key for all writes

B. Use a static string as the partition key for all items

C. Add a random suffix or hash to the partition key

D. Use a sequential number as the partition key without variation

Solution

1. Step 1: Review partition key design principles

   To prevent hot partitions, keys should distribute traffic evenly, not be identical or sequential.

2. Step 2: Identify the correct method

   Adding a random suffix or hashing the key spreads requests across partitions.

3. Final Answer:

   Add a random suffix or hash to the partition key -> Option C

4. Quick Check:

   Partition key design = add randomness or hash [OK]

Hint: Add randomness or hash to partition key to balance load [OK]

Common Mistakes:

• Using static or identical keys causing hot partitions
• Using sequential keys that cluster traffic
• Ignoring the need for key variation

3. Given this partition key design: userId#randomSuffix, what is the expected effect on hot partitions?

medium

A. It will prevent hot partitions by spreading requests across suffixes

B. It will cause errors because suffixes are not allowed

C. It will cause hot partitions because all keys start with userId

D. It will reduce storage capacity of the table

Solution

1. Step 1: Analyze the key structure

   The key combines a userId with a random suffix, creating many unique partition keys.

2. Step 2: Understand impact on partitions

   Random suffixes spread requests across partitions, preventing any single hot partition.

3. Final Answer:

   It will prevent hot partitions by spreading requests across suffixes -> Option A

4. Quick Check:

   Random suffixes spread load = prevent hot partitions [OK]

Hint: Random suffixes spread requests, avoiding hot partitions [OK]

Common Mistakes:

• Assuming prefix causes hot partitions ignoring suffix effect
• Thinking suffixes cause errors in keys
• Confusing partition key design with storage limits

4. You notice a hot partition in your DynamoDB table caused by a partition key with a timestamp. What is the best fix?

medium

A. Remove the timestamp and use a static key

B. Delete old data to reduce partition size

C. Increase the read capacity units without changing keys

D. Add a hash or random suffix to the timestamp key

Solution

1. Step 1: Identify the problem with timestamp keys

   Using timestamps as keys causes many requests to target the same partition if timestamps are similar.

2. Step 2: Choose the best fix

   Adding a hash or random suffix spreads requests across partitions, preventing hot spots.

3. Final Answer:

   Add a hash or random suffix to the timestamp key -> Option D

4. Quick Check:

   Fix hot partition = add randomness to key [OK]

Hint: Add randomness to timestamp keys to fix hot partitions [OK]

Common Mistakes:

• Using static keys increases hot partition risk
• Only increasing capacity doesn't fix key design
• Deleting data doesn't solve partition key issues

5. You have a DynamoDB table with a partition key orderId that is sequential and causes hot partitions. How can you redesign the key to prevent this while keeping orderId in the key?

hard

A. Keep orderId as partition key and add a timestamp suffix

B. Use orderId as the sort key and add a hashed prefix as the partition key

C. Use only orderId as partition key without changes

D. Remove orderId and use a static string as partition key

Solution

1. Step 1: Understand the problem with sequential keys

   Sequential orderId as partition key causes hot partitions because requests cluster on recent keys.

2. Step 2: Redesign key to spread load

   Using a hashed prefix as partition key spreads requests; orderId can be used as sort key to keep uniqueness.

3. Final Answer:

   Use orderId as the sort key and add a hashed prefix as the partition key -> Option B

4. Quick Check:

   Hash prefix + orderId sort key = balanced partitions [OK]

Hint: Hash prefix partition key, keep orderId as sort key [OK]

Common Mistakes:

• Adding timestamp suffix doesn't fix partition key clustering
• Keeping orderId as partition key causes hot partitions
• Using static partition keys causes all traffic to one partition