Batch limits and retries in DynamoDB - Mini Project: Build & Apply

Handling Batch Limits and Retries in DynamoDB

📖 Scenario: You are managing a DynamoDB table that stores customer orders for an online store. You need to insert multiple orders at once using batch operations. However, DynamoDB limits batch writes to 25 items per request. Also, sometimes requests may not process all items, so you need to retry unprocessed items until all are saved.

🎯 Goal: Build a DynamoDB batch write operation that respects the 25-item limit per batch and retries unprocessed items until all orders are successfully written.

📋 What You'll Learn

Create a list called orders with exactly 30 order items, each with OrderId and CustomerName attributes.

Define a batch size variable called BATCH_SIZE set to 25.

Write code to split orders into batches of size BATCH_SIZE.

Implement a loop to send each batch to DynamoDB using batch_write_item and retry unprocessed items until none remain.

💡 Why This Matters

🌍 Real World

Batch writing is common when inserting or updating many items in DynamoDB efficiently while respecting service limits.

💼 Career

Understanding batch limits and retries is essential for backend developers and cloud engineers working with AWS DynamoDB to build scalable and reliable applications.

1

Create the orders list with 30 items

Create a list called orders containing 30 dictionaries. Each dictionary must have keys 'OrderId' with values from 1 to 30 as integers, and 'CustomerName' with values 'Customer1' to 'Customer30' as strings.

DynamoDB

# Create the list 'orders' with 30 order dictionaries
# Your code here

Need a hint?

Use a list comprehension with range(1, 31) to create 30 dictionaries.

2

Define the batch size variable

Define an integer variable called BATCH_SIZE and set it to 25.

DynamoDB

orders = [{'OrderId': i, 'CustomerName': f'Customer{i}'} for i in range(1, 31)]
# Define BATCH_SIZE as 25
# Your code here

Need a hint?

Just assign the number 25 to the variable BATCH_SIZE.

3

Split orders into batches

Create a list called batches that splits orders into sublists each of size BATCH_SIZE. Use a list comprehension with slicing and range.

DynamoDB

orders = [{'OrderId': i, 'CustomerName': f'Customer{i}'} for i in range(1, 31)]
BATCH_SIZE = 25
# Create batches by slicing orders into chunks of BATCH_SIZE
# Your code here

Need a hint?

Use a list comprehension with range(0, len(orders), BATCH_SIZE) and slice orders[i:i + BATCH_SIZE].

4

Write batch write with retries for unprocessed items

Write code that loops over each batch in batches. For each batch, create a RequestItems dictionary with key 'Orders' and value as a list of {'PutRequest': {'Item': item}} for each item in the batch. Then, use a while loop to call dynamodb.batch_write_item(RequestItems=RequestItems) and retry unprocessed items until none remain. Assume dynamodb is a valid DynamoDB client.

DynamoDB

orders = [{'OrderId': i, 'CustomerName': f'Customer{i}'} for i in range(1, 31)]
BATCH_SIZE = 25
batches = [orders[i:i + BATCH_SIZE] for i in range(0, len(orders), BATCH_SIZE)]
# Loop over batches and write to DynamoDB with retries
# Your code here

Need a hint?

Use a for loop over batches, build RequestItems, then a while loop to call batch_write_item and update RequestItems['Orders'] with unprocessed items.