Unprocessed items handling in DynamoDB - Step-by-Step Execution

Concept Flow - Unprocessed items handling

Send Batch Write Request

↓

Receive Response

↓

Check for Unprocessed Items?

No→Done

Yes↓

Retry Unprocessed Items

↓

Send Batch Write Request

This flow shows how DynamoDB batch write requests may return unprocessed items, which need to be retried until all are processed.

Execution Sample

DynamoDB

batchWrite({
  RequestItems: {
    'Table': [
      { PutRequest: { Item: { /* item attributes */ } } },
      { DeleteRequest: { Key: { /* key attributes */ } } }
    ]
  }
})

Send a batch write request to DynamoDB and handle any unprocessed items by retrying.

Execution Table

Step	Action	Request Items Sent	Response Unprocessed Items	Next Action
1	Send batch write request	5 items	2 items unprocessed	Retry unprocessed items
2	Retry batch write request	2 items	1 item unprocessed	Retry unprocessed items
3	Retry batch write request	1 item	0 items unprocessed	Done - all processed

💡 All items processed when unprocessed items count is zero

Variable Tracker

Variable	Start	After 1	After 2	Final
UnprocessedItemsCount	5	2	1	0

Key Moments - 2 Insights

Why do some items remain unprocessed after the first batch write?

Because DynamoDB may throttle requests or have temporary capacity limits, causing some items to be returned as unprocessed (see execution_table step 1).

Do we stop retrying after the first retry?

No, we keep retrying unprocessed items until none remain, as shown in execution_table steps 2 and 3.

Visual Quiz - 3 Questions

Look at the execution table, how many items were unprocessed after the first batch write?

A 1 item

B 5 items

C 2 items

D 0 items

💡 Hint

Check the 'Response Unprocessed Items' column in step 1 of the execution_table.

At which step does the unprocessed items count become zero?

A Step 3

B Step 2

C Step 1

D Never

💡 Hint

Look at the 'UnprocessedItemsCount' variable in variable_tracker after each step.

If the first batch write had no unprocessed items, what would happen next?

A Retry unprocessed items

B Finish processing

C Send another batch write with same items

D Throw an error

💡 Hint

Refer to the concept_flow where 'No' from 'Check for Unprocessed Items?' leads to 'Done'.

Concept Snapshot

DynamoDB batch writes may return unprocessed items due to throttling.
Always check response for unprocessed items.
Retry only those unprocessed items until all succeed.
This ensures data consistency and reliability.

Full Transcript

When you send a batch write request to DynamoDB, it may not process all items immediately. Some items can be returned as unprocessed due to temporary limits. You must check the response for these unprocessed items. If there are any, retry sending only those items. Repeat this process until no unprocessed items remain. This way, you ensure all your data is written successfully.

Practice

1. What does DynamoDB return when some items in a batch write are not processed immediately?

easy

A. An error stopping the entire batch operation

B. A partial success with no retry information

C. A success message without details

D. A list of unprocessed items to retry later

Solution

1. Step 1: Understand batch write behavior

   DynamoDB may not process all items in a batch due to throttling or limits.

2. Step 2: Identify returned data for unprocessed items

   It returns unprocessed items so the client can retry sending them.

3. Final Answer:

   A list of unprocessed items to retry later -> Option D

4. Quick Check:

   Unprocessed items = retry list [OK]

Hint: Unprocessed items mean retry those items later [OK]

Common Mistakes:

• Thinking DynamoDB throws an error instead
• Assuming all items are processed without feedback
• Ignoring unprocessed items in code

2. Which of the following is the correct way to check for unprocessed items in a DynamoDB BatchWriteItem response in Python?

easy

A. if 'UnprocessedItems' in response and response['UnprocessedItems']:

B. if response['UnprocessedItems'] == None:

C. if response['UnprocessedItems'] is False:

D. if 'UnprocessedItems' not in response:

Solution

1. Step 1: Understand response structure

   The response includes 'UnprocessedItems' key if any items were not processed.

2. Step 2: Correctly check for unprocessed items

   Check if 'UnprocessedItems' exists and is not empty to retry those items.

3. Final Answer:

   if 'UnprocessedItems' in response and response['UnprocessedItems']: -> Option A

4. Quick Check:

   Check key exists and is not empty [OK]

Hint: Check key exists and is not empty to find unprocessed items [OK]

Common Mistakes:

• Checking for None instead of empty dict
• Using 'not in' which misses unprocessed items
• Checking for False which is incorrect

3. Given this Python snippet handling DynamoDB batch write:

response = client.batch_write_item(RequestItems=batch)
if response.get('UnprocessedItems'):
    retry_items = response['UnprocessedItems']
else:
    retry_items = None
print(retry_items)

What will be printed if all items were processed successfully?

medium

A. List of all items

B. {}

C. None

D. An error

Solution

1. Step 1: Understand response when all items processed

   If all items are processed, 'UnprocessedItems' key is empty or missing.

2. Step 2: Analyze code behavior

   response.get('UnprocessedItems') returns None or empty dict; if empty dict is falsy, else branch sets retry_items to None.

3. Final Answer:

   None -> Option C

4. Quick Check:

   No unprocessed items means retry_items = None [OK]

Hint: If no unprocessed items, retry_items is None [OK]

Common Mistakes:

• Expecting empty dict instead of None
• Assuming an error is raised
• Thinking retry_items contains all items

4. You wrote this code to resend unprocessed items but it never stops retrying:

while True:
    response = client.batch_write_item(RequestItems=items)
    items = response.get('UnprocessedItems', {})
    if not items:
        break

What is the likely bug causing infinite retries?

medium

A. Missing a delay between retries causing throttling

B. Using 'break' inside the loop incorrectly

C. Not updating 'items' with the new unprocessed items correctly

D. Not checking for empty dictionary properly

Solution

1. Step 1: Analyze the retry loop

   The code updates items correctly but retries immediately without delay.

2. Step 2: Identify the cause of infinite retries

   Missing delay or exponential backoff leads to rapid retries, causing persistent throttling. DynamoDB continually returns the same unprocessed items.

3. Final Answer:

   Missing a delay between retries causing throttling -> Option A

4. Quick Check:

   Add sleep/backoff between retries [OK]

Hint: Add delay or backoff between batch retries to avoid throttling [OK]

Common Mistakes:

• Blaming item update logic instead of throttling
• Assuming empty dict check prevents infinite loops
• Overlooking need for backoff in tight retry loops

5. You want to ensure all items in a large batch are saved in DynamoDB without loss. Which approach best handles unprocessed items efficiently?

hard

A. Split items into single writes to avoid unprocessed items

B. Check for unprocessed items after each batch write and retry them until none remain

C. Send all items once and ignore unprocessed items since they are rare

D. Use batch write once and log unprocessed items without retrying

Solution

1. Step 1: Understand batch write limitations

   DynamoDB batch writes can return unprocessed items due to throttling or limits.

2. Step 2: Best practice for data safety

   Retry unprocessed items repeatedly until all are saved to avoid data loss.

3. Final Answer:

   Check for unprocessed items after each batch write and retry them until none remain -> Option B

4. Quick Check:

   Retry unprocessed items until empty for safe writes [OK]

Hint: Always retry unprocessed items until all are saved [OK]

Common Mistakes:

• Ignoring unprocessed items causing data loss
• Writing items one by one causing inefficiency
• Logging without retrying unprocessed items