DynamoDB Streams concept - Step-by-Step Execution

Concept Flow - DynamoDB Streams concept

Data change in DynamoDB table

↓

DynamoDB Streams captures change

↓

Stream records created

↓

Stream consumers read records

↓

Process or react to changes

↓

Done

When data changes in a DynamoDB table, DynamoDB Streams captures these changes as records. Consumers then read these records to process or react to the changes.

Execution Sample

DynamoDB

1. Insert item into DynamoDB table
2. DynamoDB Streams captures INSERT event
3. Stream record created with new item data
4. Consumer reads stream record
5. Consumer processes the new item

This example shows how inserting an item triggers a stream record that a consumer reads and processes.

Execution Table

Step	Action	Stream Record Created?	Record Content	Consumer Action
1	Insert item into table	Yes	INSERT event with new item data	No action yet
2	Update item in table	Yes	MODIFY event with old and new data	No action yet
3	Delete item from table	Yes	REMOVE event with old item data	No action yet
4	Consumer reads first stream record	No	N/A	Processes INSERT event data
5	Consumer reads second stream record	No	N/A	Processes MODIFY event data
6	Consumer reads third stream record	No	N/A	Processes REMOVE event data
7	No more records	No	N/A	Consumer waits for new records

💡 All stream records processed; consumer waits for new changes.

Variable Tracker

Variable	Start	After Step 1	After Step 2	After Step 3	After Step 4	After Step 5	After Step 6	Final
Stream Records Queue	Empty	INSERT record added	MODIFY record added	REMOVE record added	INSERT record removed	MODIFY record removed	REMOVE record removed	Empty

Key Moments - 3 Insights

Why does a stream record appear after each data change?

Because DynamoDB Streams captures every data modification (insert, update, delete) as a record, as shown in execution_table rows 1-3.

Does the consumer create stream records?

No, the consumer only reads and processes existing stream records created by DynamoDB Streams, as seen in execution_table rows 4-6.

What happens when there are no new changes in the table?

The consumer waits for new stream records since none are created without data changes, shown in execution_table row 7.

Visual Quiz - 3 Questions

Look at the execution_table, what type of event is recorded at Step 2?

A INSERT event

B MODIFY event

C REMOVE event

D No event

💡 Hint

Check the 'Stream Record Created?' and 'Record Content' columns at Step 2.

At which step does the consumer process the REMOVE event?

A Step 6

B Step 4

C Step 3

D Step 7

💡 Hint

Look at the 'Consumer Action' column for processing REMOVE event.

If no data changes happen, what will the stream records queue state be?

A Contains MODIFY records

B Contains INSERT records

C Empty

D Contains REMOVE records

💡 Hint

Refer to variable_tracker final state and execution_table Step 7.

Concept Snapshot

DynamoDB Streams capture table data changes as records.
Each INSERT, MODIFY, or REMOVE creates a stream record.
Consumers read these records to react to changes.
Stream records queue holds changes until processed.
No data change means no new stream records.

Full Transcript

DynamoDB Streams work by capturing every change made to a DynamoDB table. When you insert, update, or delete an item, a stream record is created that describes this change. These records are stored in a queue. Consumers then read these records one by one to process or react to the changes. If no changes happen, no new stream records are created, and consumers wait for new data. This flow ensures you can track and respond to all data changes in your table.

Practice

1. What is the primary purpose of DynamoDB Streams?

easy

A. To back up the entire DynamoDB table automatically every hour

B. To capture and record changes made to items in a DynamoDB table in real time

C. To increase the read capacity of a DynamoDB table

D. To encrypt data stored in DynamoDB tables

Solution

1. Step 1: Understand DynamoDB Streams functionality

   DynamoDB Streams capture data changes such as inserts, updates, and deletes on a table.

2. Step 2: Compare options with Streams purpose

   Only To capture and record changes made to items in a DynamoDB table in real time describes capturing changes in real time, which is the core function of Streams.

3. Final Answer:

   To capture and record changes made to items in a DynamoDB table in real time -> Option B

4. Quick Check:

   DynamoDB Streams = capture changes [OK]

Hint: Streams track table changes live, not backups or capacity [OK]

Common Mistakes:

• Confusing Streams with backups
• Thinking Streams increase capacity
• Assuming Streams encrypt data

2. Which of the following is the correct way to enable DynamoDB Streams on a table?

easy

A. Enable Auto Scaling on the table

B. Add a Global Secondary Index with StreamEnabled = true

C. Create a Lambda function and attach it to the table without enabling Streams

D. Set the StreamSpecification with StreamEnabled = true and choose a StreamViewType

Solution

1. Step 1: Recall how to enable Streams

   Streams are enabled by setting StreamSpecification with StreamEnabled true and selecting a StreamViewType.

2. Step 2: Evaluate options

   Set the StreamSpecification with StreamEnabled = true and choose a StreamViewType correctly describes enabling Streams. Other options relate to indexes, Lambda, or scaling, which do not enable Streams.

3. Final Answer:

   Set the StreamSpecification with StreamEnabled = true and choose a StreamViewType -> Option D

4. Quick Check:

   Enable Streams = StreamSpecification true + StreamViewType [OK]

Hint: Enable Streams by setting StreamSpecification true with view type [OK]

Common Mistakes:

• Confusing Streams with indexes
• Assuming Lambda enables Streams automatically
• Thinking Auto Scaling enables Streams

3. Given a DynamoDB Stream with StreamViewType set to NEW_AND_OLD_IMAGES, what data will each stream record contain?

medium

A. Only the new image of the item after the change

B. Only the old image of the item before the change

C. Both the new and old images of the item

D. Only the keys of the changed item

Solution

1. Step 1: Understand StreamViewType NEW_AND_OLD_IMAGES

   This view type captures both the item image before and after the change.

2. Step 2: Match options to the view type

   Both the new and old images of the item correctly states both new and old images are included. Other options only mention one or keys.

3. Final Answer:

   Both the new and old images of the item -> Option C

4. Quick Check:

   NEW_AND_OLD_IMAGES = new + old images [OK]

Hint: NEW_AND_OLD_IMAGES means both before and after data included [OK]

Common Mistakes:

• Thinking only new or only old image is included
• Confusing keys-only view with full images
• Ignoring the StreamViewType setting

4. You have enabled DynamoDB Streams but your Lambda function is not receiving any events despite recent writes to the table. What is a possible reason?

medium

A. The Lambda function is not properly configured as an event source for the stream

B. The table has no write activity, so no stream records are generated

C. All of the above

D. The StreamViewType was set to KEYS_ONLY, but your Lambda expects full images

Solution

1. Step 1: Analyze the scenario

   Streams enabled with recent writes means records are generated, so issue is likely with Lambda setup to consume stream events.

2. Step 2: Evaluate options

   The Lambda not being configured as an event source for the stream explains no events. All of the above doesn't apply since writes occurred. No write activity contradicted by scenario. StreamViewType affects record content, not event delivery to Lambda.

3. Final Answer:

   The Lambda function is not properly configured as an event source for the stream -> Option A

4. Quick Check:

   No events despite writes = Lambda event source not mapped [OK]

Hint: Lambda must be mapped as event source to DynamoDB Stream [OK]

Common Mistakes:

• Ignoring that no writes means no stream records
• Forgetting to attach Lambda as event source
• Assuming KEYS_ONLY always sends full data

5. You want to build a system that reacts only when an item's attribute status changes in your DynamoDB table. Which StreamViewType should you choose to efficiently detect this change?

hard

A. NEW_AND_OLD_IMAGES

B. OLD_IMAGE

C. NEW_IMAGE

D. KEYS_ONLY

Solution

1. Step 1: Identify data needed to detect attribute changes

   To know if 'status' changed, you need both old and new values to compare.

2. Step 2: Match StreamViewType to requirement

   Only NEW_AND_OLD_IMAGES provides both before and after images for comparison.

3. Final Answer:

   NEW_AND_OLD_IMAGES -> Option A

4. Quick Check:

   Detect attribute change = need old + new images [OK]

Hint: Compare old and new images to detect attribute changes [OK]

Common Mistakes:

• Choosing only NEW_IMAGE or OLD_IMAGE which lacks comparison data
• Assuming KEYS_ONLY includes attribute data
• Not realizing both images are needed to detect changes