Stream processing patterns in DynamoDB - Mini Project: Build & Apply

Stream Processing Patterns with DynamoDB

📖 Scenario: You are working on a simple inventory management system that tracks product stock levels. Whenever a product's stock changes, the update is recorded in a DynamoDB table. You want to process these changes in real-time using DynamoDB Streams to keep another table updated with the latest stock status.

🎯 Goal: Build a DynamoDB stream processing setup that captures changes from a Products table and updates a StockStatus table accordingly.

📋 What You'll Learn

Create a DynamoDB table called Products with a primary key ProductID and an attribute Stock.

Enable DynamoDB Streams on the Products table to capture item updates.

Create a configuration variable stream_arn to hold the stream ARN of the Products table.

Write a Lambda function that processes stream records and updates the StockStatus table with the latest stock for each product.

Ensure the Lambda function uses the stream_arn to connect to the stream.

💡 Why This Matters

🌍 Real World

Real-time inventory tracking systems use DynamoDB Streams and Lambda to keep stock data synchronized across services.

💼 Career

Understanding stream processing patterns with DynamoDB is valuable for backend developers and cloud engineers working on scalable, event-driven applications.

1

Create the Products table with stock attribute

Create a DynamoDB table named Products with a primary key called ProductID (string type). Add an attribute called Stock to store the stock quantity (number type). Use AWS CLI or AWS SDK commands to define this table.

DynamoDB

# Your AWS CLI or SDK command to create the Products table with ProductID as primary key and Stock attribute

Need a hint?

Use the AWS CLI create-table command with --attribute-definitions and --key-schema to define the primary key.

2

Enable DynamoDB Streams and set stream ARN variable

Enable DynamoDB Streams on the Products table with the view type set to NEW_IMAGE. Create a variable called stream_arn that stores the stream ARN of the Products table.

DynamoDB

aws dynamodb update-table \
    --table-name Products \
    --stream-specification StreamEnabled=true,StreamViewType=NEW_IMAGE

# Create a variable called stream_arn to hold the stream ARN
# Your code here

Need a hint?

Use update-table to enable streams and describe-table to get the stream ARN.

3

Write Lambda function to process stream records

Write a Lambda function in Python named process_stream_records that takes event and context as parameters. For each record in event['Records'], extract the ProductID and Stock from the NewImage. Use these values to update the StockStatus table with the latest stock quantity.

DynamoDB

import boto3

dynamodb = boto3.resource('dynamodb')
stock_table = dynamodb.Table('StockStatus')

def process_stream_records(event, context):
    # Your code here

Need a hint?

Loop through event['Records'], extract ProductID and Stock from NewImage, then update StockStatus table.

4

Configure Lambda to use stream ARN and complete setup

Configure the Lambda function to use the stream_arn variable as the event source. Add the necessary permissions and complete the setup so that the Lambda function triggers on changes in the Products table stream.

DynamoDB

# Use AWS CLI or SDK to create event source mapping for Lambda with stream_arn
# Your code here

Need a hint?

Use create-event-source-mapping AWS CLI command to connect the Lambda function to the DynamoDB stream using stream_arn.