Lambda with DynamoDB Streams in AWS - Mini Project: Build & Apply

Lambda with DynamoDB Streams

📖 Scenario: You are building a simple serverless application where changes in a DynamoDB table trigger a Lambda function. This Lambda function processes the stream records from DynamoDB to react to data changes.

🎯 Goal: Create an AWS Lambda function triggered by DynamoDB Streams. The Lambda function will receive stream events and log the event details.

📋 What You'll Learn

Create a DynamoDB table with streams enabled

Create a Lambda function with the correct IAM role

Configure the Lambda function to be triggered by the DynamoDB stream

Write Lambda code to process and log stream events

💡 Why This Matters

🌍 Real World

Many serverless applications use DynamoDB Streams to react to data changes in real time, such as updating caches, sending notifications, or triggering workflows.

💼 Career

Understanding how to connect DynamoDB Streams with Lambda is a common task for cloud engineers and developers working with AWS serverless architectures.

1

Create DynamoDB Table with Streams Enabled

Create a DynamoDB table named Orders with a primary key OrderId of type string. Enable DynamoDB Streams with NEW_IMAGE view type.

AWS

# Create a DynamoDB table resource named 'Orders' with 'OrderId' as the partition key and streams enabled
# Your code here

Hint

Use AWS CloudFormation syntax to define a DynamoDB table resource. Remember to enable streams with StreamSpecification.

2

Create IAM Role for Lambda Function

Create an IAM role named LambdaExecutionRole with a trust policy for Lambda service and attach the managed policy AWSLambdaBasicExecutionRole. This role will allow Lambda to write logs.

AWS

Resources:
  OrdersTable:
    Type: AWS::DynamoDB::Table
    Properties:
      TableName: Orders
      AttributeDefinitions:
        - AttributeName: OrderId
          AttributeType: S
      KeySchema:
        - AttributeName: OrderId
          KeyType: HASH
      BillingMode: PAY_PER_REQUEST
      StreamSpecification:
        StreamViewType: NEW_IMAGE

# Create IAM role 'LambdaExecutionRole' with trust policy for Lambda and attach AWSLambdaBasicExecutionRole
# Your code here

Hint

Define an IAM role resource with a trust policy for Lambda and attach the AWS managed policy for basic Lambda execution.

3

Create Lambda Function with DynamoDB Stream Trigger

Create a Lambda function named ProcessOrderStream using runtime python3.12. Assign the role LambdaExecutionRole. Configure the function to be triggered by the DynamoDB stream of the Orders table with batch size 5 and starting position TRIM_HORIZON.

AWS

Resources:
  OrdersTable:
    Type: AWS::DynamoDB::Table
    Properties:
      TableName: Orders
      AttributeDefinitions:
        - AttributeName: OrderId
          AttributeType: S
      KeySchema:
        - AttributeName: OrderId
          KeyType: HASH
      BillingMode: PAY_PER_REQUEST
      StreamSpecification:
        StreamViewType: NEW_IMAGE

  LambdaExecutionRole:
    Type: AWS::IAM::Role
    Properties:
      RoleName: LambdaExecutionRole
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
          - Effect: Allow
            Principal:
              Service: lambda.amazonaws.com
            Action: sts:AssumeRole
      ManagedPolicyArns:
        - arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole

# Create Lambda function 'ProcessOrderStream' with runtime 'python3.12' and role 'LambdaExecutionRole'
# Configure event source mapping for DynamoDB stream from 'Orders' table
# Your code here

Hint

Define a Lambda function resource with inline Python code. Then create an event source mapping resource to connect the DynamoDB stream to the Lambda function.

4

Add Permissions for Lambda to Read DynamoDB Stream

Add an IAM policy named LambdaDynamoDBStreamPolicy to the LambdaExecutionRole that allows dynamodb:DescribeStream, dynamodb:GetRecords, dynamodb:GetShardIterator, and dynamodb:ListStreams actions on the DynamoDB stream ARN of the Orders table.

AWS

Resources:
  OrdersTable:
    Type: AWS::DynamoDB::Table
    Properties:
      TableName: Orders
      AttributeDefinitions:
        - AttributeName: OrderId
          AttributeType: S
      KeySchema:
        - AttributeName: OrderId
          KeyType: HASH
      BillingMode: PAY_PER_REQUEST
      StreamSpecification:
        StreamViewType: NEW_IMAGE

  LambdaExecutionRole:
    Type: AWS::IAM::Role
    Properties:
      RoleName: LambdaExecutionRole
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
          - Effect: Allow
            Principal:
              Service: lambda.amazonaws.com
            Action: sts:AssumeRole
      ManagedPolicyArns:
        - arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole

  ProcessOrderStream:
    Type: AWS::Lambda::Function
    Properties:
      FunctionName: ProcessOrderStream
      Runtime: python3.12
      Role: !GetAtt LambdaExecutionRole.Arn
      Handler: index.lambda_handler
      Code:
        ZipFile: |
          def lambda_handler(event, context):
              for record in event['Records']:
                  print('Stream record:', record)

  OrderStreamEventSourceMapping:
    Type: AWS::Lambda::EventSourceMapping
    Properties:
      EventSourceArn: !GetAtt OrdersTable.StreamArn
      FunctionName: !Ref ProcessOrderStream
      StartingPosition: TRIM_HORIZON
      BatchSize: 5

# Add IAM policy 'LambdaDynamoDBStreamPolicy' to 'LambdaExecutionRole' allowing DynamoDB stream read actions
# Your code here

Hint

Add an inline policy to the Lambda execution role allowing it to read from the DynamoDB stream ARN.

Practice

1. What is the main purpose of connecting AWS Lambda to DynamoDB Streams?

easy

A. To manually trigger Lambda functions from the AWS console

B. To automatically run code when data in the DynamoDB table changes

C. To backup DynamoDB data to S3 automatically

D. To increase the read capacity of the DynamoDB table

Solution

1. Step 1: Understand DynamoDB Streams role

   DynamoDB Streams capture data changes in the table as events.

2. Step 2: Understand Lambda integration

   Lambda can be triggered automatically by these stream events to run code.

3. Final Answer:

   To automatically run code when data in the DynamoDB table changes -> Option B

4. Quick Check:

   Lambda triggers on stream events = A [OK]

Hint: Streams trigger Lambda on data changes automatically [OK]

Common Mistakes:

• Thinking Lambda triggers manually from console
• Confusing streams with backups
• Assuming streams increase table capacity

2. Which of the following is the correct way to enable DynamoDB Streams on a table?

easy

A. Create a Lambda function and attach it to the table

B. Enable versioning on the DynamoDB table

C. Set the stream specification to NEW_AND_OLD_IMAGES in the table settings

D. Increase the write capacity units of the table

Solution

1. Step 1: Identify how to enable streams

   DynamoDB Streams are enabled by setting the stream specification on the table.

2. Step 2: Choose correct stream view type

   NEW_AND_OLD_IMAGES captures both before and after images of changes.

3. Final Answer:

   Set the stream specification to NEW_AND_OLD_IMAGES in the table settings -> Option C

4. Quick Check:

   Stream enabled via table settings = B [OK]

Hint: Enable streams in table settings, not in Lambda [OK]

Common Mistakes:

• Trying to enable streams by creating Lambda
• Confusing versioning with streams
• Changing capacity units instead of streams

3. Given this Lambda event handler code snippet triggered by DynamoDB Streams:

def lambda_handler(event, context):
    for record in event['Records']:
        if record['eventName'] == 'INSERT':
            print(record['dynamodb']['NewImage']['id']['S'])

What will this code print when a new item with id '123' is added to the DynamoDB table?

medium

A. '123' printed once for the new item

B. 'id' key printed instead of its value

C. An error because 'NewImage' does not exist

D. Nothing, because eventName is not 'INSERT'

Solution

1. Step 1: Understand event structure for INSERT

   For an INSERT event, 'NewImage' contains the new item data including 'id'.

2. Step 2: Analyze code behavior

   The code prints the string value of 'id' from 'NewImage' when eventName is 'INSERT'.

3. Final Answer:

   '123' printed once for the new item -> Option A

4. Quick Check:

   INSERT event prints new id value = D [OK]

Hint: INSERT events have NewImage with new item data [OK]

Common Mistakes:

• Assuming eventName is different
• Expecting OldImage on INSERT
• Confusing key name with value

4. You deployed a Lambda function triggered by DynamoDB Streams but it never runs. Which of the following is the most likely cause?

medium

A. The DynamoDB table does not have streams enabled

B. The Lambda function code has a syntax error

C. The DynamoDB table has no read capacity units

D. The Lambda function has no execution role

Solution

1. Step 1: Check stream enablement

   Lambda triggers on DynamoDB Streams events, so streams must be enabled on the table.

2. Step 2: Consider other causes

   While code errors or roles cause failures, the function would still be invoked if streams exist.

3. Final Answer:

   The DynamoDB table does not have streams enabled -> Option A

4. Quick Check:

   Streams must be enabled for Lambda trigger = C [OK]

Hint: Check if streams are enabled on the table first [OK]

Common Mistakes:

• Assuming code errors prevent invocation
• Ignoring missing streams
• Confusing read capacity with triggers

5. You want your Lambda function to process only updated attributes from DynamoDB Streams and ignore inserts and deletes. Which stream view type and event filter should you use?

hard

A. Enable NEW_AND_OLD_IMAGES stream view and filter events where eventName is 'INSERT'

B. Enable KEYS_ONLY stream view and filter events where eventName is 'INSERT'

C. Enable OLD_IMAGE stream view and filter events where eventName is 'REMOVE'

D. Enable NEW_IMAGE stream view and filter events where eventName is 'MODIFY'

Solution

1. Step 1: Choose stream view for updated data

   NEW_IMAGE provides the new state after update, suitable for processing changes.

2. Step 2: Filter events for updates only

   Filtering on eventName 'MODIFY' ensures only update events trigger Lambda.

3. Final Answer:

   Enable NEW_IMAGE stream view and filter events where eventName is 'MODIFY' -> Option D

4. Quick Check:

   Updates = NEW_IMAGE + 'MODIFY' filter [OK]

Hint: Use NEW_IMAGE and 'MODIFY' event filter for updates only [OK]

Common Mistakes:

• Using KEYS_ONLY which lacks attribute data
• Filtering on INSERT or REMOVE instead of MODIFY
• Choosing OLD_IMAGE which shows old data only