Import from S3 in DynamoDB - Mini Project: Build & Apply

Import Data from S3 into DynamoDB

📖 Scenario: You work for an online store that stores product information in DynamoDB. The product data is backed up in JSON files on Amazon S3. You want to import this product data from S3 into your DynamoDB table to keep your database updated.

🎯 Goal: Build a step-by-step DynamoDB import setup that reads product data from an S3 bucket and imports it into a DynamoDB table called Products.

📋 What You'll Learn

Create a DynamoDB table named Products with a primary key ProductID.

Specify the S3 bucket and JSON file path where the product data is stored.

Write the import configuration to connect the S3 data source to the DynamoDB table.

Complete the import setup with the necessary IAM role and import command.

💡 Why This Matters

🌍 Real World

Importing data from S3 into DynamoDB is common when migrating data, restoring backups, or syncing offline data sources.

💼 Career

Knowing how to import data into DynamoDB from S3 is valuable for cloud engineers, database administrators, and backend developers working with AWS.

1

Create DynamoDB Table

Create a DynamoDB table named Products with ProductID as the partition key of type String.

DynamoDB

# Define the DynamoDB table creation parameters
# Your code here

Hint

Use the AWS CLI create-table command with the correct attribute and key schema.

2

Specify S3 Data Source

Create a variable called s3_bucket and set it to the exact S3 bucket name store-product-backups. Also create a variable called s3_key and set it to the JSON file path backup/products.json.

DynamoDB

aws dynamodb create-table \
    --table-name Products \
    --attribute-definitions AttributeName=ProductID,AttributeType=S \
    --key-schema AttributeName=ProductID,KeyType=HASH \
    --provisioned-throughput ReadCapacityUnits=5,WriteCapacityUnits=5

# Define the S3 bucket and key variables
s3_bucket = "store-product-backups"
s3_key = "backup/products.json"
# Your code here

Hint

Assign the exact bucket name and file path to variables s3_bucket and s3_key.

3

Configure Import from S3

Create a JSON configuration string called import_config that specifies the S3 bucket and key as the data source, and the DynamoDB table Products as the target. Use the format required by AWS CLI for import from S3.

DynamoDB

aws dynamodb create-table \
    --table-name Products \
    --attribute-definitions AttributeName=ProductID,AttributeType=S \
    --key-schema AttributeName=ProductID,KeyType=HASH \
    --provisioned-throughput ReadCapacityUnits=5,WriteCapacityUnits=5

s3_bucket = "store-product-backups"
s3_key = "backup/products.json"

import_config = '{"S3BucketSource": {"S3Bucket": "store-product-backups", "S3KeyPrefix": "backup/products.json"}, "InputFormat": "DYNAMODB_JSON", "TableCreationParameters": {"TableName": "Products"}}'
# Your code here

Hint

Use a JSON string to specify the S3 bucket and key prefix, input format, and table name.

4

Complete Import Setup

Write the AWS CLI command to start the import from S3 into DynamoDB using the import_config JSON. Include the required IAM role ARN arn:aws:iam::123456789012:role/DynamoDBImportRole with the --role-arn option and specify the table name Products with --table-name.

DynamoDB

aws dynamodb create-table \
    --table-name Products \
    --attribute-definitions AttributeName=ProductID,AttributeType=S \
    --key-schema AttributeName=ProductID,KeyType=HASH \
    --provisioned-throughput ReadCapacityUnits=5,WriteCapacityUnits=5

s3_bucket = "store-product-backups"
s3_key = "backup/products.json"

import_config = '{"S3BucketSource": {"S3Bucket": "store-product-backups", "S3KeyPrefix": "backup/products.json"}, "InputFormat": "DYNAMODB_JSON", "TableCreationParameters": {"TableName": "Products"}}'

aws dynamodb import-table \
    --cli-input-json "$import_config" \
    --table-name Products \
    --role-arn arn:aws:iam::123456789012:role/DynamoDBImportRole
# Your code here

Hint

Use the import-table command with the JSON config, table name, and role ARN.

Practice

1. What is the main purpose of importing data from S3 into DynamoDB?

easy

A. To run queries directly on S3 files

B. To export data from DynamoDB to S3

C. To quickly load large data sets into DynamoDB

D. To backup DynamoDB tables automatically

Solution

1. Step 1: Understand the import feature

   Importing from S3 is designed to load data into DynamoDB efficiently.

2. Step 2: Differentiate import from other operations

   Exporting and backups are different features; import specifically loads data from S3.

3. Final Answer:

   To quickly load large data sets into DynamoDB -> Option C

4. Quick Check:

   Import from S3 = Load data fast [OK]

Hint: Import means loading data into DynamoDB from S3 [OK]

Common Mistakes:

• Confusing import with export
• Thinking import backs up data
• Assuming import runs queries on S3

2. Which of the following is the correct way to specify the S3 bucket location in a DynamoDB import command?

easy

A. "http://my-bucket/data.json"

B. "ftp://my-bucket/data.json"

C. "/my-bucket/data.json"

D. "s3://my-bucket/data.json"

Solution

1. Step 1: Identify the correct S3 URI format

   S3 locations use the prefix "s3://" followed by bucket and file path.

2. Step 2: Eliminate incorrect protocols

   HTTP, FTP, or local paths are not valid for S3 import specification.

3. Final Answer:

   "s3://my-bucket/data.json" -> Option D

4. Quick Check:

   S3 URI starts with s3:// [OK]

Hint: S3 paths always start with s3:// [OK]

Common Mistakes:

• Using http:// instead of s3://
• Using local file paths
• Using ftp:// protocol

3. Given the following import command snippet, what will happen if the S3 file format is not specified correctly?

aws dynamodb import-table --s3-bucket my-bucket --s3-key data.csv --table-name MyTable --input-format JSON

medium

A. The import will fail due to format mismatch

B. The import will succeed ignoring the format

C. The import will convert CSV to JSON automatically

D. The import will partially load data with warnings

Solution

1. Step 1: Understand input format importance

   The import command requires the input format to match the actual file format.

2. Step 2: Consequence of mismatch

   If the file is CSV but input format is JSON, the import will fail due to parsing errors.

3. Final Answer:

   The import will fail due to format mismatch -> Option A

4. Quick Check:

   Format mismatch causes import failure [OK]

Hint: Input format must match file format exactly [OK]

Common Mistakes:

• Assuming automatic format conversion
• Ignoring format mismatch errors
• Expecting partial data load on error

4. You run an import from S3 but get an error: "Access Denied". What is the most likely cause?

medium

A. The S3 bucket is empty

B. The IAM role lacks permission to read the S3 bucket

C. The DynamoDB table does not exist

D. The import command syntax is incorrect

Solution

1. Step 1: Analyze the error message

   "Access Denied" usually means permission issues accessing S3 resources.

2. Step 2: Check IAM role permissions

   The IAM role used must have read access to the S3 bucket to import data.

3. Final Answer:

   The IAM role lacks permission to read the S3 bucket -> Option B

4. Quick Check:

   Access Denied = Permission problem [OK]

Hint: Check IAM permissions if Access Denied error occurs [OK]

Common Mistakes:

• Assuming table existence causes Access Denied
• Thinking empty bucket causes Access Denied
• Blaming syntax errors for permission issues

5. You want to import a large JSON file from S3 into DynamoDB. The file contains nested objects. Which approach ensures the import succeeds without errors?

hard

A. Flatten nested objects into simple key-value pairs before import

B. Import the file as-is; DynamoDB supports nested JSON natively

C. Convert JSON to CSV format and import with CSV input format

D. Split the file into multiple smaller files and import separately

Solution

1. Step 1: Understand DynamoDB import format limits

   DynamoDB import expects flat JSON or DynamoDB JSON format; deeply nested objects can cause errors.

2. Step 2: Prepare data for import

   Flattening nested objects into simple key-value pairs avoids format issues during import.

3. Step 3: Consider alternatives

   Converting to CSV loses nested structure; splitting files helps with size but not format.

4. Final Answer:

   Flatten nested objects into simple key-value pairs before import -> Option A

5. Quick Check:

   Flatten nested JSON for smooth import [OK]

Hint: Flatten nested JSON before import to avoid errors [OK]

Common Mistakes:

• Assuming DynamoDB imports nested JSON directly
• Converting JSON to CSV losing structure
• Ignoring file size but not format issues