Import from S3 in DynamoDB

Introduction

Importing from S3 lets you quickly add lots of data into your DynamoDB table from files stored in Amazon S3. It saves time and effort compared to adding items one by one.

You have a large list of customer records saved as files in S3 and want to add them to your DynamoDB table.

You want to migrate data from another system stored in S3 into DynamoDB.

You need to restore data backups saved in S3 into your DynamoDB table.

You want to load bulk data for analysis or testing from S3 into DynamoDB.

Syntax

DynamoDB

aws dynamodb import-table --s3-bucket-source S3Bucket=<bucket>,S3KeyPrefix=<prefix> --input-format <format> --table-name <table-name> [--client-token <token>]

--input-format specifies the file format like DYNAMODB_JSON or ION.

--s3-bucket-source specifies the S3 bucket and key prefix of your data files.

Examples

Imports DynamoDB JSON files from the S3 prefix 'data/' in bucket 'mybucket' into the 'Customers' table.

DynamoDB

aws dynamodb import-table --input-format DYNAMODB_JSON --s3-bucket-source S3Bucket=mybucket,S3KeyPrefix=data/ --table-name Customers

Imports DynamoDB JSON files from the S3 prefix 'backup/' in bucket 'mybucket' into the 'Orders' table.

DynamoDB

aws dynamodb import-table --input-format DYNAMODB_JSON --s3-bucket-source S3Bucket=mybucket,S3KeyPrefix=backup/ --table-name Orders

Sample Program

This command starts importing data from DynamoDB JSON files stored in the S3 bucket 'example-bucket' under the prefix 'sample-data/' into the DynamoDB table named 'SampleTable'.

DynamoDB

aws dynamodb import-table --input-format DYNAMODB_JSON --s3-bucket-source S3Bucket=example-bucket,S3KeyPrefix=sample-data/ --table-name SampleTable

Important Notes

Make sure your IAM user or role has permission to read from the S3 bucket and write to the DynamoDB table.

The import process runs asynchronously; you can check the status using AWS CLI or Console.

Supported input formats include DYNAMODB_JSON and ION.

Summary

Importing from S3 helps load large data sets into DynamoDB quickly.

You specify the S3 location and file format to start the import.

Permissions and correct file format are important for a successful import.