How to Sync a Folder to AWS S3 Using AWS CLI

Use the aws s3 sync command to copy and synchronize a local folder with an S3 bucket. This command compares files and only uploads new or changed files, making syncing efficient and easy.

Syntax

The basic syntax for syncing a folder to an S3 bucket is:

• aws s3 sync <source> <destination>
• <source> is the local folder path you want to sync.
• <destination> is the S3 bucket URL, starting with s3://.
• You can add options like --delete to remove files in the destination that are not in the source.

bash

aws s3 sync <source> s3://<bucket-name>/<optional-path> [--delete]

Example

This example syncs the local folder myfolder to the S3 bucket my-bucket under the path backup/. It uploads new and changed files only.

bash

aws s3 sync ./myfolder s3://my-bucket/backup/

Output

upload: ./myfolder/file1.txt to s3://my-bucket/backup/file1.txt
upload: ./myfolder/image.png to s3://my-bucket/backup/image.png

Common Pitfalls

Common mistakes when syncing folders to S3 include:

• Forgetting to configure AWS credentials with aws configure.
• Using the wrong source or destination path, causing files to upload to unexpected locations.
• Not using the --delete option when you want to remove files from S3 that no longer exist locally.
• Confusing aws s3 cp (copy) with aws s3 sync (sync), where sync is better for folders.

Wrong way:

aws s3 cp ./myfolder s3://my-bucket/backup/ --recursive

This copies files but does not sync changes efficiently.

Right way:

aws s3 sync ./myfolder s3://my-bucket/backup/ --delete

Quick Reference

Command	Description
aws s3 sync s3:///	Sync local folder to S3 bucket path
--delete	Delete files in destination not present in source
--exclude "pattern"	Exclude files matching pattern
--include "pattern"	Include files matching pattern
aws configure	Set up AWS credentials and region

Key Takeaways

Use aws s3 sync to efficiently copy and update files between a local folder and S3.

Always configure your AWS credentials before running sync commands with aws configure.

Add --delete to remove files from S3 that no longer exist locally.

Check your source and destination paths carefully to avoid uploading files to wrong locations.

Use aws s3 sync instead of aws s3 cp --recursive for folder synchronization.