AwsConceptBeginner · 3 min read

What is S3 Multipart Upload: Explanation and Example

S3 multipart upload is a method to upload large files to Amazon S3 by splitting them into smaller parts and uploading each part separately. This approach improves upload speed, reliability, and allows resuming uploads if interrupted.

⚙️

How It Works

Imagine you want to send a big package by mail, but the package is too large to send all at once. Instead, you break it into smaller boxes and send each box separately. When all boxes arrive, they are combined back into the original package.

S3 multipart upload works the same way for large files. It splits a big file into smaller parts. Each part is uploaded independently to Amazon S3. If one part fails, you only need to resend that part, not the whole file. After all parts are uploaded, S3 joins them into the complete file.

This method makes uploading large files faster and more reliable, especially over unstable internet connections.

💻

Example

This example shows how to start a multipart upload, upload parts, and complete the upload using AWS SDK for JavaScript (v3).

javascript

import { S3Client, CreateMultipartUploadCommand, UploadPartCommand, CompleteMultipartUploadCommand } from "@aws-sdk/client-s3";

const s3 = new S3Client({ region: "us-east-1" });

async function multipartUpload(bucketName, keyName, fileParts) {
  // Start multipart upload
  const createCommand = new CreateMultipartUploadCommand({ Bucket: bucketName, Key: keyName });
  const createResponse = await s3.send(createCommand);
  const uploadId = createResponse.UploadId;

  const uploadedParts = [];

  // Upload each part
  for (let i = 0; i < fileParts.length; i++) {
    const partNumber = i + 1;
    const uploadPartCommand = new UploadPartCommand({
      Bucket: bucketName,
      Key: keyName,
      PartNumber: partNumber,
      UploadId: uploadId,
      Body: fileParts[i]
    });
    const uploadPartResponse = await s3.send(uploadPartCommand);
    uploadedParts.push({ ETag: uploadPartResponse.ETag, PartNumber: partNumber });
  }

  // Complete multipart upload
  const completeCommand = new CompleteMultipartUploadCommand({
    Bucket: bucketName,
    Key: keyName,
    UploadId: uploadId,
    MultipartUpload: { Parts: uploadedParts }
  });

  const completeResponse = await s3.send(completeCommand);
  return completeResponse;
}

// Usage example (fileParts should be array of Buffer or Blob parts):
// multipartUpload('my-bucket', 'large-file.zip', [part1, part2, part3]).then(console.log);

Output

{ Location: 'https://my-bucket.s3.amazonaws.com/large-file.zip', Bucket: 'my-bucket', Key: 'large-file.zip', ETag: '"etagvalue"' }

🎯

When to Use

Use S3 multipart upload when you need to upload large files, typically over 100 MB, to Amazon S3. It helps when your internet connection is slow or unstable because you can retry only the failed parts instead of the whole file.

Common use cases include uploading videos, backups, large datasets, or any big files that benefit from faster and more reliable uploads.

✅

Key Points

Multipart upload splits large files into smaller parts for upload.
Each part uploads independently, allowing retries on failures.
After all parts upload, S3 combines them into the final file.
Improves upload speed and reliability for big files.
Supported by AWS SDKs for easy integration.

✅

Key Takeaways

S3 multipart upload breaks large files into parts to upload separately and faster.

It allows retrying only failed parts, making uploads more reliable.

Use it for files larger than 100 MB or when network is unstable.

AWS SDKs provide built-in support to manage multipart uploads easily.

After uploading all parts, S3 assembles them into the complete file.