Copy S3 objects across buckets

Part 18 of AWS from Zero. Today the source and destination are different buckets.

What we are learning

The high-level aws s3 cp command can copy directly from one S3 URI to another. Your identity needs read access to the source and write access to the destination.

Before you run it

REGION="ap-south-1"
SOURCE_BUCKET="aws-zero-copy-source-12345"
DEST_BUCKET="aws-zero-copy-dest-12345"
 
aws s3api create-bucket \
  --bucket "$SOURCE_BUCKET" \
  --region "$REGION" \
  --create-bucket-configuration LocationConstraint="$REGION"
 
aws s3api create-bucket \
  --bucket "$DEST_BUCKET" \
  --region "$REGION" \
  --create-bucket-configuration LocationConstraint="$REGION"
 
echo "cross-bucket copy" > report.txt
aws s3 cp report.txt "s3://$SOURCE_BUCKET/reports/report.txt"

The command

aws s3 cp \
  "s3://$SOURCE_BUCKET/reports/report.txt" \
  "s3://$DEST_BUCKET/archive/report.txt"

No local download step is required.

Inspect the result

aws s3api head-object \
  --bucket "$DEST_BUCKET" \
  --key "archive/report.txt" \
  --query "{Size:ContentLength,Type:ContentType,Updated:LastModified}" \
  --output table

One tiny variation

Use the low-level API when you need explicit copy controls:

aws s3api copy-object \
  --bucket "$DEST_BUCKET" \
  --copy-source "$SOURCE_BUCKET/reports/report.txt" \
  --key "archive/report-api-copy.txt"

Common mistake

Cross-account copies may require permissions from both sides: permission to read the source object and permission to write into the destination bucket. KMS-encrypted objects can also require KMS permissions.

For this first example, keep both buckets in the same account and Region.

Cleanup

aws s3 rm "s3://$SOURCE_BUCKET/" --recursive
aws s3 rm "s3://$DEST_BUCKET/" --recursive
aws s3api delete-bucket --bucket "$SOURCE_BUCKET" --region "$REGION"
aws s3api delete-bucket --bucket "$DEST_BUCKET" --region "$REGION"
rm report.txt

Next, we will choose exactly which files sync should transfer.

AWS CLI high-level S3 command guide