Delete many S3 objects safely

Part 21 of AWS from Zero. Today we delete a known set of keys with one API request.

What we are learning

delete-objects accepts up to 1,000 object identifiers in one request. It is useful when you already know the exact keys to remove.

Before you run it

REGION="ap-south-1"
BUCKET="aws-zero-delete-many-12345"
 
aws s3api create-bucket \
  --bucket "$BUCKET" \
  --region "$REGION" \
  --create-bucket-configuration LocationConstraint="$REGION"
 
echo "one" > one.txt
echo "two" > two.txt
echo "keep" > keep.txt
 
aws s3 cp one.txt "s3://$BUCKET/demo/one.txt"
aws s3 cp two.txt "s3://$BUCKET/demo/two.txt"
aws s3 cp keep.txt "s3://$BUCKET/demo/keep.txt"

Review the keys:

aws s3api list-objects-v2 \
  --bucket "$BUCKET" \
  --prefix "demo/" \
  --query "Contents[].Key" \
  --output table

Delete two known keys

aws s3api delete-objects \
  --bucket "$BUCKET" \
  --delete '{"Objects":[{"Key":"demo/one.txt"},{"Key":"demo/two.txt"}],"Quiet":false}'

The response reports which keys were deleted and any errors.

Inspect the result

aws s3api list-objects-v2 \
  --bucket "$BUCKET" \
  --prefix "demo/" \
  --query "Contents[].Key" \
  --output table

Only demo/keep.txt should remain.

One tiny variation

For a reusable list, put the request in delete.json:

{
  "Objects": [
    { "Key": "demo/one.txt" },
    { "Key": "demo/two.txt" }
  ],
  "Quiet": false
}

Then run:

aws s3api delete-objects \
  --bucket "$BUCKET" \
  --delete file://delete.json

Common mistake

Do not generate a mass-delete request from an unreviewed list. First print the exact keys, confirm the bucket, and only then send the delete request.

In a versioned bucket, include VersionId when you intend to permanently delete a particular version.

Cleanup

aws s3 rm "s3://$BUCKET/" --recursive
aws s3api delete-bucket --bucket "$BUCKET" --region "$REGION"
rm one.txt two.txt keep.txt

Next, we will enable S3 Versioning and create two versions of one key.

AWS CLI delete-objects reference