'Cheapest way to delete 2 billion objects from S3 IA

I have a bucket in S3 (Infrequent access) containing 2 billion objects. It is too big to delete in the console or over the api without taking years.

I can create a lifecycle rule to expire and delete the objects but the calculator predicts this will cost me >$20,000. Is that correct? Is there a better way to delete a bucket?

I have a file effectively containing a list of all the objects in that bucket if that helps.

Update 2021:

An answer below from @MAP points out that there is now an "Empty" button. I haven't tested yet, but looks like the way to go (I'll accept that answer once tested):

screenshot of empty button



Solution 1:[1]

If you have a list of all the objects available then you can certainly use Multi Delete Object action. Apparently this API is free. I would create AWS Step Functions state machine to loop through the file and delete 1000 objects at a time. 1000 appears to be the limit.

It will take around 2M step function transactions to delete all the objects in the bucket. As per the pricing for step function it will cost you around $50 + cost of Lambda invocations around $1 so total cost roughly $51.

Update

Using Lambda or Step Functions is probably not the most cost effective option because both ways you will need to read the file (that contains object keys) from some source such as S3. So I think running the script from local machine or any EC2 linux screen appears to be the best option.

Solution 2:[2]

In 2021, anyone who comes across this question may benefit to know that AWS console now provides an empty button.

Select the bucket and click on "empty" button and all objects versioned or not versioned would be emptied/deleted. Depending on the number of objects it can take minutes to days.

Solution 3:[3]

Expiration lifecycle rules are free. From the original feature announcement:

As with standard delete requests, Amazon S3 doesn’t charge you for using Object Expiration.

Solution 4:[4]

Delete operations are for free. You can create a lifecycle Policy to automate a bulk delete.

I would start with a small number of objects first and check billing report to 100% confirm that the delete will not be charged, then go for the rest.

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source
Solution 1
Solution 2 Jens
Solution 3 mR_fr0g
Solution 4 Sébastien Stormacq