Fork me on GitHub
#aws
<
2022-10-12
>
Drew Verlee16:10:13

does anyone have some aws-api code that will delete a bucket and all its objects? apparently aws doesn't believe i would ever want to do such a thing and requires this to be several steps.

😂 1
lukasz16:10:19

you need to combine ListObjectsV2 with DeleteVersion and/or DeleteObject - there's no single API call that nukes a bucket and its contents

Drew Verlee16:10:24

And then delete bucket. Yeah thats what i figured, it really feels like aws doesn't use there own api's. A file system where you cant delete a folder? That's unheard of.

lukasz16:10:11

because it's not a file system :-)

Drew Verlee16:10:06

Sure, but the fact remains, it's forcing it's users to jump through hopes for reasons i can't fathom.

lukasz16:10:48

It's because of customer support: if there was a DeleteBucketAndEverything API call, there's no way to recover, at least with object versioning (if it's enabled!) you have a chance to get your data back somehow. Source: I deleted customer data way too many times I'm caring to admit and had to build DR features into many products

Drew Verlee16:10:51

Thanks, yeah. I guess I see why they would think that, even if i fundementally disagree that adding steps makes things more secure. Where do you draw the line? What if you had to call aws and talk to a s3 supervisor to delete your bucket? Sure less buckets would get deleted, but at what cost?

lukasz16:10:55

You can't actually, I have a 2TB bucket that needs to go and support told me to do it myself, carefully

Drew Verlee16:10:58

can't what? call support? I was suggesting it would be a huge time sink if you had to call support, not that i wanted to.

lukasz16:10:08

sorry, what I meant was that I requested that from support (my co had/has a paid support tier) and they do not perform data deletion requests on your behalf - at least they didn't in my case

👍 1
lispyclouds17:10:01

depending on how big the bucket is ive seen creating a lifecycle policy of deletion after 0 daysand attaching to all objects is nicer. we had 3 ~550TB buckets, that did it much better than any direct api calls

☝️ 2
lukasz17:10:14

ah yeah, at bigger scales you can't use the API for deletions

lispyclouds17:10:35

If anyone’s wondering, that took about 2 days to delete 😅

😱 1
kenny17:10:07

Somewhat related: a technique for speeding up list object calls https://www.genui.com/open-source/s3p-massively-parallel-s3-copying

Dimitar Uzunov06:10:58

You can shell out to this: https://docs.aws.amazon.com/cli/latest/reference/s3/rb.html There is a --force option that deletes bucket contents

Dimitar Uzunov06:10:18

you may need to implement retries if you have lots of items in a bucket, I remember running this a more than once on a bucket

Dimitar Uzunov06:10:07

(clojure.java.shell/sh "aws" "s3" "rb" (str "s3://" bucket-name) "--force")

Dimitar Uzunov06:10:15

you must have the aws cli installed though