How to Delete Large S3 Bucket on CREODIAS
Due to an openstack-cli limitation, removing S3 buckets with more then 10 000 objects will fail when using the command:
openstack container delete --recursive <<bucket_name>>
showing the following error error:
Conflict (HTTP 409) (Request-ID: tx00000000000001bb5e8e5-006135c488-35bc5d520-dias_default) clean_up DeleteContainer: Conflict (HTTP 409) (Request-ID:)
To delete a large S3 bucket we can use s3cmd.
In order to acquire access to your Object Storage buckets via s3cmd, first you have to generate your own EC2 credentials with the following tutorial: How to generate and manage EC2 credentials on CREODIAS
After that, you have to configure s3cmd as explained in the following article: How to access private object storage using S3cmd or boto3 on CREODIAS
After this, you should be able to list and access your Object Storage.
List your buckets with the command:
eouser@vm01:$ s3cmd ls 2022-02-02 22:22 s3://large-bucket
Now you’re able to delete your large bucket with the command presented below, where -r means recursive removal.
s3cmd rb -r s3://large-bucket
The bucket itself and all the files inside will be removed.
WARNING: Bucket is not empty. Removing all the objects from it first. This may take some time... delete: 's3://large-bucket/example_file.jpg' delete: 's3://large-bucket/example_file.txt' delete: 's3://large-bucket/example_file.png' ... ... ... Bucket 's3://large-bucket/' removed
Your large bucket has been successfully removed and the list of buckets is empty.
eouser@vm01:$ s3cmd ls eouser@vm01:$