DevOps Asked by Scott Stensland on August 22, 2021
I need to automate deletion of aws S3 backups … evidently if I tag the backup with one of daily, weekly, monthly, yearly then aws will delete based on my desired retention counts per those periods however I see no easy way to determine which of those tags I give to my daily backup process which is
// Upload backup file
aws s3 cp /tmp/backup_2007020644.20200703.1248_blobs.tar s3://foo-bar-baz-someaccount-us-east-2/backup_2007020644.20200703.1248_blobs.tar --region us-east-2 --only-show-errors
aws s3api put-object-tagging --bucket foo-bar-baz-someaccount-us-east-2 --key backup_2007020644.20200703.1248_blobs.tar --tagging --region us-east-2 TagSet=[{Key=backuptype,Value=blobs}]
// now lets list this s3 bucket
aws s3 ls s3://foo-bar-baz-someaccount-us-east-2 --region us-east-2
2020-07-01 22:55:57 31904428 backup_2007010938.20200701.2233_blobs.tar
2020-07-01 22:55:43 893239 backup_2007010938.20200701.2233_mongo.tar
2020-07-02 15:30:36 34343354 backup_2007010938.20200702.1508_blobs.tar
2020-07-02 15:30:22 893676 backup_2007010938.20200702.1508_mongo.tar
2020-07-03 01:20:04 30596405 backup_2007020644.20200703.0055_blobs.tar
2020-07-03 01:19:51 893741 backup_2007020644.20200703.0055_mongo.tar
2020-07-03 12:48:44 34658003 backup_2007020644.20200703.1226_blobs.tar
2020-07-03 12:48:30 895294 backup_2007020644.20200703.1226_mongo.tar
2020-07-03 15:05:00 34657972 backup_2007020644.20200703.1248_blobs.tar
2020-07-03 15:04:46 895279 backup_2007020644.20200703.1248_mongo.tar
Alternatively I can code up my own logic to parse above listing then issue the delete command per backup file after my code keeps track of my retention policy. There must be a better way, any advice?
I think you're on the right track. Just need a tiny bit more logic on the backup process, then S3 can handle the rest.
Consider this:
Next, set up S3 Bucket Lifecycle Policies with each tag as the filter.
References:
Correct answer by Woodland Hunter on August 22, 2021
Get help from others!
Recent Questions
Recent Answers
© 2024 TransWikia.com. All rights reserved. Sites we Love: PCI Database, UKBizDB, Menu Kuliner, Sharing RPP