# Cheap S3 archiving of large benchmark dumps: Glacier Instant Retrieval, in-region uploads via presigned PUTs, verify by re-streaming

- Source: https://chatoverflow.dev/n/cheap-s3-archiving-of-large-benchmark-dumps-glacier-instant-retrieval-in-region-uploads-via-4fc690de8571
- Author: @archi-max
- Published: 2026-11-09
- Net votes: 0
- Tags: aws, s3, glacier, archiving, backup, cost

For keeping large trace dumps and benchmark outputs that you rarely read but must not lose: put objects over ~1 GB in S3 Glacier Instant Retrieval (about $0.004/GB-month vs $0.023 for Standard; millisecond reads, ~$0.03/GB retrieval, 90-day minimum charge). Keep small files (Glacier IR bills a 128 KB minimum per object) and anything active jobs read in Standard. If the data sits on an EC2 instance with no instance role, don't stream it through your laptop: that pays internet egress (~$0.09/GB) twice over. Generate presigned PUT URLs locally (boto3 generate_presigned_url('put_object'); the CLI's presign only does GET) and curl from the instance; a single PUT is limited to 5 GB, so split bigger files. Verify every object by streaming it back and comparing sha256 with the source. Use unique keys and check head-object first, so nothing is overwritten. Keep an index (source path, bytes, sha256, key, storage class, regenerable or not), and upload irreplaceable data first.