S3 bucket lifecycle 'expire all after N days' rules silently cover archive prefixes; use a separate no-expiry archive bucket

Field note from setting up a long-term archive for benchmark data in AWS S3.

Trap: the bucket being reused had a single lifecycle rule with an empty prefix filter (every object) that expired objects after 14 days. That suits scratch data, but an 'archive/' prefix written into the same bucket gets deleted too. A server-side copy into another prefix only restarts the clock. Worse, objects in Glacier Instant Retrieval / IA classes deleted early are still billed for their minimum storage duration (90 or 30 days).

Options and trade-offs: - Narrowing the catch-all to per-prefix rules keeps the archive, but any new scratch prefix an agent creates then never expires, so costs leak quietly. - Lifecycle rules can't exclude by tag, so 'expire everything except tagged objects' isn't possible. You'd have to tag every scratch object instead. - A separate archive bucket is cleanest: Block Public Access on, SSE-S3, versioning on (with noncurrent-version expiry), abort-incomplete-multipart rule, no expiration. Copy server-side in the same region, so there's no egress. Copying out of Glacier IR has a per-GB retrieval fee, which is cheap for tens of GB.

How to apply: - Before archiving into any existing bucket, run aws s3api get-bucket-lifecycle-configuration --bucket B and read every rule's Filter. - After setup, verify with get-bucket-lifecycle-configuration, get-public-access-block and get-bucket-versioning, and record the bucket in your archive index. - Let the old copies expire under the old rule rather than deleting them by hand.