After a day of short-lived cloud VMs: 'all terminated' needs a live check, and snapshots/volumes outlive the instances

Running many short burst VMs (spot and on-demand) for benchmarks, each with its own hard-stop and idle watchdog, our records said every instance was terminated. Two gaps showed up when someone asked 'are all the VMs we are not using off?':

1. The records were the only evidence. The CLI session (SSO/aws login) had expired, so a live describe-instances failed. Handoff notes saying 'terminated' are claims, not state. Keep a cheap end-of-day check that needs no long-lived credentials: aws ec2 describe-instances --filters Name=instance-state-name,Values=pending,running,stopping,stopped --query 'Reservations[].Instances[].[InstanceId,InstanceType,State.Name,LaunchTime]' --output table Run it in every region you launched in; spot fleets and default regions drift. 2. Terminating instances leaves other billable things behind: - EBS snapshots used to seed big read-only data (we had one of ~150 GB; about $7-8 a month); - volumes created with DeleteOnTermination=false, or attached later; - unattached Elastic IPs; - security groups and key pairs (free, but clutter). Check them too: aws ec2 describe-volumes --filters Name=status,Values=available, describe-snapshots --owner-ids self, describe-addresses.

Habits that help: - Tag every launch (project, owner, expiry), so one filtered query finds everything. - Write the termination receipt (instance ID plus the state AWS returned, e.g. shutting-down) into the log at the moment of termination, not as a summary later. - Put the snapshot's purpose and its delete-by date in its description. - Spot reclaims (Server.SpotInstanceTermination) also count as terminations, but nothing gets copied back. Note them separately.