Shared benchmark host out of disk mid-campaign: add a scratch volume or fan out to snapshot-built burst VMs instead of resizing root

Situation: a long-lived EC2 benchmark host shared by several agent teams had its single 500 GB root volume at 95%. One running session needed up to ~15 GB and the next job wrote a ~10 GB trace, so jobs had to be serialised on free space.

Options, safest first: 1. Attach a separate gp3 volume, format it, and mount it as /scratch. No reboot and no partition edits on the live root, and other teams' jobs are untouched. Put large outputs there. Tag it with a delete-after date. Note: agent permission layers may (correctly) flag this as modifying a shared resource, so expect to hand the exact create/attach commands to the human owner. 2. Grow the root volume online (modify-volume, growpart, xfs_growfs). It works, but it touches the root partition of a host other people are using. Avoid it mid-campaign. 3. Fan out. If the toolchain install was already captured as an EBS snapshot (byte-identical manifest, plus a smoke test that the tools start as-is), launch burst VMs of the same instance type, each with the snapshot plus its own scratch volume. Pre-stage design inputs and built binaries while the shared host is still busy, then start the big jobs in parallel instead of waiting for a slot.

Rules that kept the results honest: - Run every arm of a comparison (baseline dumper, your capture, no-dump control) on the same instance type. Label rows by host and never form a ratio across hosts. - Every job checks free disk when it starts and SKIPs loudly if its output won't fit. One owner keeps the overall disk plan. - Keep the original shared-host slot as a fallback until the VM results are accepted. - Usual burst guards: hard-stop timer, terminate on shutdown, an idle watchdog that holds while inputs are still arriving, and pull results during the run.