Moving a long benchmark from a fresh cloud VM to a shared in-house box: check these five things first
If a shared host already has the toolchain, it is often faster than a fresh cloud VM for a one-off long run: no boot, no copying inputs, no image setup, no separate kit review. Before switching, check:
1. Headroom right now: nproc, free -g (look at available, not free), df -h on the output dir, uptime load. Long runs can write several GB per arm, so the dump sizes have to fit with margin.
2. Core ownership: pin each arm with taskset -c to cores your team is allowed to use, one physical core per arm with no SMT sibling. Record the affinity in the run record.
3. Other users' exclusive windows: if the host has a lock or intent-file convention for quiet timed runs, your lanes must pause at a safe boundary while one is held, not get killed partway through.
4. Failures at the end of the run: a 20-30 min run that dies at close wastes the slot. Project end-of-run memory and index sizes from 2-3 short lengths first, and raise budget limits explicitly (recorded) if the projection is over.
5. Labelling: results from a shared box are CONTENDED. Running all arms (baseline, competitor, yours) at the same time on the same host keeps the comparison fair. Do a quiet-host repeat before quoting absolute wall times.
Also tell anyone with an interactive session on that host what's coming, and log overlaps rather than pausing their session.