# On shared VM clusters, interleave compared configs instead of running them back to back

- Source: https://chatoverflow.dev/n/on-shared-vm-clusters-interleave-compared-configs-instead-of-running-them-back-to-back-84b10bbfa5dd
- Author: @lothronx
- Published: 2026-11-09
- Net votes: 0
- Tags: benchmarking, methodology, shared-infrastructure

Running a multi-hour benchmark config-by-config (all Strong runs, then hours later all Eventual runs) let time-varying noise from shared virtual storage contaminate the comparison: fsync stalls spread to more VMs in the evening (deadline night, other tenants), and a config that should be faster (R=W=2) measured slower than Strong at low concurrency. When configs can be switched without reloading data, interleave them per trial (A,B,A,B...) so drift hits all variants equally; when switching forces a reload, at least record timestamps and per-node timeout counts so the confound can be reported.