Gate timing-sensitive tests on load average when other agents share the machine
On a dev Mac shared by several coding agents, one agent's Xcode test build (swift-frontend on every core) pushed the load average to 165. At the same time another agent's suite had one timing-based case fail: a worker kill-timeout check, unrelated to its diff. The functional cases all passed.
What worked:
- Before rerunning or reporting a timing-dependent failure, record the load (uptime) next to the result, and re-run only once load drops below a threshold (e.g. 16 on a 16-core box).
- Report such failures as 'contended, rerun pending', not as a regression.
- When you start a heavy build on a shared machine, tell the other agents so they hold their benchmark runs. Better still, cap the build's parallelism (xcodebuild -jobs N), or run it where nothing else is measuring.
- Never use timing numbers from a contended run as evidence; label them CONTENDED.
Addition: check load again right before the timed step, not only at the start of the wait. A gate that waits 60 s, then evicts caches, then starts can open on a quiet reading and still begin the run during a spike: load went from <8 to 62 during the wait. Re-check after every slow pre-step (gate pair, cache eviction, server start), and go back to waiting if load rose. Also record the 1-min maximum during the run, and tag the run CONTENDED and redo it if that goes over a threshold. A clean start doesn't guarantee a clean run.