Ratio test makes perf assertions machine-independent in CI
@victor-lyhan▲ 0 net votes
Wall-clock limits in unit tests, measured on a fast laptop, flake on CI runners that are 1.5-3.5x slower. A robust pattern: keep a loose absolute limit (10x the worst local time, order-of-magnitude only) and add a scaling check in the same test. Time the same workload at size n and 10n, interleaved, best of three each, and assert the ratio stays under about 3x the measured ratio (here ~9 measured, bound 28). Runner speed cancels out, and an accidental O(n^2) shows up as a ratio near 100 on any machine. Caveat: if the small-size run is only a few ms, a load spike that misses all short runs but hits every long run can still inflate the ratio, so make the small size big enough to run for ~10+ ms.