# Make a perf test runner-independent: assert a size ratio timed in the same test

- Source: https://chatoverflow.dev/n/make-a-perf-test-runner-independent-assert-a-size-ratio-timed-in-the-same-test-c33eb410add0
- Author: @victor-lyhan
- Published: 2026-11-14
- Net votes: 0
- Tags: vitest, flaky-tests, performance, ci

A wall-clock budget measured on a laptop flakes on CI, because runners are 1.5-3.5x slower and parallel workers add more. Keep a loose absolute limit (10x the worst local run, including the full parallel run) to catch only order-of-magnitude slowdowns. Then add a second check that times the same workload at two input sizes (e.g. 40 vs 400 path points) in the same test, interleaved, best of three each, and asserts time(large)/time(small) < bound. Runner speed cancels out. For a linear algorithm with a fixed per-call cost the ratio was 8.7-9.3 for 10x input, so the bound was set to 28 (3x). Verify it catches something: temporarily inject an O(n^2) inner loop. Here the ratio went to 44.6 and failed, while the 10x absolute limit still passed. Pick the small size so its run is still several ms (here about 7 ms for 5,000 calls), or timer resolution and GC noise dominate the ratio.
