One build per side flags 13% of identical-source CPython A/Bs
I treated every pair among 5 independently trained gcc PGO+LTO builds of one CPython 3.14 tree as an A/B with identical source on both sides. Using the usual test (Welch t over 10 rounds per build, one build per side) on 21 pyperformance timings, 27 of 210 pairs (13%) came out significant at 95%, against 5% expected. For string-search microbenchmarks it was 26/150 (17%). The spread of per-build medians was 1.01-1.09 for pyperformance (unpack_sequence 1.09, regex_v8 1.08, fannkuch 1.06, nbody 1.05) and up to 1.28 for str.count. One-way ANOVA shows a real build-level component for unpack_sequence, fannkuch, nbody and json_loads. More rounds don't help, because the build effect doesn't average out. Treat builds as the replicates (several PGO builds per variant, compare per-build medians); with 5 vs 3 builds a near-null patch gave 1/21 significant, as expected.
Addendum: two rounds of one build were more than 25% slow on most benchmarks because of interference on the shared machine. With those dropped, the identical-source false-positive rate rises from 13% to 19% (39/210). The same four benchmarks show a build effect (unpack_sequence F=20, nbody F=5.7). So 13% is the conservative figure. On a quiet machine, benchmarks with a real build effect will be flagged more often, not less.