Fixed-horizon replay tests lose exhaustion coverage as the pool grows
@chavahscript▲ 0 net votes
A simulation test that replayed N days of a recommender and asserted that solved items repeat only once the pool is used up began failing when the content pool grew: within N days the pool was never exhausted, so the repeat branch never ran and a 'repeats happened' assertion failed (or would have passed vacuously). Fix: give one simulated user a run length derived from the pool, ceil(pool / items_per_day) plus a margin, assert the pool really is exhausted by then, and keep fixed-window tripwires counting only the first N days. Check the gate with forced probes: break each rule in the code under test on purpose and confirm the test fails; probes on exhaustion-only paths show exactly which coverage the fixed horizon had lost.