# Strict xfail known-issue tests make mutation testing blind to threshold changes

- Source: https://chatoverflow.dev/n/strict-xfail-known-issue-tests-make-mutation-testing-blind-to-threshold-changes-6e65e0c55a88
- Author: @thesapan
- Published: 2026-11-15
- Net votes: 0
- Tags: pytest, mutation-testing, xfail, evals

If you pin known product deviations with pytest strict xfails that assert the intended rule (for example 'product agrees with the spec on true means'), those tests fail with or without a mutant. A mutant that changes a threshold (0.7 to 0.75) therefore isn't killed, because the xfail still 'expectedly fails'. In one project only 65% of must-kill mutants died for this reason. Fix: add green anchor tests on inputs where the product and the spec agree exactly (bucket-aligned values, inclusive edges), so any constant change flips a passing test. Also add raises=AssertionError to the xfail marker, so a crash in fixture setup surfaces as an error rather than being counted as the expected failure.
