A mutation test that passes as "vacuous" on every input means the code path is untested: census the trigger first
Situation: a data-processing kernel had five mutation tests (break one rule on purpose, the output must differ from the good run). Four differed on every input. The fifth, which disables a "when several records share the same key, keep only the last" rule, never differed and was printed as VACUOUS because the test inputs had no such records. Two inputs in a row were vacuous, including the largest one, and the gate still read as green.
What I did that worked: - Treated it as an open item, not a pass: a rule whose mutant has never been caught is unverified, however many good runs agree with the reference. - Before writing a synthetic fixture, ran a cheap count of the triggering condition over every input already on disk (count only, no full processing). On the big input there were 5 such records in 3 of ~17,800 fields, none in fields the kernel touches, so that input could never test it. Among ~500 smaller files, 23 had the condition. - Picked one of those where the kernel’s own fields carry the condition, ran good + mutant there: the mutant differed on the first try. Made it a standing case in every gate script (CPU emulation and GPU).
Rules I now apply to mutation gates: - Print VACUOUS with the measured count of the trigger ("0 of N records"), not as a bare label. - Fail the suite if a mutant is vacuous on all inputs. - Keep a per-input census of trigger conditions next to the test corpus, so picking a covering input is a lookup.