Before characterizing a tool bug, test whether your inputs respect real constraints
When you think a tool gives a wrong answer, test the premise before you scale the tests. A big, careful test matrix built on an unrealistic input only multiplies the mistake.
**What happened.** I characterized what looked like a simulator bug: Synopsys VCS returns 0 for {4{en}} == 4'b0101 when en is x, where the IEEE standard says x. It reproduced on three releases, and I built about 200 single-test files and ran about 2,000 simulator runs across x sources, surrounding code, consumers and dump/debug modes. An adversarial fact-checker verified every number. Then the person I was working with asked the obvious question: x means "could be 1", but can this comparison ever actually be true? It can't. Four copies of one bit are only ever 0000 or 1111. Every test constant was a pattern the operand can never form. Against patterns it can form (1111, 0000), VCS gave the standard's x in every run. The "bug" was a non-standard but hardware-accurate shortcut on comparisons that are always false in real hardware.
**Checklist before characterizing a tool bug:**
1. **Write down the input's real constraints**, i.e. which values it can actually take. Replicated bits are correlated; a sign-extended field ties its upper bits to its sign bit; an enum can only hold its members; an API field may be validated upstream. 2. **Test reachable and unreachable values side by side.** If the "wrong" answer appears only on unreachable inputs, the tool may be exploiting a constraint your test ignores. 3. **Compare with ground truth downstream, not only with the spec.** What would the real hardware, the synthesized netlist or the production system do? If the tool matches reality and differs only from the spec, that's a conformance note, not a correctness bug. 4. **Get one realistic repro before scaling up.** Ask: "Would someone write this on purpose, and what goes wrong for them?" If you can't answer, don't build the matrix yet. 5. **Check the premise, not just the numbers.** Fact-checkers and reviewers confirm that tables match logs. They rarely ask whether the scenario matters. Ask that question explicitly. 6. **Be suspicious of "realistic" examples you generate yourself.** Mine had a realistic setup (an unreset flop, a mux, a register) but still used the impossible comparison. A truly realistic line in the same file, comparing against a value that can occur, behaved correctly, and that was the clue.