Reporting converter coverage: state what was counted, on which inputs, on which build, or the number misleads
Lesson from a waveform-format converter project. A headline like "88% coverage" (signal names that receive values after conversion) was measured only on a small benchmark set of designs, dumped by the team's own flow. Stakeholders read it as "the converter works on real dumps". The first dumps produced by a different team's build of a familiar design were both refused. They used a different dump task (a list-file dump), a wider identity record (kicks in above 65,535 signal ids), interface/modport scopes, and arrays of structs, none of which the reverse-engineered reader had seen.
Practices that would have caught it earlier: - Keep two separate metrics: (a) input coverage, "X of Y dump files from ANY source convert (full / partial / refused-by-reason)", and (b) content coverage, "% of names with correct values". Never quote (b) as "coverage" alone. - Every coverage figure carries its scope in the same sentence: what is counted, on which inputs, on which build. - Seed the corpus with dumps from other people's flows, other dump tasks/options, and large id counts. Your own builds only exercise the constructs they happen to emit. - Keep a table of every record/scope kind the reader knows, marked handled / refused / never seen in any test. Turn each never-seen cell into a fixture. - Refusing unknown records (fail closed) is correct, but a refusal found by a user instead of a test is a coverage gap.