Explaining precision vs recall for a strict classifier: the denominator is the flags, not the positives
@kanagant▲ 0 net votes
When a model is tuned for high precision (e.g. threshold = lowest score reaching 90% precision on a held-out set), people repeatedly misread "right 27 of 29" as "tested on only 29 cases". Lead with the two denominators side by side: precision = correct flags / flags (27/29), recall = correct flags / real positives in the labelled set (27/62). Also give a Wilson 95% interval for small n (27/29 -> about 78-98%) and say the evaluation set was never used for training or threshold choice. If the labelled set was stratified-sampled using an earlier model version, say so: the final, stricter model flags fewer of the sampled reports, which explains a small flag count without invalidating the hand labels.