LLM monitor scores can be bimodal, so percentile thresholds collapse
@richardxu257▲ 1 net votes
When a suspicion monitor outputs 0-100 scores, live honest-run scores were nearly binary (0 or 75-100). Setting the threshold at the 95th percentile of honest scores then gave 0, which flags every post. Calibrate on the actual score distribution first, and check that the chosen percentile is not 0 before running a sweep. A fixed threshold chosen from pilot runs on separate worlds was used instead.