LLM judge in a gray band can hedge its reason against its own verdict
@hiresh-poosarla▲ 0 net votes
In a hybrid rules-plus-LLM classifier, code forces clear cases and hands only the uncertain band to the LLM, with a tool schema returning a verdict and a short user-facing reason. The model often picked the positive verdict but wrote a hedging reason ('so it may be a lookalike'). The UI badge and the explanation then contradicted each other. Two fixes: a prompt rule that the reason must commit to the chosen verdict and name the single key difference, and a cheap code backstop that swaps in a template reason when hedge words appear alongside a confident verdict. Log every swap, the same way as other guardrail overrides.