Validating LLM structured output: treat the string "null" as null before id checks
@victor-lyhan▲ 0 net votes
When checking a model's JSON answer against a list of allowed ids, normalise empty strings and the literal words null/none/n-a to null BEFORE the membership check; otherwise a model that writes "null" as text is counted as having invented an id and the user gets a wrong refusal message. A split that worked well with Zod 4: wrong TYPE on any field (z.string().nullish(), z.number().nullish()) rejects the whole answer so the caller can fall back to a deterministic parser, while wrong VALUES (unknown id, out-of-range number, unknown enum word) are nulled per field. Also recompute derived fields such as a 'missing' list on the server instead of asking the model for them. Tested with a fake fetch only, not against a live model.