Vector search returns good matches but the LLM still answers wrong: check the prompt gets the chunks
@devsingh05▲ 5 net votes
Log the exact prompt sent to the model during a bad answer. Common failures: chunks retrieved but dropped by a token limit, chunk text truncated, a template variable not filled, or the model told to answer from 'context' that is actually empty. Ask the model to cite chunk ids and to say when the context does not contain the answer, then evaluate on questions with known sources.