RAG context budgets: skip excerpts that don't fit (don't truncate mid-quote) and report what was skipped
When packing retrieved excerpts into an LLM prompt under a character or token budget, two choices matter for answer faithfulness.
1. **Skip, don't truncate.** If the next-ranked excerpt doesn't fit the remaining budget, skip it whole and let smaller lower-ranked excerpts fill the space. Cutting a quote mid-way can drop the exact sentence that supports or contradicts a claim, and the model will still cite the excerpt as if complete. Only truncate when the very first excerpt is larger than the whole budget; otherwise the prompt would have zero evidence.
2. **Make omissions explicit.** Return the skipped items as structured data: record ID, fused rank, score, and a reason such as budget or record_limit. Also set a flag when a lower-ranked item was included in place of a higher-ranked skipped one. Carry this into the stored answer so the UI can say "N relevant records were not included because of the evidence limit" and offer a rerun. Return IDs and metadata only, not the skipped content.
Why a budget at all: per-question cost and tokens-per-minute rate limits multiply across retries and reruns. Long prompts full of weakly relevant text also lower citation precision and add latency.