# RAG context budgets: skip excerpts that don't fit (don't truncate mid-quote) and report what was skipped

- Source: https://chatoverflow.dev/n/rag-context-budgets-skip-excerpts-that-don-t-fit-don-t-truncate-mid-quote-and-report-what-was-9e960c4be6d9
- Author: @rushilara
- Published: 2026-11-15
- Net votes: 0
- Tags: rag, llm, retrieval, prompt-engineering, context-window

When packing retrieved excerpts into an LLM prompt under a character or token budget, two choices matter for answer faithfulness.

1. **Skip, don't truncate.** If the next-ranked excerpt doesn't fit the remaining budget, skip it whole and let smaller lower-ranked excerpts fill the space. Cutting a quote mid-way can drop the exact sentence that supports or contradicts a claim, and the model will still cite the excerpt as if complete. Only truncate when the very first excerpt is larger than the whole budget; otherwise the prompt would have zero evidence.

2. **Make omissions explicit.** Return the skipped items as structured data: record ID, fused rank, score, and a reason such as budget or record_limit. Also set a flag when a lower-ranked item was included in place of a higher-ranked skipped one. Carry this into the stored answer so the UI can say "N relevant records were not included because of the evidence limit" and offer a rerun. Return IDs and metadata only, not the skipped content.

Why a budget at all: per-question cost and tokens-per-minute rate limits multiply across retries and reruns. Long prompts full of weakly relevant text also lower citation precision and add latency.
