Cross-encoder reranking (bge-reranker-base) after hybrid retrieval raised Hit@1 76%->84% but cut Hit@5 96%->88%: rerank a wider pool and keep fused order as a tiebreak
Expect a cross-encoder to trade recall for precision at the top: measure Hit@1 and Hit@5 separately before switching it on. On a RAG index over the Nessie v2 API docs (296 chunks, all-MiniLM-L6-v2 + rank-bm25 hybrid with RRF k=60, 25 keyword-graded questions), reranking the top 15 RRF candidates with sentence-transformers CrossEncoder('BAAI/bge-reranker-base') and keeping 5 gave: - vector only: Hit@1 76%, Hit@3 84%, Hit@5 84% - hybrid RRF: Hit@1 68%, Hit@3 88%, Hit@5 96% - RRF + reranker: Hit@1 84%, Hit@3 88%, Hit@5 88% So the reranker moved the best chunk to the top more often, but on a net 2 of 25 questions it pushed every chunk that answered the question below position 5. The 'required fields' topic (which endpoint needs merchant_id, status, amount_cents) dropped from 4/4 to 2/4 under the reranker; short schema chunks seem to lose to prose (likely, not verified per chunk). With 25 questions each question is 4 pp, so this is directional.
Options: if the LLM gets several chunks, keep plain RRF (best Hit@5). If you show one answer, use the reranker. Or combine: rerank a larger pool (k*4 to k*6), take the reranker's top 2-3, then fill the rest from RRF order. Fusing the reranker score with the RRF score via min-max normalization also avoids discarding lexical matches outright. On CPU, reranking 15 pairs per query is noticeably slower than retrieval; batch with predict(pairs, batch_size=16).