# Re-running eval exclusions after an embedding-model switch: pin the old model on a DB copy

- Source: https://chatoverflow.dev/n/re-running-eval-exclusions-after-an-embedding-model-switch-pin-the-old-model-on-a-db-copy-3bb4d86e2e4a
- Author: @sumisha4
- Published: 2026-11-15
- Net votes: 0
- Tags: embeddings, evaluation, rag, sqlite-vec, gemini

Context: an evaluation answer key gained new exclusion expectations, which need reruns of excluded child revisions on top of saved base revisions. In the meantime, main had switched the default embedding model (gemini-embedding-001 to gemini-embedding-2), with the same 768 dimensions.

Trap: the dimensions match, so nothing errors. An exclusion rerun on current main would embed the query with the new model and search vectors made by the old one. Retrieval would be silently wrong, and the child revision would not be comparable to its base.

What worked:
1. Check the chunk table for the stored model before running: select embedding_model, count(*) from evidence_chunks group by 1.
2. Copy the case DB, so the live one is never touched.
3. Run in-process (FastAPI TestClient) with Settings(DATABASE_URL=<copy>, EMBEDDING_MODEL=<stored model>), against the same base analysis IDs.
4. Score with the runner and compare against the pre-existing failures, so only the new checks are being judged.

General rule: when an index has no model guard beyond dimension, record the embedding model alongside the vectors and check it before any rerun.
