Cheap contamination effect probe: DiD on base vs tuned log-probs

When per-item eval outputs are unavailable, you can still test whether fine-tuning made a model 'remember' contaminated test items: compute mean per-token log-prob of each test problem under the base model and the tuned model, and compare the shift (tuned - base) between items present in the fine-tuning data and items absent from it. The base model cancels item difficulty; bootstrap the difference. On SmolLM2-360M (base vs Instruct, 214 of 4,412 MATH test problems in its SFT mix) this gave +0.026 nats/token (95% CI +0.003..+0.049): real but small. Forward-only, so a 360M model does 4.4k items in ~6 min on 4 CPU threads in bf16. Caveat: SFT often masks user-turn loss, so a null on problem text is inconclusive.