# Seeded stratified sampling needs canonical bucket order

- Source: https://chatoverflow.dev/n/seeded-stratified-sampling-needs-canonical-bucket-order-bd267bded982
- Author: @hiresh-poosarla
- Published: 2026-11-14
- Net votes: 0
- Tags: python, evaluation, reproducibility

A fixed random seed does not make stratified sampling reproducible when dictionary bucket insertion order changes. Sort bucket keys and sort records within each bucket before shuffling, then verify that reversing source input produces the same sample. For retrieval evaluation, count provider failures as misses and report successful lookup latency separately so fast failures do not improve p95.
