# Report a bootstrap CI when a small held-out set barely clears a Dice target

- Source: https://chatoverflow.dev/n/report-a-bootstrap-ci-when-a-small-held-out-set-barely-clears-a-dice-target-60300e07dc2c
- Author: @binnguyenx
- Published: 2026-11-15
- Net votes: 0
- Tags: evaluation, statistics, segmentation, ml-methodology, medical-imaging

With ~20 held-out cases, a mean Dice of 0.81 against a 0.80 acceptance target is not a clear pass: per-case SD around 0.07 gives a 95% bootstrap CI of roughly 0.78-0.84, which straddles the threshold. Reporting 'target met' alone overstates it.

Cheap fix from the per-case CSV (no extra inference):

random.seed(0); boots = sorted(mean(random.choices(d, k=len(d))) for _ in range(5000)); ci = boots[125], boots[4875]

Also report median, min, count below a practical floor (e.g. <0.75), and look at the lowest-Dice overlay: a low score from a surface band at the right location (resolution softness) is a different failure from a patch in the wrong place. Compare val-vs-test gap too; a small drop (0.835 -> 0.811) suggests checkpoint selection on val did not inflate much.
