Swapping in a retrained detector on a robot: replay the robot's own saved photos through the full pick rules, not just mAP
When a team retrains the object detector a robot uses to choose what to fetch (here YOLO, 7 -> 9 classes, one new class being the drop-off container itself), the useful check is not the model's validation mAP but a replay of every photo the robot saved during earlier runs through the robot's real decision function (detector + the hand rules around it: aspect ratio, colour, edge cut-off, 'already in the container'), old model vs new, and a look at only the photos where the chosen target differs.
What that showed in one case (287 search photos): same pick in 237; the new model dropped several long-standing false targets (a coloured wall panel, big boxes, the other robot's arm) but also lost some small far objects. Raw per-class counts had suggested the new model was much worse at the main class (block 269 -> 198 at conf >= 0.5); most of that drop was the container itself, which the old model had labelled 'cup'/'block' because it had no class for it.
Three rules that came out of it: 1. A new class that names the container must be excluded from 'things to fetch' in code, not only in a config whitelist - with no whitelist every class counts. 2. 'Carries a fiducial tag' is a cheap, strong negative for debris: arena markers and robots wear tags, debris does not. Make the exclusion zone in tag widths and only BELOW the tag (the tag sits on top of the marker/robot); a zone that also reached above the tag excluded a real object lying behind the other robot. 3. Letting the detector replace a slow VLM confirmation needs a second, independent cue: here class score >= 0.6 AND >= 25 % of the box white-and-smooth (texture test). Score alone accepted white furniture, a heater grille and the white squares of a floor tag at up to 0.86.
Also check the export before shipping: ir_version and the set of ONNX op types should match a model that already runs on the target runtime.