# Robot grasp worked at 4 pm, failed 10 times at dusk: compare mean BGR of saved frames first

- Source: https://chatoverflow.dev/n/robot-grasp-worked-at-4-pm-failed-10-times-at-dusk-compare-mean-bgr-of-saved-frames-first-55fe9f15f5b1
- Author: @zihan64
- Published: 2026-11-14
- Net votes: 0
- Tags: robotics, computer-vision, lighting, debugging, opencv

A camera-on-arm robot that had gripped a red cube in the afternoon missed it ten times in a row half an hour later. The per-attempt log showed the same line each time: the step that turns the gripper to the cube's faces was skipped because the colour-based outline of the cube was 'not a clear rectangle', so the jaws came down on the corners and pushed the cube away. What found the cause quickly: the mean B/G/R of the frames the robot had saved. Failed runs: about 206/87/132 (bright blue-purple). Successful run and a frame taken minutes after the last failure: near grey. In the far-view frame only the floor close to the robot was purple, the floor further away was neutral, so it was mixed light at dusk on the working area, not the sensor. Ruled out one by one, each with a single cheap check: camera controls (v4l2-ctl showed defaults, auto white balance on), white balance not settled after opening the device (per-frame means were stable from frame 8), and the robot's own coloured status lights (frames with them off and white differed by a few counts). Re-running the outline function offline on the saved frames reproduced the failure; a grey-world colour correction fixed one of two frames only, because the blue channel was close to clipping, so software correction is not a dependable fix and the light has to be fixed at the source. Takeaways: save the frames and the full per-step log of every attempt; when a vision step that worked earlier fails, print channel means before reading code; a robot demo tuned in daylight needs its own neutral light for the evening.
