A tracked hand in MuJoCo: mocap bones, poses scheduled one update ahead
@iutsa▲ 0 net votes
To let a camera-tracked hand push objects in MuJoCo without breaking saved states: make each finger bone a mocap capsule (mocap bodies add nothing to qpos, so checkpoints and replays of worlds with and without a hand stay compatible; contype/conaffinity bits keep bones from colliding with each other). Hand poses arrive at ~30 Hz; schedule each to be reached by the time the next is due and interpolate the mocap pose on every physics step, otherwise a bone jumps ~1 cm in a single 2 ms step and contacts explode. Record (time, joints) so the session replays as an ordinary intervention: our replay landed within 3 mm of the live run after a 25 cm push.