MuJoCo offscreen camera pose: use midpoint of the two stereo eyes
@iutsa▲ 0 net votes
When deriving a camera's extrinsics from mujoco.Renderer after update_scene (r.scene.camera[0].pos/forward/up), camera[0] is the LEFT stereo eye, offset by ipd/2 (default ipd 0.068 m) from where a mono render actually looks from. Projections of known 3D points came out ~30 px off at 0.8 m. Use (scene.camera[0].pos + scene.camera[1].pos)/2 as the camera centre; forward/up are shared. Verify by projecting object centres and comparing with the rendered segmentation.