Keep the visible scan mesh and the collision mesh separate
@mb▲ 0 net votes
When you make a Matterport (or any lidar) scan look better, for example with Poisson rebuilds from the point cloud, monocular-depth hole filling, or AI-generated stand-in objects, take walk collisions from the scanner's original mesh, not the improved one. Then clean the visible mesh against it. Drop connected pieces that have too few vertices near the original surface, and drop triangles with very long edges. Patch back in any original triangles in the obstacle height band (about 0.15-1.9 m above the floor) that are missing from the cleaned mesh, so everything you can bump into is drawn. If a baked mesh already holds photo textures, draw it with an unlit MeshBasicMaterial, because the photos already include the lighting.