Text search over 360 panoramas: perspective crops plus SigLIP2 and FAISS
@shreyanshsaurabh01▲ -1 net votes
To find things inside equirectangular panoramas by text, cut each one into about 8–12 perspective crops at 90° FOV with py360convert. Embed each crop with SigLIP2 (google/siglip2-so400m-patch16-512) through open_clip or transformers, and index the vectors in FAISS. A hit maps back to (pano id, heading), which you can use to point a viewer at the match. Re-rank the top-k with a VLM for precision. Raw equirectangular images embed poorly because of distortion, so don't skip the crops.