SGA3R: Spherical Geometry-Aware 3D Reconstruction from Panoramas

Panoramic cameras capture the surrounding scene in a single exposure and eliminate errors caused by external multi-view registration. Nevertheless, a critical limitation remains when adapting perspective-pretrained 3D reconstruction models to panoramic imagery: conventional planar projection and positional encodings do not adequately reflect spherical geometry. We present SGA3R, a framework that incorporates spherical geometry into view projection and attention for 3D reconstruction from single or multiple panoramas. To represent each panorama with a compact set of informative perspective views, we formulate view projection as information-guided spherical-cap optimization under explicit coverage and overlap constraints. To encode spherical relationships within the pretrained backbone, we introduce Dual-Path Harmonic Encoding (DPHE) that integrates Additive Harmonic Encoding and Rotary Harmonic Encoding, where the latter lifts RoPE from SO(2) to SO(3) relative rotation between patch directions via irreducible subspaces of multiple orders. We further extend SGA3R to multiple panoramas using alternating attention across view, frame, and global domains, together with frame-conditioned pose refinement, while preserving the single-frame architecture. Experiments on indoor datasets Matterport3D, Stanford 2D-3D-S, as well as the outdoor dataset Mapillary Metropolis show that SGA3R achieves the lowest absolute relative depth error (AbsRel) among the evaluated panoramic methods, and the full model reduces AbsRel on Matterport3D by 11.9% relative to the strongest evaluated baseline, while using only six views per panorama and no cross-view layering. Multi-frame experiments on Holo360D further demonstrate improved performance in point cloud reconstruction. Code will be released.