acceptodds
Under review as a conference paper at ICLR 2027

HEAR 360: Geometry-Aware Panoramic Depth Estimation from Multiple Posed Acoustic Observations

Abstract

We address the problem of audio-only panoramic depth estimation from multiple posed binaural observations. Existing audio-to-depth methods typically rely on a single forward-facing binaural pair and predict depth over a limited field of view, providing insufficient directional constraints for panoramic reconstruction. The challenge is further amplified by directional ambiguity and reverberation, which entangle geometric evidence from multiple propagation paths. We propose HEAR 360, which formulates multichannel audio depth estimation as a geometry-aware association rather than a conventional channel-stacking approach. Our framework introduces: (i) pose-conditioned acoustic encoding that preserves observation-specific evidence while incorporating receiver heading and ear identity, and (ii) geometry-aware cross-attention that represents each panoramic location as a 3D ray and associates it with the most relevant posed acoustic observations. We evaluate HEAR 360 on panoramic depth benchmarks derived from Matterport3D and Replica. Experiments show that HEAR 360 outperforms existing echo-based methods and controlled channel-stacking baselines with substantially fewer parameters. Further analyses show consistent gains from additional observations, as well as robustness to irregular observation subsets and unseen receiver headings, supporting the effectiveness of pose-aware association.

Then back it, or bet against it.

Related papers

Open the market on this paper to see 7 more related papers.