acceptodds
Under review as a conference paper at ICLR 2027

D2AVN: Efficient Direction-Decoupled Audio-Visual Navigation in Multi-Source and Reverberant Soundscapes

Abstract

Audio-Visual Navigation (AVN) empowers embodied agents to leverage the target-specific acoustic cues to complement visual perception, enabling them to locate sound sources that are visually occluded or outside their immediate field of view. Previous AVN methods predominantly derive directional guidance by estimating the geometric source direction—the direct spatial vector from the agent to the target. However, this paradigm struggles in authentic indoor environments, where multi-source interference obscures target cues and reverberation causes the perceived acoustic Direction of Arrival (DOA) to diverge significantly from the direct source path. To this end, we propose D2AVN, a novel direction-decoupled framework that explicitly isolates target acoustic arrival evidence before translating it into actionable navigation guidance. Importantly, our D2AVN employs a novel target-conditioned DOA (TargetDOA) module that extracts target-specific arrival distributions directly from complex acoustic mixtures, utilizing paired target-only acoustic references that exclude competing sources while preserving environmental propagation. Then, our Depth-conditioned Reward-Aligned Egocentric Affordance Modeling (DREAM) module reshapes these arrival distributions into reward-aligned navigation direction preferences by incorporating local geometry and egomotion-aligned history. To facilitate robust policy learning in these complex spaces, we implement a training pipeline with effective online acoustic rendering and reinforcement learning. Extensive validation demonstrates that D2AVN achieves a 33.0% success rate with First-Order Ambisonics (FOA) and 15.8% with binaural audio. Notably, the binaural configuration improves Success weighted by Path Length (SPL) and action efficiency (SNA) by 74% and 90%, respectively, over the fine-tuned ENMuS³ baseline.

Then back it, or bet against it.

Related papers

Open the market on this paper to see 7 more related papers.