- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper introduces DANCE, a new audio-visual navigation benchmark in continuous 3D environments where an embodied agent must navigate to a semantically grounded acoustic target that may be either moving or stationary. The paper also proposes DyGMap: a GDN extended with visual cross-attention, plus a dynamic goal map with EMA + temporal decay to suppress stale target evidence.
Critical strengths
The paper proposes a well-motivated task with a well-documented method.
Critical weaknesses
The proposed benchmark includes only three semantic categories, which makes the task considerably less challenging than the open-vocabulary grounding setting commonly adopted in recent work. The authors should expand the category set or justify the restricted scope. The method is largely incremental. The GDN is explicitly "building upon MAGNet", with minor adjustments, and the paper oversells the "dynamic" contribution. The experimental setup does not convincingly isolate why DyGMap wins, and several comparisons are of uncertain fairness. For example, the ablation studies do not provide enough factorial isolation of the claimed axes. In addition, the authors should more clearly document how the baselines were adapted to the task and why they fail.
Questions for authors
We kindly ask that the authors address each of the weaknesses raised in the review above, and clarify how the proposed benchmark and experiments account for the concerns noted therein.
4 comments
Response to the Reviewer, Part 2/2
4. Baseline adaptations, fairness, and failure modes. Appendix D.2 documents how each baseline is implemented and trained. As detailed in our common response Adaptation of DAV-Nav, we retained DAV-Nav's central design and replaced its grid-based waypoint selector with direct action prediction, the action interface used by all learning-based methods in our evaluation. We consider this a relatively fair comparison under this constraint.
The baselines differ in how they use multimodal observations, especially target-related cues.
(1) AV-Nav, SMT + Audio, and DAV-Nav do not explicitly estimate the target's semantic category and location, while SAVi and MAGNet do not use visual observations to improve these estimates. DyGMap uses complementary audio-visual cues to improve target estimation.
(2) AV-Nav, SMT + Audio, and DAV-Nav do not account for intermittent target sounds. When the audio input is all zeros, SAVi retains its previous category estimate and updates its location estimate using the agent's pose. In noisy settings, however, distractor sounds or background noise can keep the audio input nonzero even when the target is silent. SAVi may therefore continue updating its estimate from those sounds. DyGMap uses a memory-augmented GDN that, as in MAGNet, draws on episodic memory to continue estimating the target's category and location during temporary silence and after the target stops emitting sound entirely.
(3) AV-Nav, SMT + Audio, SAVi, and MAGNet do not use explicit spatial maps, while DAV-Nav lacks an explicit mechanism for updating target-location evidence as the target moves. DyGMap maps obstacles, explored regions, and the agent's trajectory alongside category-specific goal maps. Its dynamic goal-map update mechanism records when the target evidence at each location was last updated and gradually reduces the weight of that evidence if no new estimates support it. This can happen when the target moves away or when a transient prediction error places evidence at an incorrect location. The agent can thus distinguish recently supported target locations from earlier estimates while retaining the scene geometry. These maps also allow the agent to reason about which regions are occupied or have been explored, where targets of each category may be located, and how strong the evidence is at each location.
Our experiments and ablations support these design choices and help explain DyGMap's substantial performance gains over the baselines.
The authors’ response addresses most of my concerns. After considering it, I have decided to keep my original score.
Thank you for carefully considering our responses and for acknowledging that they address most of your concerns. We respect your decision and sincerely appreciate your constructive feedback and suggestions for improving our work.
Response to the Reviewer, Part 1/2
We thank the reviewer for recognizing the motivation behind DANCE. We would like to clarify the scope of the benchmark, the contributions beyond MAGNet, and the basis for our comparisons.
1. Scope of the target categories. Our common response
Limited Number of Target Categories in DANCEexplains our category choices and addresses open-vocabulary grounding in point 4. DANCE places greater emphasis on grounding the target sound in the visible object that emits it than on recognizing the sound category alone. For example, when the target in DANCE is a dog, a dog is present in the scene and its bark originates from the dog's location. As the dog moves through the scene, both the sound source and the visual asset move with it. The agent can therefore use spatially and semantically aligned audio-visual cues to locate the target. This grounding requirement makes expanding the category set more demanding, since each new category requires both suitable sound recordings and corresponding visual assets.2. Contributions beyond MAGNet and what the ablations isolate. DyGMap builds on MAGNet's goal descriptor network (GDN) by incorporating vision to estimate the target's category and location. In our revised evaluation along the same oracle trajectories, adding vision improves most target estimation metrics.
The larger change is dynamic goal mapping. DyGMap maps obstacles, explored regions, and the agent's trajectory alongside category-specific goal maps. For each map location, it records both the strength of the target evidence and the time of its last update. This allows the policy to distinguish recently supported target locations from earlier estimates while retaining the scene geometry.
In the noisy-setting ablations (Table 5), the first row provides a MAGNet-style reference. The SR gains from adding the geometric map (rows 2–3), the goal map (rows 3–4), and dynamic updates (rows 4 and 6) are 5.23, 1.69, and 2.31 percentage points, respectively. Rows 5 and 6 isolate the effect of incorporating vision into the GDN on navigation performance, showing an SR gain of 5.27 percentage points. These comparisons identify what each addition contributes in the tested configurations.
3. Contribution of dynamic goal-map updates. We agree that the original submission did not adequately establish how the benefit of dynamic updates varies with target motion. Our common response
Dynamic Goal-Map Ablations, Decay-Factor Sensitivity, and the Effect of Target Speedreports results on test sets constructed at seven target-to-agent speed ratios from 0.00 to 1.50 in steps of 0.25. The gains from dynamic goal-map updates vary with both target speed and acoustic conditions. Relative to DyGMap w/o dynamic, the largest SR gain is 3.51 percentage points at 0.25× (57.49% to 61.00%) in the clean setting and 4.24 percentage points at 0.75× (from 37.75% to 41.99%) in the noisy setting.