acceptodds
Under review as a conference paper at ICLR 2027

TraSNav: Transient-Sound-Aware Audio-Visual Navigation

Abstract

Audio-visual navigation has received considerable attention in recent years. However, in realistic multi-source environments, target sounds may occur as brief and temporally sparse acoustic events, providing intermittent and limited evidence for navigation. The key challenge is that these transient target cues can be easily affected by competing sounds, while their short-lived information must also be integrated with navigation context accumulated over much longer time scales. To address this problem, we propose TraSNav, a navigation framework for transient target sounds. Our key idea is to maintain an event-level memory that accumulates and consolidates information from fragmented acoustic observations to improve robustness against competing sounds, and then adaptively fuse this event memory with long-term navigation memory to handle temporal-scale differences. Experimental results demonstrate that TraSNav consistently outperforms representative audio-visual navigation methods, achieving up to a 26.41% relative improvement in success rate when each target-sound event is shortened to 10% of its original duration in multi-source environments.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Audio-Visual Navigation, Transient Sound

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 4 reviews, above 68% of the others.
Reviewer_yb7POpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents TraSNav, an embodied navigation method for the indoor audio-visual navigation task, where the target sound is brief, intermittent, and interfered with by other sounds. The agent utilizes the RGB-D observations, acoustic cues, and historical information to find the target sounding objects. To acquire a more reliable target representation, this paper proposes RTEM and TSAMF, which successfully improve the navigation performance in scenes where the target sound events are brief.

Critical strengths
  1. The evaluation protocol controls key variables well. The authors progressively shorten the target sound event to 90% - 10% of the original duration while preserving the event onset times, event counts, and the configurations of the interfering sound events.
  2. TraSNav outperforms other baseline methods in SR and SPL across multiple conditions, including the original settings and shortened scenarios.
  3. The authors provide ablation studies and analysis of training stability and module behavior.
Critical weaknesses
  1. Insufficient support for the claimed reliability improvement of RTEM. RTEM is used to improve the target-event representation's reliability. However, from the ablation studies in the main paper, the performance gain from RTEM alone is minimal. It is recommended that the author provide more direct evidence of the advantage of RTEM, such as the sound event detection accuracy and the sound event localization accuracy with and without the proposed RTEM.
  2. The contribution of the temporal-scope modulation factors is not isolated. In Equation 9 of the main paper, the authors use λt\lambda_t, a factor that decreases with the progress of navigation, thereby progressively reducing the influence of sound-event information in constructing the current scene memory. However, the current ablation experiments do not isolate the contribution of this factor. It is recommended that the author provide ablation experiments for this factor in different settings, such as a no-decay consistent factor.
  3. The authors only conduct experiments on multi-source scenarios. It is recommended that the authors provide more experiments on single-source scenarios and noisy scenarios to confirm the capabilities of the proposed RTEM and TSAMF in terms of target-sound information acquisition and noise resistance.
  4. Figure S1 shows that the interfering sound event remains active throughout the navigation process. However, in the multi-sound scenarios, the interfering sound event is also intermittently active. The authors also state that the interfering sound is active for about 44.11% of the navigation process. It is recommended that the authors clarify this ambiguity.
Questions for authors
  1. Could the authors provide direct evidence of the improved reliability of the target sound representations?
  2. Could the authors provide ablation studies of the scaling factor λt\lambda_t in TSAMF?
  3. Could the authors provide experiments on single-source and noisy scenarios?
8 comments

We sincerely thank you for your careful evaluation of our work and your positive assessment of our experimental controls, navigation performance improvements, and ablation analyses. We particularly appreciate your recognition of our controlled evaluation protocol under different target-sound duration conditions and the performance improvements achieved by TraSNav under brief target-sound conditions.

In response to the four major concerns you raised, we have conducted additional experiments and analyses to provide a more comprehensive evaluation of the proposed method. Specifically, regarding the reliability of target-event representations in RTEM, we further investigate the roles and stability of its individual components through actual attention retrieval behaviors, evidence-branch ablations, conflict projection calibration ablations, and multi-seed experiments. To examine the contribution of the temporal-scope modulation factor in TSAMF, we introduce a test-time ablation that retains the learned event gate while fixing the temporal decay factor. For single-source and noisy environments, we conduct additional evaluations using the existing model checkpoints to examine the performance of our method under different acoustic conditions. Furthermore, we have re-examined the actual activity schedules of interfering sounds and will further revise Figure S1 to accurately illustrate their intermittent activity patterns.

We hope that these additional experiments and analyses provide further insights into the design rationale of RTEM and TSAMF, the contributions of their individual components, and the applicability of TraSNav under different acoustic environments.

Response to Critical Weakness 4

We thank you for pointing out the potential ambiguity in the presentation of interfering-sound activity in Figure S1. We have further examined the interfering-sound schedules and the corresponding statistics of the original episodes.

In the actual multi-source test scenarios, interfering sounds are not continuously active throughout the navigation process. Instead, they occur intermittently according to predefined temporal schedules. According to the statistics of the original test set, interfering sounds are active for an average of 44.11% of the navigation duration, while the remaining 55.89% of the time corresponds to inactive periods.

The interfering-sound bar in the original Figure S1(b) was primarily intended to illustrate the experimental control principle that the interfering-sound schedule remains unchanged across different target-sound duration conditions, rather than to precisely represent its actual active intervals throughout the timeline. We acknowledge that the current illustration may lead readers to interpret the interfering sound as being continuously active.

To eliminate this ambiguity, we will revise Figure S1(b) in the revised manuscript to explicitly illustrate the intermittent active intervals according to the actual interfering-sound schedules. We will also clarify that the interfering-sound schedule remains unchanged across all target-sound duration conditions.

Q1. Could you provide direct evidence of the improved reliability of the target-sound representations?

We thank you for the suggestion regarding the reliability of target-event representations in RTEM. We agree that it is necessary to provide evidence beyond downstream navigation performance to demonstrate how RTEM improves target-event representations by integrating acoustic evidence from different sources.

We further analyze the role of RTEM from four aspects: actual evidence retrieval behaviors, ablation studies of different evidence branches, the individual contribution of conflict projection calibration, and multi-seed training stability.

First, Figure S2 in the supplementary material analyzes the actual retrieval behaviors of RTEM from spatial, temporal, and cross-episode perspectives. Figure S2(a) shows the spatial retrieval distributions of the Support and Conflict branches within the same representative episode. Figure S2(b) illustrates the attention distributions of the two branches across historical time steps, together with the activity intervals of target and interfering sounds. Figure S2(c) further reports cross-episode attention enrichment statistics over 100 test episodes. The results show that the Support branch exhibits stronger attention enrichment over target-sound-active history, whereas the Conflict branch exhibits corresponding attention enrichment over interfering-sound-active history. These results provide evidence from actual retrieval behaviors supporting the functional division of the two evidence branches, indicating that RTEM can retrieve supporting information and potential conflicting information relevant to target events from historical memory.

Second, the original Table 5 analyzes the contributions of different evidence sources to target-event representations by progressively introducing the Candidate, Support, and Conflict branches. The results show that introducing either recent supporting evidence or historical conflicting evidence on top of Candidate-only brings positive improvements in navigation performance, while the complete RTEM achieves better overall results by jointly utilizing these two complementary sources of evidence.

To distinguish the contributions of conflict-evidence retrieval and conflict projection calibration, we introduce an additional ablation variant, Full RTEM w/o Projection Calibration. In this setting, target-candidate retrieval, recent supporting-evidence retrieval, historical conflicting-evidence retrieval, and the subsequent negative-conflict fusion are all retained, while only the projection calibration operation in Eq. (5) is removed.

Table R1. Ablation of conflict projection calibration under different target-sound conditions (SR, %).

VariantOriginal80%50%20%
Full RTEM w/o Projection46.1942.4634.8919.71
Full RTEM46.2842.5835.2219.98

As can be observed, the complete RTEM improves SR by 0.09, 0.12, 0.33, and 0.27 percentage points under the four test conditions, respectively. This indicates that conflict projection calibration further suppresses components of the target representation that align with the conflicting-evidence direction, building upon the retained conflict-evidence retrieval, and provides a positive contribution to downstream navigation performance.

Finally, to further investigate whether the performance gains brought by RTEM are affected by training randomness, we conducted three independent training runs of the RTEM-only variant and compared the results with the three independent training runs of the complete TraSNav model previously reported in Supplementary Table S4.

Table R2. Multi-seed stability analysis of RTEM-only and full TraSNav (SR, %).

VariantOriginal80%50%20%
Baseline45.9641.7733.3018.67
RTEM-only46.31±0.1642.62±0.1835.21±0.2220.08±0.25
Full TraSNav48.96±0.1445.22±0.1638.37±0.2423.49±0.23

The results show that the standard deviations of SR for RTEM-only do not exceed 0.25 percentage points across the four target-sound duration conditions, which is comparable to the training variation of the complete TraSNav model, indicating good training stability across different random seeds. Further comparison shows that the mean SR of RTEM-only is higher than that of the original Baseline under all four test conditions, with improvements of 0.35, 0.85, 1.91, and 1.41 percentage points, respectively, demonstrating a consistently positive performance trend.

Reviewer_eaxMOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies audio-visual navigation in multi-source environments where the target sound is heard only briefly and sparsely. The authors argue that a single short event gives only partial evidence, which becomes unreliable under competing sources, and that event-level cues and long-horizon navigation context differ in temporal scale. They propose TraSNav, which adds two modules to the baseline. RTEM uses three learnable queries to retrieve candidate, supporting, and conflicting evidence from memory, and combines them through conflict calibration into an event representation b_t. TSAMF refines the current observation token toward b_t, weighted by a learned event gate and a temporal-scope coefficient that decays with memory length. For evaluation, the authors shorten only the durations of target events (90% down to 10%) at test time.

Critical strengths
  1. Consistent gains. TraSNav improves SR over ENMuS^3 by about 3 to 5 points in all 10 conditions. Detailed implementation information (Tables S1 to S3), three-run statistics for the proposed method (std 0.11 to 0.24).
Critical weaknesses
  1. The protocol does not capture the problem the paper poses. The claim that transient target sounds are common is not supported by examples, statistics, or citations. Truncating sustained sounds does not produce inherently short sounds, which differ in onset, decay, and reverberation characteristics. Shortening events also reduces total target information (active ratio 44.61% to 4.46% at 10%), so the unreliability of short events is confounded with a simple lack of information. Since all models are trained only on the original condition, the experiments measure robustness to a test-time distribution shift rather than the handling of transient sounds.

  2. The results neither show transient-specific robustness nor attribute the gains to the proposed mechanism. From the original to the 10% condition, SR drops almost equally for ENMuS^3 (32.14) and TraSNav (31.47), and the gap stays at about 3 to 5 points across conditions. RTEM, the module that targets the stated problem, adds less than 2 SR points on its own, and its sub-variant differences are within seed variance. TSAMF, which provides most of the gain, also helps without transients and is therefore not transient-specific. Because the integration of b_t in the RTEM-only variant is unspecified, the two contributions cannot be separated.

Minor:

  • notations are inconsistent. (Eq. 2, S_t/C_t, Fig. 2)
  • the supplementary README from baseline and hardcoded paths containing user names should be cleaned up for anonymity.
Questions for authors

Q1. What evidence supports the claim that transient target sounds are common in real environments, and why are truncated sustained sounds representative of them?

Q2. How is b_t integrated into memory in the RTEM-only variant? The supplementary code suggests that only the slots and recent memory are passed to MSMT. If so, how should the TSAMF contribution in Tables 3 and 4 be interpreted?

Q3. Eq. 8 defines the event gate input as [h_t, b_t, |b_t − h_t|], but the code appears to use additional terms and gate drop-path. Which configuration produced the reported results?

Q4. Were training budgets and settings matched for MAGNet and SAVi? What are the parameter counts and inference costs of each method?

Q5. How do the methods perform with inherently short sound categories as targets, and in an experiment that fixes total active time while splitting it into more events?

Q6. Does the gap to the baselines persist when transient conditions are included in training?

Q7. Could you report Candidate + TSAMF, a simple b_t + TSAMF, and TSAMF without λ_t?

Q8. Could you report multi-seed results for the main baselines and ablations, and a comparison with a capacity-matched ENMuS³?

I am willing to raise my rating given convincing answers on the validity of the problem formulation.

6 comments

We sincerely thank you for your careful evaluation of our work and your positive assessment of the performance improvements achieved by TraSNav under different target-sound duration conditions, as well as the implementation details provided. We particularly appreciate your insightful comments on the problem formulation and experimental design, which have helped us further improve the explanations of our methodology and strengthen our experimental validation.

In response to the major concerns you raised, we have conducted additional experiments, analyses, and clarifications of implementation details. Specifically, regarding the prevalence of naturally transient sounds and the limitations of the original controlled evaluation protocol, we provide evidence from real-world acoustic studies and construct an independent Natural Transient-Sound Test Set. Regarding the integration of the event representation in RTEM-only, we further clarify how (b_t) is organized and utilized as an explicit event-memory slot. Regarding the implementation of the Event Gate, we clarify its actual input configuration and the behavior of Gate Drop-Path during training and testing. In addition, we conduct a test-time ablation of the temporal decay factor in TSAMF, supplementary evaluations under single-source and noisy scenarios, and an event-fragmentation experiment with a fixed total active time. We also report additional multi-seed results for RTEM-only and the complete parameter counts of different methods, while further multi-seed experiments for other baselines are currently ongoing.

We hope that these additional experiments and clarifications provide a better understanding of the problem formulation of TraSNav, the roles and applicability of its individual components, and further support the effectiveness and experimental reliability of the proposed method.

Q1. What evidence supports the claim that transient target sounds are common in real environments, and why are truncated sustained sounds representative of them?

We thank you for raising this important question. We agree that it is necessary to provide evidence of naturally occurring transient sounds in real-world environments and to clarify the differences between artificially shortened sustained sounds and naturally transient sounds.

First, previous environmental acoustics studies have shown that naturally transient sounds are an important component of real-world soundscapes. For example, Alías and Socoró (2017) [1] analyzed the occurrence, duration, and acoustic salience of various anomalous sound events based on recordings from real urban and suburban environments. Their study covered sound events with different temporal characteristics, including vehicle braking, horns, dog barking, and door closing. The results revealed substantial variations in event duration in real-world environments: some impulsive sounds exhibit extremely short transient characteristics, whereas sounds such as sirens and music may persist for considerably longer periods. Abeßer et al. (2025) [2] further investigated the recurrence patterns of transient sounds across different acoustic scenes using real-world recordings, identifying transient sound events with distinct temporal patterns in environments such as parks and street traffic. These studies demonstrate that naturally occurring transient sounds and their intermittent temporal structures are indeed present in real-world acoustic environments.

In our original experiments, we adopted a controlled target-sound duration-shortening protocol. Specifically, while preserving the number and onset times of target events, their source locations, and the interfering-sound schedules, we shortened only the active duration of each target-sound event and replaced the removed portions with silence. This design aims to isolate target-sound temporal availability as a controlled factor and systematically investigate how the reduction of acoustic evidence provided by individual target events affects navigation performance.

We also acknowledge that artificially truncating sustained sounds may differ from naturally transient sounds in terms of their natural onset and decay envelopes, spectral variations, and reverberation characteristics. Therefore, the original controlled experiments are not intended to fully reproduce all the physical acoustic characteristics of naturally transient sounds, but rather to establish controlled experimental conditions with varying temporal availability.

To further address this limitation, we constructed an independent Natural Transient-Sound Test Set based on the original ENMuS³ test episodes. This test set retains the original episode scenes, navigation configurations, target-source locations, and interfering-sound schedules. We replace the original target events with recordings of naturally occurring transient sounds while preserving the original number and onset times of target events.

After filtering based on recording quality and category matching, we constructed a natural transient-sound test set containing 7,769 episodes, accounting for 78.69% of the original 9,873 test episodes and covering 10 target-sound categories. The dataset uses 16 real Freesound recordings with identifiable sources and licensing information, from which natural sound segments are extracted and assigned to 44,016 original target events.

Based on a 10-ms RMS activity detection rule, the mean and median effective active durations of the natural transient events are 0.972 s and 0.620 s, respectively, with P25, P75, and P90 values of 0.310 s, 1.640 s, and 2.080 s.

Through these additions, we investigate navigation under short-duration target-sound conditions from two complementary perspectives: the original controlled duration-shortening experiments isolate the effect of limited acoustic evidence availability, while the newly constructed natural transient-sound test set further evaluates navigation performance using recordings of naturally occurring transient sounds.

References

[1] ALÍAS F, SOCORÓ J C. Description of anomalous noise events for reliable dynamic traffic noise mapping in real-life urban and suburban soundscapes[J]. Applied Sciences, 2017, 7(2): 146. DOI: 10.3390/app7020146.

[2] ABEßER J, LIANG Z, SEEBER B. Sound recurrence analysis for acoustic scene classification[J]. EURASIP Journal on Audio, Speech, and Music Processing, 2025, 2025(1): 1-15. DOI: 10.1186/s13636-024-00390-2.

Q2. How is btb_t integrated into memory in the RTEM-only variant? The supplementary code suggests that only the slots and recent memory are passed to MSMT. If so, how should the TSAMF contribution in Tables 3 and 4 be interpreted?

We thank you for raising this concern regarding how the event representation is integrated into memory in the RTEM-only variant. We would like to clarify one key point: RTEM-only does not discard btb_t; instead, it uses btb_t as an explicit event-memory slot that is passed to MSMT.

Specifically, RTEM generates the final event representation btb_t from the current observation, which, together with the conflict-calibrated target representation, supporting evidence, and conflicting evidence, forms the event-memory slots. In the RTEM-only configuration, the memory representation actually passed to MSMT is:

[b_t,\widetilde r_t,S_t,C_t,M_t^r].$$ where $M_t^r$ denotes the recent memory window consisting of the most recent 28 historical tokens. All four event-related memory slots are assigned valid masks and concatenated with the recent historical memory before being passed to MSMT. Thus, $b_t$, as the first explicit event-memory slot, directly participates in subsequent cross-attention, navigation-state encoding, and action prediction. In the full TraSNav, the event representation $b_t$ is instead adaptively fused with the current observation representation $h_t$ through TSAMF: $$h_t+g_t(b_t-h_t),$$ where $g_t=g_t^e\lambda_t$ jointly incorporates the learned event gate and the temporal-scope modulation factor. Subsequently, the fused current observation representation is updated into the full temporal scene memory: $$M_t'=[h_{a_t},\ldots,h_{t-1},h_t^{\mathrm{ref}}].$$ Therefore, the distinction between RTEM-only and the full TraSNav does not lie in whether $b_t$ is used, but rather in how event information is organized and fused before being passed to MSMT. RTEM-only uses the complete RTEM output as explicit event-memory slots, which are jointly processed with recent historical memory for subsequent decision-making. In contrast, the full TraSNav further employs TSAMF to adaptively fuse the event representation into the current observation representation while retaining the full temporal scene-memory structure. Therefore, the comparison between RTEM-only and the full TraSNav in Tables 3 and 4 reflects the overall benefit of introducing TSAMF on top of the complete RTEM, including its temporal-scope adaptive fusion mechanism and the corresponding memory organization. RTEM is responsible for constructing reliable target-event representations from multi-source acoustic observations and historical memory, whereas TSAMF further regulates the fusion of these event representations with long-term navigation context. The two components are functionally interdependent and complementary, jointly forming the complete event-aware and temporally adaptive memory mechanism of TraSNav.
Reviewer_nszkOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Paper presents an audio-visual navigation system that outperforms baselines and is specifically tailored for brief and intermittent target sounds and the presence of interfering sound sources. The work follows ENMuS^3 temporal-level memory design, the dataset and training split, but adds two new components, RTEM and TSAMF, both of which are shown in the ablations to contribute to the performance.

Critical strengths

The paper is generally easy to follow (with some exceptions with notations in the technical section). The results show improvements with respect to recent baselines and the ablations are convincing enough. The availability of the code is also a nice contribution to the community.

Critical weaknesses

The paper often feels repetitive - for example, the Related Work section does not give any more information with respect to the Introduction section; same works are cited and explained, with Introduction being just a little bit more condensed.

Notation in Method section is not easy to follow - some examples: target-hypothesis queries, supporting-cue queries, and conflict-evidence queries are first referenced as q_i, s_i, c_i respectively; but later r_t is used as sound representation and C_t and S_t are used supportive and conflicting evidence.

I'm not really convinced by the story that methods is targeted at intermittent target sounds. The method is simply better overall, and, at least from the Figure 1, the decay of performance is as step or even worse than baselines when target-sound events are progressively shortened. The figure should show much bigger relative decrease in performance for other methods and much more stable curve for proposed method in order to justify the claim. Right now the method performs better than baselines at 10% level simply because is performed better from the beginning at 100%; percentage-wise I would say that proposed method lost more going from 100% to 10% than the baselines.

Questions for authors

Why is ENMuS^3 (best performing baseline) not shown in Fig 1?

How does the model distinguish target from interfering sounds?

What forces the supporting and conflicting evidence queries to actually retrieve what they are claimed to retrieve? The ablations show that both help the performance but there is nothing that shows that they actually perform claimed tasks (inductive bias in conflicting evidence is the only thing I can see as trying to justify the name)

4 comments

We sincerely thank you for your thoughtful and constructive evaluation of our work. We particularly appreciate your positive assessment of the presentation, research value, experimental results, and code availability. We are also grateful for your recognition that both RTEM and TSAMF contribute to the overall performance of TraSNav.

In response to your concerns, we have further examined our methodological presentation and experimental results. In the revised manuscript, we will refine the organization of the Introduction and Related Work, clarify the notation and definitions in the Method section, and incorporate the normalized success-rate retention analysis to provide a more direct comparison of relative performance degradation across methods with different initial performance levels. We will also further clarify the target-conditioned nature of the navigation task and the functional roles of the supporting- and conflicting-evidence retrieval branches in RTEM, together with the corresponding attention analysis in the supplementary material.

We believe these clarifications and additional analyses help better explain the motivation, design, and effectiveness of TraSNav, particularly its robustness to increasingly brief and sparse target-sound events.

Q1. Why is ENMuS^3 (best performing baseline) not shown in Fig 1?

We thank you for pointing out the incomplete presentation of the comparison methods in Figure 1.

Figure 1 is primarily intended to illustrate the progression of target-sound conditions in audio-visual navigation, from continuously available sounds to intermittent sounds and brief, sparse events, as well as the corresponding navigation performance trends under different target-sound durations. AV-Nav and MAGNet were selected as representative methods for continuously available and intermittently available target-sound settings, respectively, to illustrate the evolution of different sound-availability paradigms.

We acknowledge that the omission of ENMuS³, the strongest baseline in our experiments, from Figure 1 may limit the reader's ability to directly compare the performance trends of our method with those of the strongest baseline.

Therefore, we will add the ENMuS³ performance curve to Figure 1(d) in the revised manuscript. The corresponding data are directly taken from the existing results reported in Tables 1 and 2, without requiring any additional experiments. The complete quantitative results of ENMuS³ under both the original and different target-sound duration conditions remain reported in the corresponding tables.

Q2. How does the model distinguish target from interfering sounds?

We thank you for raising this question. TraSNav uses the target-sound category and its corresponding GoalDescriptor as conditioning information, together with current binaural audio observations and historical scene memory, to extract acoustic evidence relevant to the specified target. Specifically, RTEM distinguishes and verifies target events through three interconnected retrieval processes.

First, the target-hypothesis queries retrieve multiple candidate target-event representations from the full historical scene memory to form an initial sound-event representation rtr_t. Second, the supporting-evidence branch retrieves recent evidence relevant to the current target hypothesis from the most recent 28 historical tokens, providing temporal consistency support for the target representation. Meanwhile, the conflicting-evidence branch incorporates discrepancy information between the current event representation and the initial event features to retrieve potentially inconsistent evidence from the full historical memory.

Based on these retrieved representations, RTEM further employs the conflict projection calibration in Eq. (5) to suppress components of the target representation that align with the conflicting-evidence direction. It then performs adaptive weighted fusion in Eq. (6) over the calibrated target representation, supporting evidence, current observation, and negatively weighted conflicting evidence, ultimately producing a reliable target-event representation btb_t.

Therefore, TraSNav primarily extracts target-related information from mixed binaural observations containing competing sound sources through target-semantic conditioning, recent supporting-evidence verification, and historical conflict-evidence calibration, and uses the resulting representation for subsequent navigation decisions.

Q3. What forces the supporting and conflicting evidence queries to actually retrieve what they are claimed to retrieve? The ablations show that both help the performance but there is nothing that shows that they actually perform claimed tasks (inductive bias in conflicting evidence is the only thing I can see as trying to justify the name).

We thank you for raising this important question. We agree that observing improved navigation performance through ablation studies alone is insufficient to fully establish whether the two branches perform their intended evidence retrieval functions. Therefore, we further explain the roles of the two branches by considering the query designs in RTEM, the end-to-end optimization mechanism, and the existing attention retrieval analysis.

First, the Support and Conflict branches are not trained with explicit semantic labels for individual historical observations. Instead, they introduce different functional inductive biases through distinct query conditions, retrieval scopes, and subsequent evidence fusion mechanisms, and learn the corresponding retrieval behaviors under the shared downstream navigation objective.

Specifically:

(1) Support Branch: Recent Supporting-Evidence Retrieval.

The Support query is jointly generated from the current initial event representation, event features, and current multimodal state:

qtS=s+ϕs([rt,et,ht]).q_t^S=s+\phi_s([r_t,e_t,h_t]).

Meanwhile, its Key and Value are obtained exclusively from a local memory window consisting of the most recent 28 historical tokens:

\{h_{\max(a_t,t-N_r+1)},\ldots,h_t\}.$$ Therefore, this branch introduces an explicit bias toward recent historical retrieval, primarily extracting supporting information relevant to the current target hypothesis from historical observations around the current event, thereby providing temporally consistent evidence for the target-event representation. **(2) Conflict Branch: Historical Conflicting-Evidence Retrieval.** Unlike the Support branch, the Conflict query additionally incorporates the discrepancy between the current event representation and the initial event features: $$q_t^C= c+\phi_c([r_t,e_t,h_t,|r_t-e_t|]).$$ Meanwhile, this branch accesses the full historical scene memory rather than being restricted to a recent temporal window: $$C_t= \operatorname{TFBlock} (Q=q_t^C,K=M_t,V=M_t).$$ The inclusion of discrepancy information provides an explicit inductive bias for identifying historical evidence that is inconsistent with the current target hypothesis. Furthermore, the retrieved conflicting evidence is used to suppress components of the current target representation that align with the conflicting-evidence direction through the projection calibration in Eq. (5), and contributes negatively to the construction of the target-event representation in the final evidence fusion in Eq. (6). **(3) Analysis of Actual Attention Retrieval Behaviors.** Beyond the aforementioned architectural designs and ablation results, Figure S2 in the supplementary material already analyzes the actual retrieval behaviors of the two branches from spatial, temporal, and cross-episode perspectives. Figure S2(a) shows the spatial retrieval distributions of the two branches within the same representative episode. The Support branch focuses more on target-related historical regions, whereas the Conflict branch assigns greater attention to regions that may contain competing-source evidence. Figure S2(b) illustrates the attention distributions of the two branches across historical time steps, together with the activity intervals of target and interfering sounds, revealing their distinct spatiotemporal retrieval patterns. Figure S2(c) further reports cross-episode attention enrichment statistics over 100 test episodes. The results show that the Support branch exhibits stronger attention enrichment over target-sound-active history, whereas the Conflict branch exhibits corresponding attention enrichment over interfering-sound-active history.

For Critical Weakness 3

We thank you for raising the concern regarding the performance degradation trends under different target-sound duration conditions. We agree that comparing only the absolute success rates (SR) under different duration conditions may be affected by differences in the initial performance levels of individual methods, and therefore may not fully reflect their relative robustness to shortened target sounds.

To address this issue, we re-examined the experimental results in Figure 1 and Tables 1 and 2, and further calculated the normalized SR retention of each method relative to its own performance under the original sound condition, enabling a fairer comparison of relative performance degradation across methods with different initial performance levels.

We define the relative SR retention as the ratio of the SR under the current target-sound duration condition to the SR under the original sound condition:

R(\rho)=\frac{SR(\rho)}{SR(100\\%)}\times 100\\%

where ρ\rho denotes the target-sound duration ratio and SR(100SR(100\\%) denotes the success rate under the original sound condition. This metric measures the proportion of each method's original success rate retained as the target-sound duration is shortened.

The complete normalized retention results are presented below.

Table R1. Normalized SR retention under different target-sound duration conditions (%).

Method90%80%70%60%50%40%30%20%10%
AV-Nav94.0187.1881.8276.4068.9459.7250.0740.0129.53
SAVi94.8889.8285.5674.6567.8859.6849.0137.0128.26
SMT+Audio94.8788.6779.1973.8867.0558.4748.5540.9629.10
ENMuS³96.3490.8885.2979.3772.4562.5153.0240.6230.07
MAGNet94.7489.8685.5875.9970.2560.9749.6640.0626.59
TraSNav98.1892.4289.5683.6978.3868.2959.2248.1035.70

As can be observed, TraSNav achieves the highest normalized SR retention across all nine target-sound duration conditions, indicating that its advantage is reflected not only in absolute navigation performance but also in its ability to retain performance relative to its own original condition.

In particular, under the most challenging 10% condition, TraSNav achieves a normalized SR retention of 35.70%, compared with 30.07% for the strongest baseline, ENMuS³, yielding a difference of 5.63 percentage points. In other words, although both methods experience performance degradation as the target-sound duration is shortened, TraSNav retains a higher proportion of its original navigation success rate, exhibiting a lower relative performance degradation than ENMuS³.

Furthermore, this advantage is not limited to the extreme 10% condition but remains consistent across all nine duration conditions from 90% to 10%. These results further indicate that the performance advantage of TraSNav does not merely stem from its higher initial success rate under the original sound condition; it also demonstrates better relative robustness as target sounds become progressively shorter and the available acoustic evidence becomes increasingly limited.

Reviewer_WoiAOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper looks at audio-visual navigation when the target sounds are short and intermittent. The setting also includes competing sound sources. TraSNav has two main modules. Reliable Transient Event Memory is great at getting hold of target hypotheses, evidence that supports them, and evidence that doesn't. Temporal-Scope Adaptive Memory Fusion is what controls how this information ends up in the current memory representation.

The evaluation shortens target events while keeping their starts and other settings the same. All methods are trained under the original sound condition. Each method uses a fixed checkpoint for evaluation.

Critical strengths

1.I think the evaluation is well controlled. The authors keep the start times of events, the number of events, the schedules for things getting in the way, and the settings for each episode the same for all the different times it takes place. This makes it easier to judge the effect of shorter target sounds. Appendix A.2 also explains how to select checkpoints based on validation SPL. Each method uses the same checkpoint for all test ratios.

2.TraSNav makes the reported SR and SPL better over time. At the 10% ratio, SR goes up from 13.82% for ENMuS³ to 17.47% for TraSNav. That's an increase of 3.65 percentage points. Table S4 also reports three independent TraSNav training runs. These results back the full model when the tested conditions are the same.

Critical weaknesses

1.I'm still not sure how much of the gain comes from conflict calibration itself. If you remove the conflict branch, you will also remove the conflict retrieval and the projection in equation 5. The ablation changes several things at the same time. The gain could come from the projection, the extra retrieval branch, or the negative conflict term used later in fusion. Figure S2 shows that the branches learn different roles. But it doesn't separate these effects. This is important because the paper's main idea is that conflict calibration is an important part of the research. 2.I also don't understand the role of temporal decay. The TSAMF ablation removes the learned event gate and the predefined decay together. The weights in Figure S3 are getting smaller, and this is partly because of equation 9. So the visualisation alone doesn't show if the decay schedule helps. 3.I think the authors need to keep the learned gate and disable decay to test the explanation about time. The paper also needs a clear definition of L_t and the explicit fusion equation for RTEM-only. I have some questions about how it works, and that's the main reason for my weak rejection.

Questions for authors

1.Have you tested a variant that keeps conflict retrieval but disables the projection calibration in Eq. (5)? This would directly test what Eq. (5) contributes.

2.What happens when the learned event gate stays active and lambda_t is fixed at 1? How exactly is L_t defined, including when memory reaches capacity? What fusion equation does RTEM-only use?

3.Are the gains from individual components consistent across independent training runs? Table S4 shows some variation across TraSNav runs. Results from multiple training runs for the main ablation controls would help assess the smaller component gains.

5 comments

We sincerely thank you for your thoughtful and constructive evaluation of our work. We particularly appreciate your recognition of our carefully controlled evaluation protocol, the consistent checkpoint selection across different target-sound duration conditions, and the improved navigation performance achieved by TraSNav, especially under the challenging 10% duration setting.

In response to your concerns, we have conducted additional ablation experiments and multi-seed evaluations to further investigate the individual contributions of RTEM and TSAMF. Specifically, we provide an additional ablation to isolate the effect of conflict projection calibration, a test-time intervention to examine the contribution of the temporal-scope decay factor while retaining the learned event gate, and additional multi-seed results for RTEM-only. We also further clarify the corresponding mechanisms and implementation details to provide a more comprehensive understanding of the proposed framework.

Q1. Have you tested a variant that keeps conflict retrieval but disables the projection calibration in Eq. (5)? This would directly test what Eq. (5) contributes.

We thank you for this important suggestion. We agree that it is necessary to further distinguish the respective roles of conflict-evidence retrieval and conflict projection calibration to more accurately assess the contribution of the mechanism introduced in Eq. (5).

To this end, we introduce a new ablation variant, Full RTEM w/o Projection Calibration. In this setting, we retain the target-candidate retrieval, recent supporting-evidence retrieval, and historical conflict-evidence retrieval, while also preserving the subsequent use of negative conflict evidence in the fusion stage. We only remove the conflict projection calibration operation in Eq. (5). All other RTEM components and experimental settings remain unchanged. As in the original Table 5, TSAMF is disabled to isolate the contribution of conflict projection calibration to RTEM.

The additional experimental results are presented below:

Table R1. Ablation of conflict projection calibration under different target-sound conditions (SR, %).

VariantOriginal80%50%20%
Candidate only46.0141.9433.5219.02
Cand. + Conflict46.1342.2134.2319.44
Full RTEM w/o Projection(New)46.1942.4634.8919.71
Full RTEM46.2842.5835.2219.98

As can be observed, when all conflict-evidence retrieval operations and the subsequent negative-conflict fusion are retained, introducing the projection calibration in Eq. (5) further improves SR by 0.09, 0.12, 0.33, and 0.27 percentage points under the four test conditions, respectively.

These results indicate that conflict-evidence retrieval and projection calibration play distinct yet complementary roles. Conflict retrieval obtains inconsistent evidence related to the current target event from historical memory, whereas projection calibration further exploits this evidence to suppress components of the current target representation that align with the conflicting direction, thereby improving the reliability of the event representation.

Notably, although the gains from projection calibration are relatively modest, they remain positive across all four target-sound duration conditions. This indicates that the operation does not merely duplicate the function of conflict-evidence retrieval, but further improves the target-event representation based on the retrieved conflicting information.

Q2. What happens when the learned event gate stays active and λt\lambda_t is fixed at 1? How exactly is LtL_t defined, including when memory reaches capacity? What fusion equation does RTEM-only use?

We thank you for raising this concern regarding the role of the temporal-scope modulation factor in TSAMF. We agree that it is necessary to further distinguish the respective roles of the learned event gate and the predefined temporal decay factor to verify the actual contribution of the temporal-scope modulation mechanism.

To this end, we introduce an additional ablation variant, TSAMF w/o Decay, to specifically examine the effect of the temporal decay factor. Specifically, we retain the learned event gate while fixing the temporal-scope modulation factor to:

λt=1.\lambda_t=1.

Accordingly, the final fusion weight becomes:

gt=gte.g_t=g_t^e.

This ablation is implemented through a test-time ablation: we directly load the trained Full TraSNav checkpoint and fix λt\lambda_t to 1 only during inference. All other network architectures, learned event gates, model weights, and evaluation configurations remain unchanged.

We evaluate the model on the original test set under four target-sound duration conditions: Original, 80%, 50%, and 20%. The results are reported below.

Table R2. Ablation of the temporal-scope modulation factor in TSAMF (SR, %).

VariantOriginal80%50%20%
RTEM-only46.2842.5835.2219.98
TSAMF w/o Decay(new)48.4844.6636.9521.95
Full TraSNav48.9445.2338.3623.54

The results show that, while retaining the learned event gate and removing only the temporal decay factor, TSAMF w/o Decay achieves SRs of 48.48%, 44.66%, 36.95%, and 21.95% under the four test conditions, respectively, outperforming RTEM-only in all cases.

Furthermore, compared with the full TraSNav under the same fixed checkpoint, removing the temporal decay factor leads to SR reductions of 0.46, 0.57, 1.41, and 1.59 percentage points, respectively. The performance gaps become more pronounced, particularly under the 50% and 20% target-sound duration conditions.

These results indicate that the learned event gate itself effectively regulates the influence of event information on the current navigation state, while the temporal-scope modulation factor provides additional performance gains. The two components serve different purposes: the event gate determines the fusion strength of event information based on the relationship between the current event representation and the navigation state, whereas the temporal decay factor further modulates this fusion strength according to the temporal context accumulated during navigation.

Regarding the definition of LtL_t, it denotes the number of valid tokens in the current temporal scene memory MtM_t. With a memory capacity of 160 tokens, including 159 historical tokens and the current observation token, LtL_t is defined as:

Lt=∣Mt∣=min⁡(t,160).L_t=|M_t|=\min(t,160).

Before the memory reaches its capacity, LtL_t increases with each newly accumulated observation. Once the capacity is reached, the memory is updated using a sliding-window strategy: the oldest token is removed and the current observation is appended, keeping Lt=160L_t=160. Consequently, the temporal-scope factor λt\lambda_t remains at the value determined by the maximum memory capacity rather than continuing to decrease.

For the RTEM-only variant, the output of RTEM, btb_t, is not discarded. Instead of applying the residual fusion in Eq. (10), RTEM-only explicitly organizes the event representations and recent historical observations into memory slots:

[bt,r~t,St,Ct,Mtr],[b_t,\widetilde r_t,S_t,C_t,M_t^r],

where MtrM_t^r denotes the recent memory window containing up to 28 valid tokens. The four event-related slots are assigned valid masks and are concatenated with the recent memory before being passed to MSMT. Thus, btb_t serves as the first explicit event-memory token and directly participates in subsequent cross-attention and action prediction.

This formulation clarifies that RTEM-only retains the complete RTEM and utilizes its final event representation, while disabling the temporal-scope adaptive residual fusion introduced by TSAMF.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.