acceptodds
Under review as a conference paper at ICLR 2027

RecallWAM: Unifying Recall and Foresight for Object Navigation

Abstract

Object navigation requires agents to find specified objects in unfamiliar environments through exploration, using past observations to avoid redundant search. World-action models (WAMs) exploit such visual experience by coupling action learning with future-observation prediction (Foresight), providing dense supervision for navigation. However, identical scene appearances can correspond to different exploration states and call for different search decisions. We therefore extend world-action learning to recover history-dependent exploration state alongside predicting future observations, making past coverage explicit for navigation decisions. To this end, we introduce RecallWAM, which augments foresight-based world-action learning with a recall capability that recovers exploration state from historical observations to guide navigation decisions. To teach the policy how recalled coverage should affect its actions, we construct balanced exploration trajectories that trade off shortest-path guidance with randomized branch exploration, providing history-dependent search experience while limiting unnecessary detours. Experiments on the benchmarks HM3D, MP3D, and OVON demonstrate state-of-the-art performance. Controlled ablations show that joint Recall and Foresight learning yields gains exceeding the sum of their individual gains.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Object Navigation, Active Exploration, World-Action Model

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 5 reviews, above 85% of the others.
Reviewer_MtJyOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes RecallWAM for object navigation, extending world-action learning with an explicit Recall capability. The model combines recent observations with spatially retrieved history to predict which traversable regions in the current view have already been explored, and directly conditions the navigation policy on this predicted state. Recall is jointly trained with future-observation prediction and action learning, while balanced exploration trajectories are introduced to provide history-dependent search experience. The method achieves strong results on HM3D, MP3D, and OVON, with ablations supporting the contributions of the exploration-state input, memory design, and Recall objectives.

Critical strengths

The main idea is simple and well motivated: visually similar observations may require different actions depending on which areas have already been explored, and representing this information explicitly is useful for avoiding repeated search. The method is also evaluated across three benchmarks, and the ablations clearly separate the effects of spatial and temporal memory, the explicit exploration-state input, and the Recall and Foresight objectives.

Critical weaknesses

The spatial retrieval procedure relies on accurate odometry and field-of-view overlap, but its robustness to pose or localization errors is not evaluated. The paper also does not report the memory and runtime overhead of storing and retrieving historical visual latents, which would be useful for understanding how the method scales to longer episodes.

Questions for authors
  1. How sensitive is spatial retrieval and exploration-state prediction to noise in the estimated camera pose?

  2. What are the memory and runtime costs of maintaining and retrieving the historical latents as the episode becomes longer?

1 comment

We thank Reviewer MtJy for recognizing the motivation behind history-dependent exploration, the effectiveness of explicit exploration-state conditioning, and our carefully controlled ablations. We address the concerns through evaluation with estimated poses and profiling of long-term memory overhead.

1. Evaluate localization robustness using estimated poses

RecallWAM maintains comparable navigation performance when ground-truth poses are replaced with RGB-based MASt3R estimates. We evaluate this change on OVON Val-Unseen using the main model jointly trained on HM3Dv2, MP3D, and OVON.

Table S1. Navigation performance with different pose sources on OVON Val-Unseen.

Pose sourceSR (%) ↑SPL (%) ↑
Simulator ground-truth62.336.9
MASt3R-estimated61.636.8

Performance decreases by only 0.7 SR points and 0.1 SPL points, indicating that Recall-guided navigation remains effective when spatial retrieval uses estimated camera poses rather than perfect simulator poses.

2. Quantify memory and retrieval overhead over longer episodes

Historical cache usage and retrieval latency both scale approximately linearly with episode length, while the policy maintains a fixed-size context of four spatially retrieved and four recent observations. We profile both costs on a single NVIDIA RTX 5090.

Table S2. GPU history-cache usage and retrieval latency at different episode lengths.

Episode stepsGPU cache (MiB) ↓Retrieval (ms) ↓
10064.3410.52
200162.8429.59
300216.1136.78
400283.0848.19
500333.6056.65

Episodes average 153 steps in our evaluation. Even at 500 steps, the maximum episode length allowed by the benchmark, the historical cache occupies only 333.60 MiB, with 56.65 ms retrieval latency. The policy input remains fixed in size regardless of episode length.

The current retrieval implementation evaluates candidate views sequentially, resulting in approximately linear scaling with history length. Further acceleration is possible by batching or parallelizing candidate-view projection, coverage computation, and scoring.

Reviewer_ZAicOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper argues that world-action models predict future observations but never represent coverage, and that the same view can demand different decisions depending on prior search. RecallWAM adds a Recall branch that reconstructs a current-view exploration mask from recent and field-of-view-retrieved latents and feeds it to the action policy, jointly trained with foresight flow matching, action imitation, and balanced exploration trajectories. A single model trained on HM3D, MP3D, and OVON reports 77.3/41.8 SR/SPL on HM3D, 59.3/26.0 on MP3D, and 62.3/36.9 on OVON Val-Unseen, with exploration-mask IoU rising from 62.6 to 76.1 as the Recall losses are added. With ground-truth target masks, the OVON model reaches 82.5/55.1. The contribution is to make exploration state a learned prediction target and an explicit policy input rather than an implicit by-product of visual history. The failure analysis and the ground-truth-mask experiment indicate that target perception, not search-budget exhaustion, dominates the remaining errors.

Critical strengths
  1. Turning prior coverage into a state that is both predicted and supplied to the policy is a clear, well-motivated extension of foresight-based world-action learning.
  2. The ablation grid is unusually clean, with history composition, latent budget, explicit state input, and each Recall loss separated, so supervision is not conflated with usable state.
  3. Results are consistent across three benchmarks with a single model, and the paper is candid that much of the gap is perceptual rather than exploratory.
Critical weaknesses
  1. The SOTA claim conflates perception with policy, since RecallWAM uses frozen SAM3 masks while most baselines use weaker detectors and Table 5 shows that mask quality alone moves OVON SR by 25 points.
  2. Because no baseline is retrained under the same data, backbone, or perception budget and Fast-WAM, the extended model, is absent from Table 1, the reported margins cannot be attributed to Recall.
  3. Super-additivity rests on a single run without seeds or significance tests, and since Foresight alone changes SR by only -0.20, the 5.04 joint gain versus the 2.27 sum of individual gains on OVON is within noise and is never repeated for the main model.
  4. The paper motivates the method through exploration, yet only 5.6% of failures are exploration failures, with stopping and perception dominating instead, so the framing overstates the bottleneck.
Questions for authors

See the weakness.

1 comment

We thank Reviewer ZAic for recognizing Recall's formulation as both a prediction target and policy input, the carefully controlled ablations, and the consistent results across three benchmarks. We address the four concerns below.

1. Clarify the role of SAM3 and isolate Recall's contribution

Recall's gains cannot be attributed to SAM3 alone. Multiple methods in Table 1, such as DRIVE-Nav and OpenFrontier, also employ SAM3 for target perception. Moreover, the 25.33-point SR gap in Table 5 measures the improvement from replacing SAM3 with oracle target masks within RecallWAM, rather than SAM3's advantage over competing systems.

More importantly, Table 2(c) directly isolates exploration-state conditioning under identical perception. Both variants use the same frozen SAM3 detector, training data, backbone, and Recall supervision. Adding the exploration-state input improves SR/SPL from 53.94/32.03 to 57.19/34.49, demonstrating Recall's contribution independently of perception differences.

2. Clarify the matched Fast-WAM-style baseline and isolate Recall

Table 3(a) already provides a matched Fast-WAM-style baseline through the Foresight + Action configuration without Recall. It retains future-observation supervision during training and direct action prediction at inference, following Fast-WAM's formulation.

Both configurations share the same training data, backbone, frozen SAM3 detector, and training budget. Adding Recall improves SR/SPL from 51.95/31.84 to 57.19/34.49, gains of 5.24/2.65 points. This directly isolates Recall's additional contribution to the WAM, rather than relying on differences between independently developed systems.

We will explicitly identify this configuration as a Fast-WAM-style baseline (w/o Recall) and clarify its relationship to Table 1's full-system comparisons.

3. Add multi-seed evaluation of Recall–Foresight gains

Joint Recall–Foresight learning shows consistent super-additive gains across independent training seeds. We evaluate all four configurations in Table 3(a) using three independent training seeds, keeping the training data, backbone, perception model, and training budget fixed.

Table S1. Recall–Foresight ablation on OVON Val-Unseen (mean ± sample standard deviation across three seeds).

RecallForesightSR (%) ↑SPL (%) ↑
××52.15 ± 0.3831.15 ± 0.34
×✓52.08 ± 0.1331.86 ± 0.19
✓×54.63 ± 0.4232.21 ± 0.27
✓✓57.28 ± 0.5634.53 ± 0.49

We calculate the interaction as I=SRF−SR−SF+S0I = S_{RF} - S_R - S_F + S_0, measuring the joint gain beyond the sum of individual improvements. The mean interaction is +2.72 SR points and +1.61 SPL points, consistent with the super-additive benefit reported in Table 3(a).

4. Clarify the purpose of failure analysis

Figure 5 analyzes the remaining failures of the final RecallWAM system to identify directions for future improvement, rather than establishing exploration as the dominant failure mode. The analysis highlights perception and stopping as major remaining challenges, providing guidance for future improvements.

Reviewer_9aYROpenReview ↗
Rating
4
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

RecallWAM proposes unifying past observations into decision-making in object navigation tasks alongside foresight, to avoid redundant search. Instead of working on visual memory, which is the mainstream of research, authors proposed learning an exploration state, or exploration mask, to summarize previous observations. This recall is leveraged in generating the action tokens. Authors also proposed a data collection method for balanced exploration trajectories to train the mapping from exploration state to action. Experimental results on HM3D, MP3D and OVON show that RecallWAM outperforms SOTA object navigation models.

Critical strengths
  1. The research philosophy is correct. Leveraging history is a major research concern for object navigation tasks and more broadly, to WAM research. Among the literature of visual memory, the authors further identify a specific exploration state as a summary, which is novel.
  2. The method is impressive and well-designed. The methodology of reconstructing the exploration mask is elegant, the loss functions makes sense intuitively and experimentally, and the design in balanced data collection is also worth noting.
  3. Good experimental results. Although improvements over second bests are small, but it should be noted that second bests come from different models while the paper achieves SOTA on all metrics in Table 1.
  4. Paper is well-written and illustrative. Though it contains solid ideas, the method section is easy to read. Figures simple and informative.
Critical weaknesses
  1. Discussion: Is the “Balanced Exploration” inside Sec3.4 the optimal way of collecting data? It is fully generated by programs, and may have a large distribution gap with model policies. Have you tried distilling some trajectories from human or large models?
  2. Does it work for small object navigation? The benchmarks seem to focus on large objects, which is not challenging.
Questions for authors

Please answer the question in weaknesses.

1 comment

We appreciate Reviewer 9aYR's insightful comments, which raise two important questions: how scalable programmatic supervision compares with more adaptive demonstrations, and whether RecallWAM generalizes to small-object navigation. These suggestions help clarify our design trade-offs and strengthen the scope of our evaluation.

1. Clarify the design of Balanced Exploration and discuss alternative demonstrations

Balanced Exploration is not claimed to be globally optimal, but achieves the best performance among our tested configurations. It combines shortest-path guidance with randomized branch exploration to provide diverse search and backtracking experience while maintaining path efficiency. As shown in Table 3(c), β = 12 achieves the highest SR (57.19%) and SPL (34.49%) among the tested settings.

Human- and large-model-generated demonstrations could leverage scene-semantic priors, such as searching kitchens first when looking for a knife, to guide more efficient exploration. Although we have not yet explored these alternatives, Balanced Exploration addresses a complementary challenge: generating diverse, history-dependent exploration experience at scale, without expensive human annotations, large-model inference, or iterative DAgger-style expert interventions. Combining semantic guidance with our scalable data generation framework is a promising direction for future work.

2. Add small-object navigation results

RecallWAM achieves 62.44% SR on small-object categories in OVON, including drinking glasses and books, using the same model as Table 1. This demonstrates its ability to navigate to small targets, beyond the larger objects emphasized in typical ObjectNav benchmarks.

We will add a detailed breakdown of navigation performance across small, medium, and large target objects in the appendix.

Reviewer_i7GZOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes RecallWAM, a world-action model for ObjectNav that jointly learns exploration-state recovery (Recall), future visual prediction (Foresight), and action prediction. Specifically, Recall predicts explored/unexplored regions in the current view using spatially retrieved and temporally recent visual observations, and explicitly conditions the navigation policy on the predicted exploration state. The authors also introduce balanced exploration trajectories that combine shortest-path guidance with randomized exploration to provide history-dependent training demonstrations.

Experiments on HM3D, MP3D, and OVON simulation benchmarks report competitive navigation performance. Ablation studies investigate the contributions of Recall, Foresight, memory configurations, and exploration trajectory generation.

Critical strengths
  1. Identifying a limitation of foresight-based WAMs for object navigation.

The paper highlights a relevant limitation of existing foresight-based WAMs when applied to ObjectNav, where effective exploration requires reasoning about previously visited regions rather than solely predicting future observations. The proposed Recall mechanism addresses this limitation by explicitly recovering exploration states from visual history and incorporating them into action prediction, providing a task-oriented extension of the WAM framework.

  1. Encouraging empirical evidence for the effectiveness of Recall.

The ablation studies provide evidence that incorporating Recall improves navigation performance. In particular, Table 3(a) shows that adding Recall to Foresight–Action learning improves SR from 51.95% to 57.19%. The additional ablation in Table 2(c) further demonstrates the benefit of explicitly conditioning the policy on Recall-predicted representations. These results provide meaningful empirical support for the effectiveness of the proposed integration.

Critical weaknesses
  1. Limited methodological novelty and insufficient justification for the proposed Recall mechanism.

The paper presents its central novelty as endowing WAM with a Recall capability that recovers exploration states from historical observations to guide navigation decisions. However, this functionality is already well established in conventional map-based navigation, which can maintain richer geometric and semantic information.

More importantly, given that RecallWAM already relies on odometry for spatial retrieval, constructing an explicit map from the same RGB observations appears to be a practical alternative. For example, existing off-the-shelf reconstruction models such as LingBot-Map and MapAnything can recover 3D geometry and camera poses from RGB observations. These representations could be used to maintain explored regions and provide map-derived coverage information to the same navigation policy, potentially without requiring a separately trained Recall mechanism.

While such alternatives would introduce their own computational costs and reconstruction errors, the paper does not investigate this trade-off or demonstrate what advantages the proposed learned Recall offers over explicit spatial memory. Consequently, the existing ablations establish the benefits of exploration-state conditioning within the proposed WAM, but do not sufficiently justify the necessity or distinct contribution of its learned Recall formulation.

  1. Incomplete empirical comparisons and insufficient validation of practical robustness.

The claimed state-of-the-art performance is not adequately supported by the reported comparisons. Several relevant prior works are omitted, including Hydra-Nav (ICML 2026), which reports higher SR and SPL on both MP3D and OVON Val-Unseen using RGB observations without depth, albeit with additional panoramic scans and a different training procedure. VLingNav also reports a higher HM3D-v2 success rate (83.0% vs. 77.3%) using egocentric RGB video, while AstraNav-Memory (CVPR 2026), which similarly uses RGB observations and camera-pose information, reports competitive performance on OVON Val-Unseen. Although these methods differ in their observation protocols and training strategies, their omission makes the paper's unqualified SOTA claim insufficiently supported.

Furthermore, RecallWAM relies on odometry to retrieve historical observations according to camera field-of-view overlap, but the source and accuracy of this odometry are insufficiently specified. If the evaluation uses noiseless simulator poses, the robustness of the proposed memory mechanism under realistic localization errors remains untested.

Questions for authors
  1. Advantages over explicit spatial memory.

Given that RecallWAM already relies on odometry for spatial retrieval, what specific advantages does the learned Recall mechanism offer over conventional map-based representations? For example, off-the-shelf RGB-based reconstruction models such as LingBot-Map or MapAnything could potentially construct an explicit spatial map from historical observations, from which explored regions could be derived and supplied to the same navigation policy. Could you compare against such a map-conditioned baseline using the same RGB and odometry inputs, while accounting for reconstruction accuracy and computational overhead? This would help clarify whether the benefits arise from explicit exploration-state conditioning or specifically from the proposed WAM-based Recall mechanism.

  1. Incomplete state-of-the-art comparisons.

Several relevant prior methods, including Hydra-Nav, AstraNav-Memory, and VLingNav, are omitted from Table 1 despite reporting competitive or stronger performance on some of the evaluated benchmarks. Could you clarify these omissions and update the comparisons accordingly? Given potential differences in observation modalities and evaluation protocols, the claimed state-of-the-art performance should be substantiated under clearly defined, comparable settings, or the claim should be qualified appropriately.

  1. Reliance on localization accuracy.

Does RecallWAM rely on ground-truth simulator poses at inference? If so, how does its performance change under noisy odometry, and what evidence supports its applicability to real-world navigation?

4 comments

We thank Reviewer i7GZ for recognizing the motivation behind Recall and the empirical evidence supporting its effectiveness. We address the concerns through an explicit-map comparison, clarification of prior-work settings, and evaluation with estimated poses.

1. Add a controlled comparison with explicit spatial memory

We thank the reviewer for suggesting this comparison to clarify the advantages and trade-offs of learned Recall relative to explicit spatial memory. Learned Recall outperforms explicit map-derived exploration state in both prediction accuracy and navigation performance. We construct a MapAnything-based baseline that reconstructs scene geometry and derives exploration state from the same RGB observations and odometry. Both methods use identical demonstrations, SAM3 detector, action policy architecture, and training budget, with each policy trained on its corresponding exploration-state estimate.

Table S1. Explicit map vs. Recall. IoU: Table 3(b) evaluation set; SR/SPL: OVON Val-Unseen.

MethodIoU (%) ↑SR (%) ↑SPL (%) ↑
Explicit Map (MapAnything)52.8453.6232.48
RecallWAM (Ours)76.1057.1934.49

Recall improves IoU by 23.26 points, SR by 3.57 points, and SPL by 2.01 points. Unlike explicit mapping, which relies on intermediate geometry reconstruction and visibility estimation, Recall learns exploration state directly from visual history and jointly optimizes it with action prediction.

Computational overhead. On a single RTX 5090, the MapAnything-based reconstruction and coverage-estimation pipeline takes 2036.65 ms per navigation step. Recall recovers exploration state within the shared WAM without requiring this separate reconstruction stage.

2. Explain the omitted comparisons and clarify evaluation settings

We initially omitted these methods because their reported results could not be verified under directly comparable evaluation settings. As illustrated below, important test-time implementation details are not always evident from paper descriptions alone.

AstraNav-Memory. Its released evaluation pipeline uses privileged simulator navigation geometry, specifically a navmesh-based oracle follower to execute predicted destinations, as documented in SuperNav, Appendix B.5. RecallWAM does not use simulator navigation geometry for action execution. This difference makes a direct comparison under identical information constraints inappropriate.

Hydra-Nav and VLingNav. Their implementations and checkpoints were unavailable, and our reproduction attempts did not recover the reported performance. Some evaluation details also remain unclear. For example, Hydra-Nav does not specify how its four-view panoramic scans count toward the episode-step budget for SR/SPL evaluation, preventing us from verifying protocol equivalence.

Updated comparisons. We will add their paper-reported results to Table 1, annotate known protocol differences and unresolved details, and qualify our SOTA claims accordingly. Recall's contribution is further supported by the controlled ablations and the explicit-map comparison above.

3. Clarify pose inputs and evaluate navigation with estimated poses

RecallWAM maintains comparable navigation performance when simulator ground-truth poses are replaced with RGB-based MASt3R estimates. We evaluate this change on OVON Val-Unseen using the main model jointly trained on HM3Dv2, MP3D, and OVON.

Table S2. Effect of pose estimation on OVON Val-Unseen.

Pose sourceSR (%) ↑SPL (%) ↑
Simulator ground-truth62.336.9
MASt3R-estimated61.636.8

Performance decreases by only 0.7 SR points and 0.1 SPL points, demonstrating robustness to visually estimated poses in simulation. This supports the feasibility of RGB-based localization for Recall.

Thank you for the detailed responses and additional experiments. I appreciate the clarification of the benchmark comparisons, the evaluation with estimated poses, and the additional empirical evidence supporting the effectiveness of Recall.

I would appreciate further clarification regarding the architecture and training objectives of the explicit spatial memory baseline.

1. The baseline appears to replace the predicted exploration state with a map-derived exploration state. However, my original concern is that an explicit geometric or preferably semantic map, which could be constructed by combining MapAnything with the existing SAM3 detector, can itself serve as a rich spatial memory representation, retaining information beyond the current field of view without requiring an additional exploration-mask prediction or estimation mechanism. Directly conditioning the policy on such a representation would more closely address the comparison between learned Recall and conventional explicit spatial memory. Could you clarify the exact representation supplied to the policy and whether Recall supervision is removed from the shared WAM backbone in this baseline? Visualizations of the reconstructed maps would also be helpful in understanding what information is retained and how it evolves throughout exploration.

2. The substantial IoU gap between MapAnything and learned Recall (52.84% vs. 76.10%) is somewhat surprising, given that exploration states can be geometrically derived from reconstructed scene geometry and ground-truth camera poses. Could you explain what causes such a large performance gap? Does it primarily originate from inaccurate geometry reconstruction or from the subsequent exploration-state estimation procedure? Visualizations of the reconstructed map, map-derived exploration mask, Recall prediction, and ground-truth mask for the same observations would be helpful in understanding this discrepancy.

Thank you for the follow-up. We would like to point out that our initial MapAnything baseline followed the comparison described in your initial review: "construct an explicit spatial map from historical observations, from which explored regions could be derived and supplied to the same navigation policy." We have now extended this comparison to directly using the accumulated map as spatial memory, as raised in your follow-up.

1. Add a direct explicit spatial memory baseline

RecallWAM continues to outperform explicit spatial memory when the accumulated map directly conditions the navigation policy. We additionally trained a baseline that integrates MapAnything geometry and SAM3 target evidence into a continuously updated BEV map, encoded as spatial tokens for action prediction. Both map-based baselines remove Recall prediction and supervision while retaining visual history, Foresight supervision, and action learning.

Table S3. Explicit spatial memory comparison on OVON Val-Unseen.

MethodSR (%) ↑SPL (%) ↑
Map-derived Exploration State53.6232.48
Direct Spatial Memory55.0331.44
RecallWAM (Ours)57.1934.49

Direct spatial memory improves SR over the map-derived baseline but yields lower SPL. RecallWAM outperforms direct spatial memory by 2.16 SR points and 3.05 SPL points. These results support the advantage of learning a task-relevant exploration state directly from visual history rather than relying on explicit geometry reconstruction.

2. Identify the source of the IoU gap using ground-truth depth

The IoU gap primarily stems from geometry reconstruction errors rather than the subsequent exploration-state estimation procedure. To verify this, we replaced MapAnything's reconstructed geometry with simulator ground-truth depth, keeping the exploration-state estimation procedure unchanged. IoU increases from 52.84% to 87.27%, confirming that reconstruction errors are the main source of the gap.

This highlights a key advantage of Recall: learning exploration states directly from visual history without relying on an intermediate geometry reconstruction stage.

We will add visualizations of the accumulated spatial maps, map-derived exploration states, Recall predictions, and ground-truth masks to the revised appendix.

Thank you for the additional experiments, which strengthen the empirical support for the proposed method and provide a convincing explanation for the IoU discrepancy.

However, I remain somewhat unconvinced of the broader advantages of the proposed Recall mechanism over conventional explicit spatial memory. First, Recall requires additional task-specific engineering, including the definition and construction of exploration-state supervision and dedicated learning objectives. Second, its advantages may diminish when extending single-object navigation to more realistic settings, such as lifelong object navigation with sequentially changing goals. In such settings, previously explored regions may become relevant again, and retaining semantic and spatial information about previously encountered objects becomes particularly important. An explicit semantic map naturally supports such persistent memory reuse, whereas the proposed Recall representation explicitly captures only prior exploration coverage.

Therefore, despite the encouraging additional results, I remain concerned about whether the relatively modest performance improvements justify the task-specific formulation and its potentially limited applicability to more general navigation settings.

Thank you again for the efforts you've made to strengthen the paper and the insightful discussion.

Reviewer_esXeOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

An agent can return to the same junction after searching a side room. The view looks familiar, but the best next action may have changed. RecallWAM addresses this ambiguity by predicting which traversable regions in the current view were observed at close range earlier in the episode. The policy receives this prediction alongside visual features and a target-object mask. Future-observation prediction provides an additional training signal, while demonstrations that mix shortest-path guidance with branch exploration teach the agent when to backtrack. A single model trained on HM3D, MP3D, and OVON achieves the best reported SPL on all three main validation sets and ties the best HM3D success rate. Separate OVON-only ablations examine the contributions of Recall and Foresight.

Critical strengths
  1. In Table 2(c), both variants retain Recall supervision, the same visual history, and visual cross-attention. Giving the policy the predicted state raises SR from 53.94% to 57.19% and SPL from 32.03% to 34.49%. Figure 6 provides complementary evidence about where that state comes from. With the current image and recent observations held fixed, restoring older, spatially relevant views recovers previously explored regions in the examples shown. Together these results give the proposed memory mechanism a credible link to navigation performance.

  2. A shortest-path expert rarely needs to return from an unsuccessful search and choose a different branch. The proposed demonstrator generates such revisits while retaining a preference for efficient routes. Table 3(c) shows why this choice matters in practice. Increasing the preferred branch’s sampling weight from 12 to 16 lowers SR from 57.19% to 49.96% in the tested setting. This is a useful finding for anyone training search policies from expert trajectories.

Critical weaknesses
  1. The coverage label has a strong geometric component, and the agent already uses odometry to retrieve past views. Table 2(c) establishes the benefit of making coverage available. It leaves open how well the same policy would perform with a simpler coverage estimate built from the same RGB observations and poses. This distinction affects how much credit to assign to the world-model formulation itself.

  2. In Table 3(a), adding Foresight changes SR by −0.20 points without Recall and by +2.57 points with Recall. That 2.77-point difference supports complementarity on OVON Val-Unseen. The paper gives no estimate of variation across training runs, so it is difficult to tell how reliably the joint gain exceeds the sum of the separate gains. This matters because the abstract gives that claim considerable weight.

Questions for authors
  1. How does learned Recall compare with a simple geometry-based coverage estimate projected into the current view, with the same RGB and odometry inputs, demonstrations, target detector, and action-policy architecture? Reporting both exploration-mask IoU and navigation SR/SPL would show whether better coverage estimates translate into better decisions.
  2. Across independent training seeds, how variable is the 2.77-point SR interaction in Table 3(a)? Even an uncertainty estimate from paired evaluation episodes for the existing checkpoints would help distinguish evaluation noise from the separate question of training variability.
  3. In the existing evaluation trajectories, how often does each variant re-enter a previously searched branch before first seeing the target? A comparison on matched starting episodes, excluding backtracking needed to exit a dead end, would test whether the behavior illustrated in Figure 4 explains the aggregate improvement.
2 comments

We thank Reviewer esXe for recognizing the empirical evidence linking Recall to navigation performance, particularly the state-input ablation and retrieved-history examples, as well as the value of balanced exploration demonstrations. We address the reviewer's concerns through a geometric-coverage comparison, multi-seed evaluation, and quantitative analysis of repeated exploration.

1. Add a controlled comparison with geometric coverage

We thank the reviewer for suggesting this comparison to distinguish the value of explicit coverage information from that of learned Recall. Learned Recall outperforms geometric coverage in both exploration-state accuracy and navigation performance. We compare Recall with a geometric-coverage estimator using the same RGB observations and odometry. Both methods share identical demonstrations, SAM3 detector, action policy architecture, and training budget, with each policy trained using its corresponding exploration-state estimate.

Table S1. Geometric coverage vs. Recall. IoU: Table 3(b) evaluation set; SR/SPL: OVON Val-Unseen.

MethodIoU (%) ↑SR (%) ↑SPL (%) ↑
Geometric coverage46.2851.7031.66
RecallWAM (Ours)76.1057.1934.49

Recall improves IoU by 29.82 points, SR by 5.49 points, and SPL by 2.83 points. To understand this substantial gap, we further analyzed the predictions through visualization and found that geometric coverage fails to account for occlusions. Specifically, pixels corresponding to walls or furniture may be projected onto an assumed ground plane, incorrectly marking unseen regions as explored. In contrast, Recall learns history-dependent exploration state from visual observations, better reflecting what the agent has actually observed.

2. Add multi-seed evaluation of Recall–Foresight gains

Joint Recall–Foresight learning consistently provides super-additive gains across three training seeds. We repeat the four configurations in Table 3(a), keeping the data, backbone, perception model, and training budget fixed.

Table S2. Recall–Foresight ablation on OVON Val-Unseen (mean ± sample standard deviation across three seeds).

RecallForesightSR (%) ↑SPL (%) ↑
××52.15 ± 0.3831.15 ± 0.34
×✓52.08 ± 0.1331.86 ± 0.19
✓×54.63 ± 0.4232.21 ± 0.27
✓✓57.28 ± 0.5634.53 ± 0.49

We compute the interaction for each seed as I=SRF−SR−SF+S0I = S_{RF} - S_R - S_F + S_0, measuring the joint gain beyond the sum of individual improvements. The interaction averages 2.72 ± 0.46 SR points and 1.61 ± 0.37 SPL points (mean ± sample standard deviation), supporting the reliability of the super-additive benefit.

3. Quantify repeated entry into previously searched branches

We thank the reviewer for suggesting branch re-entry as a quantitative measure of the redundant exploration. Recall reduces redundant branch re-entry by 54.5%. We compare RecallWAM with its w/o Recall variant on matched OVON Val-Unseen starting episodes, keeping visual history, Foresight, and perception fixed.

We measure repeated entries into previously searched branches before the target is first seen or the episode terminates, excluding necessary backtracking from dead ends. The re-entry rate decreases from 27.38% to 12.45%, a 14.93-point absolute reduction. This quantitatively supports the behavior illustrated in Figure 4: Recall helps the agent avoid redundant search and select unexplored branches.

Reviewer_esXeOpenReview ↗

Response to Author Rebuttal

Thank you for the response and clarifications. I have considered the additional information and will maintain my current rating.

Comments on the paper

3 comments
Area_Chair_na1EOpenReview ↗

Invitation to Discuss the Authors’ Response

Dear Reviewers,

The authors have now provided their response. I encourage all reviewers to read it and participate in the discussion with the authors to clarify any remaining questions or concerns.

Thank you to both the authors and reviewers for your time and effort throughout the review process.

Overall Response

We thank all reviewers for their constructive feedback and recognition of our motivation and novelty (i7GZ, 9aYR, ZAic, MtJy), methodological design (9aYR, ZAic, MtJy), experimental evidence (esXe, i7GZ, ZAic, MtJy), and Balanced Exploration and benchmark results (esXe, 9aYR, ZAic, MtJy).

To address the shared concerns, we provide additional experiments and clarifications summarized below.

1. Validate learned Recall against explicit spatial memory

Learned Recall outperforms geometric coverage, map-derived exploration states, and explicit spatial memory under controlled settings. For the explicit spatial memory baseline, the policy directly consumes a continuously accumulated BEV map rather than a map-derived exploration state. RecallWAM outperforms this baseline by 2.16 SR points and 3.05 SPL points (57.19/34.49 vs. 55.03/31.44). These comparisons further demonstrate the advantage of learning task-relevant exploration memory directly from visual history over explicit map-based alternatives. (esXe, i7GZ)

2. Isolate Recall's contribution from perception and Foresight

Recall's gains cannot be attributed to SAM3 alone. Multiple baselines, such as DRIVE-Nav and OpenFrontier, also employ SAM3. More importantly, under identical perception, data, and backbone, adding Recall to the matched Foresight–Action baseline improves SR/SPL by 5.24/2.65 points. (ZAic)

3. Validate training reliability and exploration behavior

Joint Recall–Foresight learning consistently provides super-additive gains across three independent training seeds, with an interaction of 2.72 ± 0.46 SR points and 1.61 ± 0.37 SPL points (mean ± sample standard deviation). Furthermore, Recall reduces repeated branch entry from 27.38% to 12.45%, a 54.5% relative reduction, directly supporting its ability to reduce redundant exploration. (esXe, ZAic)

4. Evaluate localization robustness and long-term memory efficiency

RecallWAM maintains comparable performance with RGB-based MASt3R pose estimates, with only 0.7/0.1-point SR/SPL decreases. At the benchmark's maximum episode length of 500 steps, historical storage requires 333.60 MiB, with 56.65 ms retrieval latency, while the policy context remains fixed in size. (i7GZ, MtJy)

We also clarify evaluation differences with prior methods and qualify SOTA claims where appropriate (i7GZ), discuss the scalability of Balanced Exploration and provide small-object navigation results (9aYR), and clarify that the failure analysis identifies remaining challenges for future work (ZAic).

Detailed results and explanations are provided in our individual responses.

Thank you for the author's thoughtful efforts to restore confidence in the paper and for the clarifications. The author's response addressed my concerns regarding Recall’s contribution under matched perception, the Fast-WAM-style baseline, multi-seed super-additivity, and the purpose of the failure analysis. However, after careful consideration, I still find that core issues—particularly the system-level SOTA attribution in Table 1 and the motivation framing—remain insufficiently resolved, so I will keep my rating.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.