acceptodds
Under review as a conference paper at ICLR 2027

Learning When to Recall and Where to Search for Lifelong Multi-Modal Navigation

Abstract

Lifelong multi-modal navigation requires reusing past observations under changing goals. Yet storing observations selectively or in compressed form can omit details that later requests need, and retrieved evidence must still be interpreted to decide whether to revisit a known location or explore further. We introduce RecallNav, which separates goal-agnostic observation storage from goal-conditioned memory access and learns when to recall and where to search. It retains original observations across goals and uses a frozen multimodal encoder to retrieve a small set of candidate frames. A multimodal large language model (MLLM) reads retrieved, current, and frontier views within a bounded policy context and learns from expert decisions to ground targets or select destinations for further search. Explicit recall supervision ties return decisions to supporting retrieved views. Even with a 3B MLLM backbone, RecallNav outperforms all compared baselines in navigation success on every GOAT-Bench validation split. It also surpasses the reported GPT-4o-based snapshot baselines in both success rate and path efficiency on Val-Unseen. On LMEE-Bench, the same 3B policy exceeds the 7B memory-based systems in both navigation success and path efficiency. The same RGB archives support later question answering through a separate pretrained MLLM without QA-specific fine-tuning. Ablations show that goal-relevant memory access and explicit recall supervision each contribute to navigation performance.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Embodied navigation, lifelong navigation, memory retrieval, vision-language models

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_83rCOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes RecallNav, a model for lifelong navigation. Its main contribution is to introduce goal-agnostic visual memory storage and the corresponding image retrieval mechanism. This allows the agent to retrieve relevant images in advance when handling subsequent subtasks in a lifelong navigation setting, providing additional visual evidence for downstream decisions. The experiments demonstrate that RecallNav achieves strong performance using a relatively small backbone model of around 3B–4B parameters, together with a 2B-parameter memory retrieval module.

Critical strengths

1.The paper achieves strong empirical performance using relatively small models, with a 3B or 4B backbone and a 2B embedding model, outperforming some 7B models. This suggests that lifelong navigation can be addressed effectively with smaller models through a well-designed overall architecture. This is a meaningful finding.

2.The ablation studies validate the contribution of visual memory retrieval, demonstrating that it improves the model's performance on subtasks.

Critical weaknesses

1.The visual memory in the experiments is stored without any capacity constraints. This seems inconsistent with the goal of lifelong navigation, where memory scalability is an important consideration. Since the number of subtasks within an episode is still limited in current benchmarks, the method can complete the tasks by relying on this effectively unbounded storage strategy. However, unlimited memory storage is not a reasonable assumption for genuine lifelong navigation.

2.The title and abstract claim that the method learns when to recall and where to search, but the proposed framework does not appear to substantiate these claims. Both when to recall and where to search are determined by fixed components of the framework, rather than being explicitly learned or decided by the policy model. Essentially, the model is trained to make decisions based on the available, more relevant visual evidence, such as retrieved memories. Recall is performed as part of the ongoing process rather than being triggered by a learned decision about when it is needed. Meanwhile, the search location is determined using retrieved images and their associated depth maps. Therefore, the framework does not explicitly learn when to recall or where to search, despite the claims made in the paper.

Questions for authors

None.

5 comments
AuthorsOpenReview ↗

Response to Reviewer 83rC (1/2)

We thank the reviewer for recognizing the empirical results and for raising concerns about memory capacity and the role of the learned policy. We address these points with fixed-budget archive evaluations and a clarification of the learned action interface.


[W1]: Memory capacity

RecallNav largely preserves its navigation performance under fixed archive budgets. Our focus is on using previously collected observations to support navigation under subsequent goals, as also noted by Reviewer hEny. In the static scenes evaluated here, repeated visits can produce overlapping views. We used this spatial redundancy to decide which observations to remove in Table R6. Developing adaptive memory management policies that better preserve evidence for unknown future goals under fixed storage budgets remains a direction for future work.

We evaluated 100- and 200-frame limits without retraining on all 360 Val-Unseen episodes. Each limit is fixed across all goals and episodes in its condition. The retention rule uses overlap among views in the static scenes evaluated here: panoramas captured before policy decisions are prioritized; the current panorama and images representing frontiers are retained and counted toward the budget. We first remove other images collected during movement or episode initialization, followed by past panoramas if needed. Within each group, the views most redundant in position and viewing direction are removed first.

Table R6. Memory retention in the main NavMesh setting: Qwen2.5-VL-3B, all 360 GOAT-Bench Val-Unseen episodes / 2,669 goals. Frame counts and storage are averaged over policy decisions; storage is in decimal MB.

Archive settingSR / SPL (%)Mean framesMean RGB (MB)
Unrestricted73.3 / 56.5279386
200-frame limit72.5 / 55.2148204
100-frame limit71.5 / 54.986120

Table R5 in our response to Reviewer Qpvm reports a separate TSDF evaluation on 36 Val-Unseen episodes (278 goals), in which both the 100- and 200-frame limits were satisfied at every decision. The two experiments use the same retention priorities but differ in frontier construction and episode coverage. Appendix K.3 includes both evaluations. Section 5 now distinguishes these fixed-budget results from the remaining challenge of adaptive retention for unknown future goals and memory updates in dynamic environments.

AuthorsOpenReview ↗

Response to Reviewer 83rC (2/2)


[W2]: Learned recall and search decisions

RecallNav learns whether to recall a target and which frontier to explore. The policy either selects frontier jj through EXPLORE(jj) or selects image ii and predicts the target's bounding box bb through FOUND(i,bi,b) (Eq. 1). Eq. 2 supervises the complete action output. This learned action choice is also recognized in Reviewer hEny's W2 and Reviewer Qpvm's summary.

“When to recall” is the policy's decision to select a historical image through FOUND(i,bi,b). Retrieval runs at every decision to supply candidate images. The policy then chooses whether to ground the target in memory, ground it in a current view, or continue exploring. Section 3.3 defines recall as the return visit triggered by grounding in memory, followed by a new panorama and another policy decision. We have clarified this distinction between candidate retrieval and an executed recall.

“Where to search” is the policy's selection of frontier jj through EXPLORE(jj). The depth-based destination computation mentioned in the review applies to executing FOUND(i,bi,b): it converts the policy-selected image ii and predicted target box bb into a reachable destination using depth and camera pose. This computation follows the policy's grounding prediction and does not select the exploration frontier.

Recall supervision improves navigation with the retrieval procedure and controller fixed. Replacing memory-recall training examples with an equal number of current-view examples reduces SR from 81.0% to 62.5% (Table 4). On identical held-out contexts, RecallNav achieves 80.4% macro-averaged agreement with expert decision categories, compared with 45.2% for No SFT (Table R2).

AuthorsOpenReview ↗

Gentle Reminder

Dear Reviewer 83rC,

Thank you again for your time and effort in reviewing our paper.

As the discussion period draws to a close, we would like to kindly remind you that approximately two days remain for further comments or questions. We would greatly appreciate any feedback on our rebuttal and additional experimental results, and welcome the opportunity to address any remaining questions or concerns before the discussion period ends.

Thank you very much for your time and consideration.

Best regards,
Authors

Reviewer_QpvmOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes RecallNav for navigation across successive category, description, and image goals. The system retains original observations, retrieves frames relevant to the current goal, and trains a multimodal policy to choose between recalling a past sighting and exploring a frontier. Depth and camera pose convert visual grounding into a navigation destination. The paper evaluates three policy backbones on GOAT Bench and LMEE Bench. It also tests whether the collected observations support later question answering.

Critical strengths

On GOAT Bench Val Unseen, replacing relevant retrieval with the four most recent frames reduces success from 81.0% to 55.3%. Replacing recall examples with current view examples reduces success to 62.5%. These are substantial changes under the same policy interface and controller.

RecallNav improves navigation success across all three GOAT Bench validation splits. On LMEE Bench, the evaluated policies also collect observations that support later questions. This breadth makes the result more consequential than an improvement confined to one goal type.

Critical weaknesses

With the same Qwen2.5 vision language backbone, RecallNav raises Val Unseen success from 62.7% to 73.3% relative to AstraNav Memory. Path efficiency changes from 56.9% to 56.5%. The matched backbone comparison therefore supports a success gain, but no path efficiency gain. The LMEE Bench baselines also differ in sensing, control budgets, and stopping rules. The additional TSDF evaluation reduces the map concern, but it does not align those remaining conditions. The large cross system gap cannot be assigned entirely to learned recall.

Table 13 reports about 900 MB of retained data for RecallNav, compared with 317 MB for AstraNav Memory. The archive has no fixed storage limit. The retrieval ablations establish that relevant frame selection matters. They do not establish that an unbounded original frame archive is needed at a comparable storage budget. Two pertinent studies are absent from the related work. Memoir uses selective retrieval from viewpoint memories [1]. TrajRAG retrieves compact geometric and semantic experience for navigation [2]. Their tasks differ from GOAT Bench, so a direct score comparison is unwarranted. Their approaches nevertheless make the storage and retrieval distinction important to the novelty claim.

[1] Y. Xu, Y. Pan, and Z. Liu, “Dream to Recall: Imagination-Guided Experience Retrieval for Memory-Persistent Vision-and-Language Navigation,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 48, no. 8, pp. 9035–9049, 2026. doi.org

[2] Y. Wang, S. Zhang, K. Zhang, X. Song, S. Du, and S. Jiang, “TrajRAG: Retrieving Geometric-Semantic Experience for Zero-Shot Object Navigation,” Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 15166–15176, 2026.

Questions for authors

Can the authors compare RecallNav and the strongest trained memory baseline under the same frontier construction, camera configuration, action budget, and stopping rule? Please report both success and path efficiency.

How do success, path efficiency, and retrieval hit rate change when the RecallNav archive is limited to the storage budget of AstraNav Memory? This would test whether the gain requires retaining substantially more visual data.

5 comments
AuthorsOpenReview ↗

Response to Reviewer Qpvm (1/3)

We thank the reviewer for raising concerns about evaluation conditions and memory use, and for pointing us to relevant related work. We address these points through controlled navigation comparisons, fixed-budget archive evaluations, and a clearer account of memory storage, retrieval, and use.


[W1 & Q1]: Same-backbone results and evaluation conditions

RecallNav maintains a substantial performance advantage over fine-tuned 3D-Mem with a common backbone and controls for frontier construction, camera configuration, navigation budget, and success scoring. Tables R4a–b specify the controls and show their cumulative effects.

GOAT-Bench. We fine-tuned 3D-Mem with the same Qwen3.5-4B backbone, training-set size (52,961 examples), and optimization recipe (Table 6) as RecallNav. Its supervision was generated from the same source training episodes using 3D-Mem's released pipeline. RecallNav used its original checkpoint, trained with NavMesh-based frontier supervision, throughout the evaluation.

Table R4a specifies the controls. We match 3D-Mem's camera resolution, HFOV, height, and tilt, and restrict RecallNav to at most 50 policy decisions and 50 m of travel per goal, enforced during rollout. Its original low-level action limits, including panorama turns, remain in place. The methods retain their respective controllers.

The camera and budget controls largely preserve RecallNav's performance relative to the TSDF setting. Applying these controls cumulatively changes SR/SPL from 76.6/58.6 in Table 12 to 76.2/59.9 before changing success scoring (Table R4b). Applying the baseline's terminal-position success criterion yields 77.4 SR and 60.2 SPL, exceeding fine-tuned 3D-Mem by 6.9 and 9.5 points, respectively.

Table R4a. Evaluation settings and controls applied to RecallNav.

ConditionRecallNav, main settingFine-tuned 3D-MemControl applied to RecallNav
(a) Frontier constructionNavMesh-based frontiers (§3.1; Appendix E)TSDF frontiersThe same TSDF algorithm and parameters
(b) Camera configuration720 × 640; 110° HFOV; 1.31 m height; level1280 × 1280; 120° HFOV; 1.5 m height; 30° downward tilt3D-Mem camera settings
(c) Action budget500 actions/goal; 5,000 actions/episode, including panorama turnsAt most 50 planner steps/goal, each moving up to 1 mAdd limits of 50 policy decisions and 50 m of travel per goal, enforced during rollout
(d) Success criterionDeclared stop within 1 m of a valid goal viewpointFinal position within 1 m, without requiring a STOP declaration3D-Mem terminal-position success criterion

Table R4b. Cumulative evaluation controls on GOAT-Bench Val-Unseen: all 360 episodes / 2,669 goals; Qwen3.5-4B.

Method and cumulative controlsSRSPL
RecallNav: (a) TSDF frontiers (Table 12)76.658.6
RecallNav: (a) + (b) 3D-Mem camera settings77.260.4
RecallNav: (a) + (b) + (c) decision and travel limits76.259.9
RecallNav: (a) + (b) + (c) + (d) terminal-position success criterion77.460.2
Fine-tuned 3D-Mem70.550.7

LMEE-Bench. Both RecallNav backbones also retain higher SR and SPL than MemoryExplorer under the controlled evaluation conditions (Table R4c). We use TSDF frontier construction, the baseline camera configuration, online limits of 50 m and 50 decisions per goal, and terminal-position success scoring. Appendix K.2 includes the protocols and results for both benchmarks.

Table R4c. LMEE-Bench navigation on all 821 goals. Controlled runs enforce the decision and travel limits during rollout.

Method and evaluation settingBackboneSRSPL
RecallNav, paper setting (Table 2)Qwen2.5-VL-3B53.135.2
RecallNav, controlled settingQwen2.5-VL-3B45.128.5
RecallNav, paper setting (Table 2)Qwen3.5-4B61.141.4
RecallNav, controlled settingQwen3.5-4B58.237.8
MemoryExplorer, re-evaluatedQwen2.5-VL-7B23.414.1
AuthorsOpenReview ↗

Response to Reviewer Qpvm (2/3)


[W2-1 & Q2]: Storage and reduced visual memory

RecallNav retains higher SR and SPL than AstraNav-Memory with the same Qwen2.5-VL-3B backbone under the 100-frame archive limit. Its mean RGB-plus-depth storage is 208 MB per decision (125 MB RGB and 83 MB depth), which is below AstraNav-Memory's mean RGB storage of 317 MB.

Furthermore, we evaluated both 100- and 200-frame limits without retraining. We prioritize panoramas captured as the robot looks around before choosing its next action. The current panorama and images representing frontiers are retained and counted toward the budget. We first remove other images collected during movement or episode initialization, followed by past panoramas if more space is needed. Within each group, the views most redundant in position and viewing direction are removed first. Also, developing adaptive memory management policies that better preserve evidence for unknown future goals under fixed storage budgets remains a direction for future work.

Table R5. Table 13 navigation setting: Qwen2.5-VL-3B, TSDF frontiers, 36 Val-Unseen episodes / 278 goals. Frame counts and storage are averaged over policy decisions; storage is in decimal MB. Both frame limits were satisfied at every decision.

Method / archiveSR / SPL (%)Mean framesMean RGB (MB)Mean depth (MB)Mean RGB + depth (MB)Retrieval hit rate (%)
AstraNav-Memory65.1 / 51.1229317—317—
RecallNav, unrestricted74.5 / 55.938953835989754.8
RecallNav, 200-frame limit72.7 / 54.616823215538752.8
RecallNav, 100-frame limit72.3 / 54.5901258320853.9

Retrieval hit rate is computed over all decisions in each setting's rollout. A hit requires a valid target satisfying Eq. 3 in at least one of up to four retrieved images captured before the current panorama. Table R6 in our response to Reviewer 83rC reports the same retention rule under both frame limits in the main NavMesh setting on all 360 Val-Unseen episodes (2,669 goals). Both evaluations are included in Appendix K.3.

AuthorsOpenReview ↗

Response to Reviewer Qpvm (3/3)


[W2-2]: Novelty in memory storage, retrieval, and use

RecallNav's technical contribution encompasses a goal-agnostic archive of original observations, direct multimodal retrieval, and learned recall decisions. Memoir [1] uses imagined future states to retrieve encoded viewpoint observations and latent behavioral histories for memory-persistent VLN, while TrajRAG [2] retrieves geometric-semantic trajectory experience and supplies its descriptions to an LLM planner for zero-shot ObjectNav. These representations may omit visual details that become relevant to subsequent goals. RecallNav retains the original RGB images so that appearance and contextual details can be re-examined for category, description, and image goals. Table R5 evaluates this design under fixed archive budgets.

RecallNav matches each new goal directly against a reusable, goal-agnostic frame index. A frozen multimodal encoder computes each frame descriptor once, allowing category, description, and image queries to reuse the index without recomputing archived frame descriptors. We compared this strategy with a 3D-Mem-inspired category-prefiltered retrieval variant, keeping the storage rule, encoder, and trained policy unchanged. Direct goal-to-frame retrieval improves SR/SPL by 4.8/4.6 points over this variant (Table 3).

The proposed explicit recall supervision teaches RecallNav to turn retrieved visual evidence into navigation decisions. The policy is supervised to select a supporting image and target box or an exploration frontier. Replacing memory-recall training examples with an equal number of current-view examples reduces SR/SPL by 18.5/15.4 points under the same retrieval procedure and controller (Table 4). We have added Memoir and TrajRAG to the related work and clarified the distinctions in observation storage and navigation decisions in Sections 2.2 and 2.3.

[1] Y. Xu, Y. Pan, and Z. Liu, “Dream to Recall: Imagination-Guided Experience Retrieval for Memory-Persistent Vision-and-Language Navigation,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 48, no. 8, pp. 9035–9049, 2026.

[2] Y. Wang, S. Zhang, K. Zhang, X. Song, S. Du, and S. Jiang, “TrajRAG: Retrieving Geometric-Semantic Experience for Zero-Shot Object Navigation,” Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 15166–15176, 2026.

Reviewer_hEnyOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces RecallNav, a navigation framework that conducts lifelong navigation by separating goal-agnostic observation storage and goal-conditioned memory retrieval. RecallNav stores all observations within the episode and retrieves images relevant to the current subtask via cosine similarity. Then an MLLM is fine-tuned to decide if the goal of the current subtask is found in the retrieved memory, or to select a frontier for further exploration. Experiments on GOAT-Bench and LMEE-Bench show substantial improvements in navigation success over the compared baselines.

Critical strengths
  1. The paper is clearly motivated, focusing on studying how previously collected visual observations can support navigation under subsequent goals.
  2. The empirical navigation performance is very promising.
Critical weaknesses
  1. Unfair comparison due to different MLLM backbones and settings.

Except for memory retrieval, RecallNav and 3D-Mem share substantial similarities, but the comparison between them is not fair. First, Table 1 must explicitly annotate whether each baseline method is evaluated in a fine-tuned or zero-shot setting. Second, the authors should report 3DMem’s performance when fine-tuned on Qwen3.5-4B—a necessary ablation to isolate the contribution of RecallNav’s design. Given the Qwen3 series’ demonstrated advances in spatial reasoning capabilities, omitting this controlled comparison prevents attribution of observed gains: it remains ambiguous whether improvements arise from RecallNav’s novel mechanisms or from the stronger underlying backbone.

  1. Missing decision-level evaluation.

The main claim of the paper is to learn when to recall and where to search, and correspondingly, RecallNav explicitly learns to choose between EXPLORE(j) and FOUND(i, b). However, the paper reports retrieval hit rate and end-to-end navigation metrics without directly evaluating the policy's decision accuracy. In particular, it remains unclear how often the policy correctly selects the action type, chooses appropriate visual evidence, and localizes the target conditional on selecting FOUND. Without an evaluation of FOUND and an analysis of its impact on final navigation performance, the main contribution of recallnav would remain unclear.

Questions for authors
  1. How much of the navigation improvement comes from better memory retrieval versus better policy learning?
5 comments
AuthorsOpenReview ↗

Response to Reviewer hEny (1/2)

We thank the reviewer for the careful assessment of baseline comparisons, decision quality, and the contributions of retrieval and policy learning. We address these points with a fine-tuned 3D-Mem comparison, a fixed-context decision evaluation, and a joint retrieval–policy ablation.


[W1-1]: Training status in Table 1

As per your comment, Table 1 now marks methods with navigation-specific policy or goal-selection training using (*). This marker applies to SenseAct-NN Monolithic, SenseAct-NN Skill Chain, MTU3D, AstraNav-Memory, and RecallNav. In contrast, 3D-Mem, SCOPE, and HGR use their navigation MLLMs without navigation-specific fine-tuning. The annotation excludes off-the-shelf perception and PointNav modules. A separate (†) marks our re-evaluated results, including MTU3D.


[W1-2]: Fine-tuned 3D-Mem

RecallNav still leads with a substantial margin over fine-tuned 3D-Mem when the backbone, training-set size, and optimization recipe are matched. Specifically, we fine-tuned 3D-Mem with the same Qwen3.5-4B backbone, training-set size (52,961 examples), and optimization recipe (Table 6) as RecallNav. Its supervision was generated from the same source training episodes using 3D-Mem's released pipeline. RecallNav uses its original checkpoint trained with NavMesh-based frontier supervision throughout these evaluations.

Table R1 uses the final evaluation conditions in Tables R4a–b of our response to Reviewer Qpvm: TSDF-based frontiers, 3D-Mem camera settings, decision and travel limits, and terminal-position success scoring. This result supports the contribution of the proposed mechanisms beyond the benefits of a stronger backbone and navigation fine-tuning. The protocol and full results are included in Appendix K.2.

Table R1. GOAT-Bench Val-Unseen: all 360 episodes / 2,669 goals; Qwen3.5-4B.

MethodSRSPL
Fine-tuned 3D-Mem70.550.7
RecallNav77.460.2

AuthorsOpenReview ↗

Response to Reviewer hEny (2/2)


[W2]: Decision-level evaluation

On identical held-out contexts, RecallNav achieves 80.4% decision-category agreement, 87.2% goal-visible image precision, and 74.4% grounding precision, compared with 45.2%, 77.9%, and 12.3% for No SFT. We evaluated both Qwen3.5-4B policies on 5,609 fixed expert-labelled contexts from all 360 GOAT-Bench Val-Unseen episodes, with identical goals, retrieved images, current panoramas, and frontier candidates. No SFT uses the pretrained backbone prompted with action examples (Appendix B.1).

Table R2 evaluates the three requested decision components:

  • Action choice: agreement with the expert over three decision categories: EXPLORE, FOUND (current), and FOUND (memory). We split FOUND by the source of the selected image and macro-average agreement over the three categories. Expert labels follow Section 3.4 and Appendix A, prioritizing current-view grounding, then useful recall, then exploration.
  • Visual evidence: the fraction of FOUND predictions whose selected image contains a valid target meeting the visibility threshold in Eq. 3.
  • Grounding: the fraction of FOUND predictions whose box has IoU ≥ 0.5 with an annotated box for a valid target in the selected image.

Both precision metrics use each policy's own FOUND predictions, irrespective of the expert action label. These definitions and results are included in Appendix K.1.

Table R2. Decision-level evaluation (%). Decision-category counts refer to expert labels; precision-row counts refer to each policy's predictions.

Component / metricRecallNavNo SFT
Decision-category agreement: macro-average80.445.2
EXPLORE agreement (expert: EXPLORE; n=2,492)79.792.0
FOUND(current) agreement (expert: FOUND(current); n=2,549)86.815.5
FOUND(memory) agreement (expert: FOUND(memory); n=568)74.828.2
Visual evidence: goal-visible image precision (all FOUND)87.2 (n=3,262)77.9 (n=1,190)
FOUND(current): goal-visible image precision87.4 (n=2,540)70.0 (n=503)
FOUND(memory): goal-visible image precision86.7 (n=722)83.7 (n=687)
Grounding precision (all FOUND; IoU ≥ 0.5)74.4 (n=3,262)12.3 (n=1,190)
FOUND(current): grounding precision73.7 (n=2,540)13.5 (n=503)
FOUND(memory): grounding precision76.7 (n=722)11.4 (n=687)

In separate end-to-end rollouts using the same goal-conditioned retrieval procedure, policy fine-tuning improves SR from 57.0% to 81.0% and SPL from 32.3% to 63.2% (Table R3). These results complement the fixed-context evaluation by measuring the overall navigation effect of policy learning.


[Q1]: Retrieval versus policy learning

With the trained RecallNav policy fixed, goal-conditioned retrieval adds 25.7 SR points and 21.8 SPL points over recent-frame inputs. With goal-conditioned retrieval fixed, policy fine-tuning adds 24.0 SR points and 30.9 SPL points. Table R3 varies the policy and memory-input condition independently; these gains are conditional on the accompanying component.

To test the role of recall supervision, we replaced the 6,250 memory-recall training examples with an equal number of current-view examples sampled from the existing training set (Recall→current). This preserves the training-set size and EXPLORE/FOUND counts (Table 4; Appendix A).

Table R3. GOAT-Bench Val-Unseen: all 360 episodes / 2,669 goals; Qwen3.5-4B; main evaluation setting. SR / SPL (%); gains are percentage points. Retrieval gains hold the policy fixed; fine-tuning gains hold the memory-input condition fixed.

Policy / comparisonMost recent 4 framesGoal-conditioned retrieval, up to 4 framesRetrieval gain
No SFT20.6 / 12.457.0 / 32.3+36.4 / +19.9
Recall→current57.8 / 43.262.5 / 47.8+4.7 / +4.6
RecallNav55.3 / 41.481.0 / 63.2+25.7 / +21.8
Policy fine-tuning gain (RecallNav − No SFT)+34.7 / +29.0+24.0 / +30.9—

The larger retrieval gain for RecallNav than for Recall→current shows that explicit recall supervision helps the policy use retrieved history. With the retrieval procedure fixed, this supervision also raises SR from 62.5% to 81.0% and SPL from 47.8% to 63.2%.

AuthorsOpenReview ↗

Gentle Reminder

Dear Reviewer hEny,

Thank you again for your time and effort in reviewing our paper.

As the discussion period draws to a close, we would like to kindly remind you that approximately two days remain for further comments or questions. We would greatly appreciate any feedback on our rebuttal and additional experimental results, and welcome the opportunity to address any remaining questions or concerns before the discussion period ends.

Thank you very much for your time and consideration.

Best regards,
Authors

Comments on the paper

1 comment
AuthorsOpenReview ↗

Common Response

Dear Reviewers and Area Chairs,

We thank you for your careful reviews and constructive feedback. RecallNav combines goal-agnostic observation storage, direct multimodal retrieval, and a learned policy that grounds targets in current or historical observations or selects a frontier for further search.

We have revised the manuscript to include:

  • References to Memoir and TrajRAG and a discussion distinguishing memory storage, retrieval, and the use of retrieved evidence for navigation decisions (Sections 2.2–2.3).
  • Explicit annotations of navigation-specific training in Table 1 and a clarification of learned recall and frontier selection (Section 3.3).
  • Decision-level evaluation of action categories, visual evidence, and target grounding on fixed held-out contexts (Section 4.3; Appendix K.1).
  • A comparison with fine-tuned 3D-Mem on GOAT-Bench using the same backbone, training-set size, and optimization recipe, included in Table 1 alongside the corresponding RecallNav results under controlled evaluation conditions. The evaluation controls for frontier construction, camera configuration, navigation budgets, and success scoring on GOAT-Bench and LMEE-Bench are detailed in Section 4.1 and Appendix K.2.
  • A comparison with four frames uniformly spaced across the navigation history (Uniform-K), using the same trained policy and memory-input budget (Table 3; Section 4.3; Appendix B.3).
  • Navigation and retrieval results under fixed archive budgets, with an updated discussion of memory scalability (Sections 3.2 and 5; Appendix K.3).

The revisions are highlighted in blue. Our responses below address each reviewer's comments in detail.

Sincerely,
The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.