acceptodds
Under review as a conference paper at ICLR 2027

NavHarness: Towards Lifelong Embodied Navigation

10 views

Abstract

Frontier models can now perform well on individual embodied navigation tasks through multi-round multimodal reasoning with simple tools. Across successive tasks, however, an agent must also rely on an evolving map and earlier search records, both of which may be incomplete or conflict with new observations. We present NavHarness, a training-free embodied harness towards lifelong navigation that makes memory processing part of the navigation loop. During navigation, its multi-round agentic session draws on maps, task records, and house knowledge, checking them against observations and recording corrections to guide its actions. NavHarness preserves this experience across fresh conversations for new tasks or recovery attempts, while outcome verification and run-end summaries support its later reuse. On GOAT-Bench, NavHarness improves s-SR over context-only independent sessions by 18.6 points with Astra and 22.6 with Opus 5. Using SLAM-estimated poses, NavHarness with GPT-6 Astra achieves state-of-the-art task success of 83.7 s-SR with 36.9 e-SR on GOAT-Bench and 85.9 s-SR on IR2R-CE. To understand these gains, we examine how experience is carried between sessions and find that structured recovery handovers outperform length-matched summaries. In extended deployments across houses, consolidation improves navigation beyond retaining maps and task records, with case studies showing how agents use earlier experience to interpret new goals, investigate unresolved questions, and resume failed searches. We suggest that progress towards lifelong navigation depends on how successive reasoning sessions build on prior experience, alongside improvements in single-task capability.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Embodied Navigation, Embodied AI, MLLM, LLM, Agentic Embodied System, Harness

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_7t8aOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The goal of the work is to build a harness for lifelong navigation. Lifelong means that, instead of just finding one object in a house and terminating the episode, the agent is tasked with subsequent goals in the same house and can use its experience and memory from prior tasks for the current one. This work approaches the task with multimodal reasoning models and, to better manage memory and context, proposes NavHarness, which orchestrates different module executions and organizes memories from past tasks to best support task execution. In a way, drawing a parallel to coding models, it can be seen as context management for long-horizon tasks. The harness here includes inductive biases specific to indoor navigation. In my impression, NavHarness is an orchestration layer around existing reasoning agents.

Critical strengths
  • It clearly improves over naively deploying a multimodal reasoning agent by managing prior memory and context smartly.
  • In Table 1, the authors improve over prior work that uses GT poses with SLAM-estimated poses.
  • NavHarness gives consistent gains across various model backbones, which suggests that the design is not overfitted to one particular model.
Critical weaknesses
  • The paper is hard to follow in places. Although core terms are defined at the start of Sec. 3, the mechanisms in Sec. 3.3-3.4 (recovery scope, the wake-up session, pre-stop verification vs. post-stop certification) are dense and would benefit from a simple worked example. Several details needed to interpret the results appear only in the appendix, e.g., that the reasoning cores run inside Claude Code, Codex, and mini-SWE-agent, and that the baselines receive only observation and action tools (Appendix B.3). Figures 1 and 2 also add little: their captions are brief and generic, and they don’t help explain the mechanisms described in the text.
  • Much of the methodology is engineered specifically for lifelong navigation, built around context and memory management on top of existing coding agents. The empirical results are strong, but it is difficult to see how the insights inform other robotics tasks. The paper’s general-sounding findings rely on navigation-specific designs: for example, the structured handover that outperforms a matched-length summary (Table 3) has fields like searched vs. ruled-out places, a reliable landmark, and untried options, and the authors note that this comparison tests information selection and organization together (Appendix B.6). It shows that this hand-designed structure helps, rather than a general principle about what should carry across sessions.
    • The memory practices themselves (persistent files, handovers, compaction, agent-directed retrieval) are adopted from coding and memory agents such as MemGPT, as the paper acknowledges. The harder and more general problem is how to learn what experience to retain and how to compress it, which this work specifies by hand.
    • Many parts of the pipeline are hand-defined based on task-specific observations, so it is unclear whether there is a general design scheme to follow or whether the system reflects navigation-specific insights gathered from extensive experimentation.
      • Is some of the handoff and compression just prompt engineering?
      • The independent-session baselines run in Claude Code and Codex but receive only observation and action tools (Appendix B.3). Would a generic coding agent given the same map and file tools and a persistent workspace, instructed to keep its own notes, recover much of the gain without NavHarness’s orchestration? This would show how much of the improvement comes from the harness design versus simply having persistent tools, and whether improving coding agents might make task-specific harnesses unnecessary.
Questions for authors
  • The baselines share NavHarness’s per-task budgets of 200 navigation-model turns and 500 steps (Table 11), but NavHarness also makes model calls outside this budget for the judge, consolidation, and recovery wake-up session, and tasks use only around 40 turns on average, so the turn cap rarely binds. Could the authors report total model calls and tokens per task for NavHarness and the baselines? This would show whether part of the gain comes from additional inference compute.
  • I would be very interested to hear how this specific harness provides insights on how to improve context and memory management for robots. It doesn’t need to be experiments, but even an interesting insight would help. Without it, I lean towards a rejection.
Reviewer_aCkTOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

NavHarness combines persistent maps, task records, recovery handovers, completion checks, and consolidated house notes for training-free navigation across model sessions. Its contribution is primarily system design and empirical evidence about transferring search experience between sessions, with limited algorithmic novelty.

Critical strengths
  • Useful navigation-specific memory design: Handovers distinguish visited places from evidence-supported exclusions and preserve untried search options, with controlled evidence supporting their benefit.

  • Broad improvements:

    • Gains hold across four executors and two benchmarks using SLAM poses.
    • Astra reaches 83.7% GOAT-Bench and 85.9% IR2R-CE task success.
  • Strong controlled evidence: With the executor fixed, structured handovers beat matched-length summaries by 8.3 points, agent-directed retrieval beats the specified fixed-access baseline by 7.0 points, and consolidation adds 7.7 points beyond retained maps and records. These controls support benefits beyond using stronger frontier models.

Critical weaknesses
  1. Distinction from existing memory-agent frameworks
  • Persistent memory, reflection, and recovery are established ingredients (as the related work acknowledges).
  • The experiments show the design’s usefulness within NavHarness, but do not demonstrate its advantage over existing memory-agent frameworks with equivalent navigation tools and budgets.
  • The handover comparison changes information selection and organization together, leaving their individual contributions unresolved.
  1. Lifelong scope and memory reliability
  • Finite, static-house deployments include repeated targets; aggregate gains do not separate repeated-goal reuse from transfer to new targets.
Questions for authors
  • How do gains differ between repeated goals and previously unseen targets?
  • How often do erroneous records cause downstream failures?
  • How does it compare with an existing memory-agent framework under matched models, navigation tools, and budgets?
4 comments
AuthorsOpenReview ↗

A kind follow-up on our response

Dear Reviewer aCkT,

We sincerely thank the reviewer again for the time and care devoted to our paper. As the discussion period is coming to an end, we would like to kindly ask whether our response has addressed the reviewer's concerns. We have revised the paper accordingly, with all changes marked in blue and summarized in the general response. If any points remain unclear, we would be glad to provide further clarification before the discussion closes.

We are very grateful for the reviewer's thoughtful feedback.

Best regards, The Authors

Dear Reviewer aCkT,

As the discussion period closes shortly, we wanted to follow up once more. We would greatly appreciate any brief comments on whether our response and revisions have addressed your concerns. Even a short note would be very helpful to us.

Thank you again for your time.

Best regards, The Authors

I thank authors for answering my questions and additional experiments. I'll keep my rating as a positive assessment.

Dear Reviewer aCkT,

Thank you for your reply and for maintaining your positive assessment. We greatly appreciate your constructive feedback, which has helped improve the paper, and we will incorporate the additional experiments and clarifications into the final version.

Best regards, The Authors

Reviewer_1mxCOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

NavHarness is a training-free framework for lifelong embodied navigation. The framework maintains spatial maps, task histories, and persistent memory across navigation sessions. It also introduces structured handovers for recovering from failed attempts, completion verification, and memory consolidation. They evaluate their approach on GOAT-Bench and IR2R-CE using 4 different MLLM backbones and show improvements over independent navigation sessions.

Critical strengths
  1. NavHarness achieves SOTA performance on GOAT-Bench and IR2R-CE, with across multiple MLLM backbones.
  2. The paper provides meaningful ablations on cross-task memory and structured recovery handovers, showing the benefits of these components over simpler alternatives.
Critical weaknesses
  1. My main concern is the methodological novelty of the approach. Most components, including persistent memory, recovery through context resets, and completion verification, have been explored in prior work. The contribution seems to lie primarily in how these components are put together. I would have liked to see a clearer discussion of what is fundamentally new here. The related work section also misses several relevant papers:

[1] CLEA: Closed-Loop Embodied Agent for Enhancing Task Execution in Dynamic Environments

[2] Learning from Trials and Errors: Reflective Test-Time Planning for Embodied LLMs

[3] Can MLLMs Find Their Way in a City? Exploring Emergent Navigation from Web-Scale Knowledge

  1. While the paper emphasizes lifelong navigation, all experiments are conducted in static simulated environments. Even the extended deployment experiments revisit the same houses without changes to object locations or scene layouts. I think this is an important limitation, since a lifelong agent should be able to handle outdated memories and changes in its surroundings. It would be useful to understand how the proposed framework performs when previously stored spatial information becomes incorrect.

  2. The main independent-session baseline does not have access to external maps, memory retrieval, or completion verification, all of which are available to NavHarness. Hence, the large improvements over this baseline is difficult to attribute to any particular contribution.

Questions for authors

Refer above

5 comments
AuthorsOpenReview ↗

A kind follow-up on our response

Dear Reviewer 1mxC,

We sincerely thank the reviewer again for the time and care devoted to our paper. As the discussion period is coming to an end, we would like to kindly ask whether our response has addressed the reviewer's concerns. We have revised the paper accordingly, with all changes marked in blue and summarized in the general response. If any points remain unclear, we would be glad to provide further clarification before the discussion closes.

We are very grateful for the reviewer's thoughtful feedback.

Best regards, The Authors

Dear Reviewer 1mxC,

As the discussion period closes shortly, we wanted to follow up once more. We would greatly appreciate any brief comments on whether our response and revisions have addressed your concerns. Even a short note would be very helpful to us.

Thank you again for your time.

Best regards, The Authors

Reviewer_1mxCOpenReview ↗

Thank you for your response

I thank the authors for their detailed response and additional experiments. While these additions address some of my concerns, my primary concern regarding the novelty of the proposed method remains unresolved. I therefore intend to maintain my original score.

Thank you again for your efforts and the constructive discussion.

Comments on the paper

2 comments
AuthorsOpenReview ↗

A kind follow-up on our response

Dear Reviewer 7t8a,

Thank you again for the constructive discussion and for indicating your intention to raise your score. We have now added the fresh-session and self-refined memory comparisons discussed in our follow-up to the revised manuscript (Appendix B.8).

Could you please let us know whether any concerns remain, or whether there is anything further we can clarify or supplement before your final assessment? We would be glad to address any outstanding points.

Thank you for your time and consideration.

Best regards, The Authors

AuthorsOpenReview ↗

Author Discussion Summary and Manuscript Revisions

Dear AC and Reviewers,

We sincerely thank the Area Chair and all reviewers for their time and feedback. As the discussion period draws to a close, we summarize our responses and the status of the final follow-ups for the Area Chair’s consideration.

  • Novelty beyond existing agent components (1mxC, aCkT, 7t8a). General Responses 1 and 3 and the replies to R1-W1 and R3-Q2 explain where coding-agent practices fall short in physical search, motivating state-preserving recovery, search-evidence handovers, and checks on records used by later tasks. Sections 1–2 now make this contribution explicit.

  • Evaluation limited to static environments (1mxC, aCkT). General response 2 and R1-W2 report the added LiSoNav-Eval experiment with changing object layouts. It tests whether the robot can check and revise outdated memories, against a no-cross-task-memory control (Section 4.5, Appendix B.10).

  • Weak baselines and attribution of gains (1mxC, aCkT, 7t8a). R1-W3 explains the role of the component ablations. R2-W1/Q3 and R3-W6 add Mem0 and a self-managed coding agent, followed by fresh-session and self-refined-note variants in the R3 follow-up (Table 3, Appendix B.8).

  • Handover content versus format (aCkT, 7t8a). R2-W1 and R3-W2/W5 discuss the added length-matched 2×2 comparison. It finds a larger effect from the search evidence retained than from the fixed format (Appendix B.9).

  • Repeated goals and erroneous memory (aCkT). R2-Q1/Q2 separate gains on repeated and previously unseen targets and report how often incorrect markers and completion records affect later tasks (Section 4.4, Appendix C.5).

  • Additional inference cost (7t8a). R3-Q1 reports navigation-call counts and total input tokens, including auxiliary-call input. Baselines consume more input tokens per task, and the response distinguishes token volume from computation (Appendix B.11).

  • Readability and transferable insights (7t8a). R3-W1/Q2 clarify backend and tool access, expand the figure captions, and add a worked example and design guidance for robotic memory (Section 4.1, Appendices A.7–A.8).

All manuscript changes are marked in blue.

Status of the final follow-ups.

  • Reviewer 7t8a confirmed that the self-managed coding-agent baseline and token accounting addressed their concerns and stated their intention to raise their score, although the score has not yet been updated. We also addressed both questions raised in their subsequent follow-up by completing experiments with fresh sessions per task and a self-refined memory scheme. We provided the results in our reply and added them to Appendix B.8. We have followed up but are still awaiting their response to these additions. We respectfully ask the Area Chair to take this discussion and the completed experiments into account in the final assessment.

  • Reviewer 1mxC received detailed explanations of our novelty in the general and reviewer-specific responses. After their follow-up, we added General Response 3 to clarify the contribution further and explicitly asked which aspects remained unresolved. We have not yet received a further response or a more specific explanation of the remaining novelty concern.

This reflects the discussion status at the time of this summary. We respectfully ask that the final assessment give full and fair consideration to our contribution, the revised manuscript, the additional experiments, and the complete discussion record.

We sincerely thank the Area Chair for considering this summary and all reviewers for their time and feedback. We would be glad to provide any further clarification.

Best regards, The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.