acceptodds
Withdrawn from ICLR 2027

Knowing Where to Look Next: Adaptive Regional Indexing and Evidence-Guided Search for Multimodal Lifelong Navigation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Multimodal lifelong navigation requires robots to find successive targets specified by categories, language descriptions, or reference images in the same initially unknown environment while reusing past observations. Three bottlenecks limit effective search. First, object-indexed retrieval relies on detections made before future goals are known, carrying early omissions and mislabels into later attempts to retrieve target evidence and search cues. Second, semantic scoring ranks frontier candidates generated by geometry-only clustering of explored–unexplored boundaries, but cannot add cue-indicated observation points absent from that set. Third, scheduling must prevent new nearby geometric frontiers from repeatedly postponing pending inspections and identify worthwhile revisits when candidates are exhausted but the target remains unfound. We present TreeNav, a framework without task-specific model training that uses visual evidence not only to recognize targets, but also to decide where to search and where to look again. Its Scene Tree organizes observations into a content-adaptive regional hierarchy and retrieves raw images through regional context for goal-specific target verification and cue extraction. Regional budgets limit redundant checks while preserving verification opportunities across relevant regions. Grounded Semantic Frontiers (GSFs) turn identified search cues—goal-related objects and relevant regional information in historical and current images—into reachable observation candidates. These candidates extend historical reuse beyond direct target sightings and add key inspection locations missed by geometric frontiers during online exploration. Evidence-first wave scheduling balances discovery benefits against navigation costs, limits repeated displacement of pending inspections by new geometric frontiers, and promptly incorporates new GSFs to respond to new visual cues. After all executable candidates are exhausted, targeted recovery reuses the Scene Tree's regional index to select goal-relevant, insufficiently observed mapped locations for reinspection. In system-level comparisons on full GOAT-Bench val-unseen, TreeNav achieves 73.14% success rate and 50.14% success weighted by path length (SPL), exceeding the best reported result for each metric under the same protocol by 6.62 and 5.44 percentage points, respectively.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Multimodal Lifelong Navigation, Hierarchical Visual Memory, Visual Evidence Retrieval, Semantically Guided Exploration, Training-Free Navigation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_gKfKOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes a specific training free method for multi-goal navigation. It structures observations into a tree with vlm guided observation deduplication.

Critical strengths

The paper tackles a valuable problem (multi-goal navigation) using a popular method (training free vlm inference). The figures seem well made and the approach seems to attempt something sensible.

Critical weaknesses

The biggest issue with this paper is the writing. The style is very hard to read and understand. Many terms are introduced without explanation, and many mechanisms of the system seem to be unexplained. Overall many sections are over-verbose and the introduction is more of a related works section than a proper introduciton. See some of my specific questions below for examples below but as is, it is hard to follow this paper without major re-writes.

For the results themselves there appear to be several missing baselines which propose training-free navigation methods. Namely mini-swe-agent, AgenticNav, HSTM [1-4]. All proposed similar memory systems and the strength of this model over those seems to go on discussed. Furthermore, a lack of real-world experiments hurts this paper.

Finally, there are no confidence intervals or signifigance tests in the reported results. The procedures as outline in Agarwal et al. [5], would make these comparisons more convincing.

Questions for authors

“distinguish mapped space from adequately observed space,” These terms are undefined at their first usage.

“Scene Tree retrieves raw observations through regional context, without requiring correct target recognition at storage time. It builds an adaptive regional hierarchy from visual content, then selects representative images to verify the target and extract search cues when a goal arrives.”

“deduplicated by acquisition location" how is the de-duplication handled?

“vision-language model (VLM) compares each new panorama with nearby candidate regions, joining a visually con- sistent region or creating a new region or subregion.” How is this done?

“This goal-independent construction gives dis tinct contexts separate indexing units, including within the same room.” What are indexing units?

“Given goal g, the VLM scores sibling regions from their labels and summaries” what does “labels and summaries” mean?

“We introduce evidence-first wave scheduling and targeted recovery: wave-based admission limits repeated displacement of pending inspections by new geometric frontiers while promptly incorpo- rating new GSFs to respond to visual cues; within each wave, GSFs and geometric frontiers are ranked jointly by comparable target-discovery benefits and navigation costs; after candidate exhaution, recovery directs reinspection to goal-relevant, insufficiently observed regions.” This sentence is very hard to understand.

Reviewer_Xi5NOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies multimodal lifelong navigation, where a robot needs to find different targets in a unknown environment while making use of observations from past tasks. The authors propose a training-free navigation framework with three main components. First, a Scene Tree organizes past visual observations into a hierarchy of regions and retrieves relevant images for new goals. Second, Grounded Semantic Frontiers (GSFs) use visual cues to generate new observation locations instead of only relying on geometric frontiers. Third, a wave-based scheduler prioritizes exploration candidates and includes a recovery mechanism for revisiting areas that may not have been sufficiently observed. The method is evaluated on GOAT-Bench using Qwen3-VL-8B. On the full val-unseen split, TreeNav achieves 73.14% success rate and 50.14% SPL, outperforming the previously reported results. The authors also conduct several ablation studies to examine the contributions of the different components.

Critical strengths

1. Strong results on the full GOAT-Bench benchmark.

I think the experimental performance is one of the strongest parts of this paper. TreeNav achieves 73.14% SR and 50.14% SPL on the full val-unseen split, which is a clear improvement over the reported baselines. Compared with EvoMemNav, which uses the same VLM backbone, the improvement is also quite large. This makes the results convincing and suggests that the proposed navigation framework contributes meaningfully to the performance.

2. The component analyses provide useful evidence for the proposed design.

I also like that the authors try to isolate the effects of the individual components rather than only reporting the final benchmark numbers. For example, the count-matched comparison in Table 5 shows that adding GSF candidates leads to a 28.49% next-observation discovery rate. This suggests that the improvement is not simply due to generating more candidate locations.

The retrieval experiments in Table 3 and the recovery comparisons in Table 4 are also useful. Overall, these experiments make me more confident that the Scene Tree and the semantic frontier generation are contributing to the final performance.

Critical weaknesses

The experimental evidence for some of the individual components is not sufficiently strong.

My main concern is that the ablation studies are conducted on a relatively small subset of 36 episodes and 278 subtasks, rather than the full benchmark. This subset also appears easier than the complete evaluation, with the full method achieving 81.65% SR compared with 73.14% on the full split.

Several of the differences used to support the individual design choices are fairly small. For example, replacing the Scene Tree with flat memory reduces SR by 3.95 percentage points, while using equal discovery priors or direct VLM scores leads to differences of around 2 to 3 points. These correspond to a relatively small number of successful subtasks, yet the paper does not report confidence intervals or statistical significance for these comparisons. I think this matters because the framework combines several components, many of which are relatively simple ideas on their own. The contribution therefore depends on showing that each component provides a reliable benefit. Without stronger statistical evidence or larger-scale ablations, I am not fully convinced that all of the reported improvements would hold across different evaluation subsets.

Limited methodological novelty compared with existing navigation frameworks.

I think the novelty of the proposed method could be better justified. TreeNav combines hierarchical visual memory, semantic frontier generation, and a wave-based scheduler, but several of these ideas have already been explored in related navigation work. The Scene Tree mainly provides a different way of organizing and retrieving historical observations, while the wave scheduling and recovery mechanisms seem to rely on relatively simple heuristics.

I am particularly interested in how Grounded Semantic Frontiers differ from [1], which also uses visual-language grounding to guide frontier-based navigation without task-specific training. Although TreeNav additionally uses historical observations and can generate observation candidates beyond existing geometric frontiers, I think the authors should explain more clearly what is fundamentally new compared with previous approaches.

Overall, I see the contribution more as a combination and refinement of existing techniques than as a substantially new navigation method. The strong experimental results are valuable, but I think the paper needs a clearer discussion of its methodological novelty.

[1] OpenFrontier: General Navigation with Visual-Language Grounded Frontiers

Important implementation details and experimental comparisons are not sufficiently clear.

Although the overall performance is impressive, I found it difficult to determine whether all of the comparisons in Table 1 use sufficiently similar experimental settings. The reported results are taken from different papers with different model backbones, but some important details, such as the navigation action space, the step or distance budget, and differences in the camera setup, are not clearly compared.

The method also introduces many manually chosen parameters, including the regional retrieval thresholds, verification budgets, and navigation candidate settings. I could not find a clear explanation of how these values were selected or how sensitive the results are to them.

This is important because the paper proposes a fairly complex system with several interacting components. Without a clearer description of the protocol and parameter selection, it is difficult to judge how much of the improvement comes from the proposed ideas and how much depends on implementation choices. The supplementary material also provides only selected code excerpts rather than a complete implementation, which makes reproducibility more challenging.

I also found the paper somewhat difficult to follow at times, especially when moving between the different system components and their corresponding ablations. A clearer presentation of the full experimental setup would make the results easier to assess.

Questions for authors
  • How does Grounded Semantic Frontier differ from existing methods such as OpenFrontier? I would like to better understand what is fundamentally new beyond using visual-language cues to guide exploration.

  • Are the navigation budgets, camera settings, and observation frequencies consistent with the baselines in Table 1? If not, could these differences partly explain the reported improvements?

  • How were the fixed discovery priors and other hyperparameters selected? How sensitive is the performance to these choices, especially in environments with different target distributions?

  • How does the Scene Tree scale as the number of stored observations increases? Could the authors report the VLM inference cost and memory overhead compared with simpler retrieval methods?

  • Table 2 shows relatively small improvements for some components. Are these gains consistent across different episodes, or are they mainly driven by a small number of successful cases?

  • How much of the improvement comes from the adaptive hierarchical structure itself rather than simply retaining and rechecking raw observations? Could the authors clarify this distinction using the existing retrieval ablations?

Reviewer_ore7OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes TreeNav for multimodal lifelong navigation. It uses a Scene Tree to organize and retrieve historical observations, Grounded Semantic Frontiers (GSFs) to generate exploration locations from visual cues, and wave scheduling and targeted recovery to improve search efficiency. TreeNav achieves 73.14% SR and 50.14% SPL on the full GOAT-Bench val-unseen split. The paper also includes ablation studies and evaluations with different VLM backbones.

Critical strengths
  1. The paper addresses an important problem in multimodal lifelong navigation. Using visual cues to generate new exploration locations, rather than only ranking existing frontiers, is an interesting idea.

  2. The results are promising. The authors evaluate TreeNav on the full GOAT-Bench val-unseen split and provide ablation studies, candidate-count-controlled comparisons, and cross-backbone evaluations to support the results.

Critical weaknesses
  1. Table 1 includes a same-backbone comparison with EvoMemNav, but stronger baselines such as 3D-Mem and MSGNav use different VLMs. A matched-backbone comparison with these methods would help better assess TreeNav's performance gains.

  2. The paper reports the runtime of some components, but the end-to-end computational cost remains unclear, especially with repeated VLM calls during navigation.

Questions for authors
  1. How does TreeNav compare with strong baselines such as 3D-Mem or MSGNav when using the same VLM backbone?

  2. What is the end-to-end runtime of TreeNav, including the overhead of repeated VLM calls?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.