acceptodds
Under review as a conference paper at ICLR 2027

GhostAnchor Navigation: Execution-Updated Topological Control with Locate–Verify–Ground Target Approach for Zero-Shot Object Navigation

Abstract

Zero-shot object navigation requires an agent to find and approach a text-specified object category in an unknown environment. Vision-language models (VLMs) can infer search directions and local actions from current observations, but a one-step decision does not retain the execution history of previously selected candidates. We present GhostAnchor Navigation (GA-Nav), a training-free framework that applies VLMs to zero-shot ObjectNav through execution-updated topological control. Its Node–Ghost Topology Memory (NGTM) associates depth-derived candidate waypoints across observations and stores candidates that are not matched to Nodes as Ghosts with explicit states. After each action, NGTM updates the selected Ghost from the measured pose change and uses the resulting topology to filter the directions available to the next VLM decision. When the target becomes visible, the Locate–Verify–Ground (LVG) procedure selects and verifies a target observation waypoint, then estimates a 3D target anchor from RGB-D for approach and stopping. We evaluate GA-Nav on the complete validation sets of HM3D v0.1, HM3D v0.2, and MP3D. Among the training-free zero-shot methods compared, GA-Nav ranks first in both SR and SPL on all three benchmarks. Specifically, GA-Nav achieves absolute SR/SPL gains of percentage points on HM3D v0.1, points on HM3D v0.2, and points on MP3D, relative to the best prior training-free zero-shot result for each metric.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Zero-Shot Object Navigation, Vision-Language Models, Topological Memory, Embodied AI

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_SMD8OpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes GhostAnchor Navigation (GA-Nav), a training-free zero-shot ObjectNav framework that uses an execution-updated Node–Ghost Topology Memory (NGTM) for VLM-guided exploration and a Locate–Verify–Ground (LVG) procedure for target approach. Experiments on HM3D v0.1, HM3D v0.2, and MP3D report improved navigation performance, supported by component ablations.

Critical strengths
  1. Practical framework. GA-Nav presents a relatively complete navigation pipeline, and the reported results demonstrate its practical effectiveness.
  2. Substantial evaluation. The experiments cover multiple ObjectNav benchmarks and include component ablations, reflecting considerable implementation and experimental effort.
Critical weaknesses
  1. Insufficient differentiation from existing topological-navigation ideas. Node–Ghost structures, cross-view candidate association, and exploration-state-based decisions have been studied in Neural Topological SLAM, ETPNav, and Frontier-enhanced Topological Memory. The paper does not yet clearly establish whether pose-based Ghost updates and direction filtering provide a sufficiently distinct capability beyond existing state maintenance, failure marking, and backtracking. This distinction should be made more explicit in the Introduction.
  2. Incomplete evidence for robustness and comparative coverage. Spatial proximity does not necessarily imply the same navigable candidate, and sufficient movement toward a Ghost does not necessarily mean that its direction has been fully explored. Erroneous association or premature reached/failed updates could exclude useful directions, yet the paper lacks corresponding error analysis and threshold sensitivity results. In addition, Table 1 should include more recent zero-shot methods, such as MSGNAV, to strengthen the direct comparison.

Overall, the framework is effective, and the experimental effort is substantial, but stronger evidence is needed to show that its success comes from the proposed central idea rather than an incremental combination of effective components.

Minor presentation suggestions: Figure 2 is dense because it includes six VLM functions and multiple control branches. Figure 3 would be clearer if it explicitly illustrated how Ghost-state changes modify the VLM’s available directions. The figures could be refined accordingly.

Questions for authors
  1. Novelty of execution-updated control. Relative to prior topological-navigation methods, what fundamentally different decision-making capability does the execution-feedback mechanism provide beyond tracking exploration states, marking failed nodes, and backtracking? What is its incremental contribution in this indoor navigation setting?
  2. Validity of Ghost state updates. In complex indoor scenes, how does the method prevent spatially close but topologically different candidates from being incorrectly associated? How is it ensured that progress toward a Ghost indicates that the corresponding exploration direction has been completed? Could the authors provide Ghost association accuracy, representative state-update failures, and sensitivity analyses for the key thresholds, particularly cases where valid directions might be prematurely filtered out?
  3. Coverage of recent baselines. Could the authors include more recent zero-shot ObjectNav methods (e.g., MSGNAV) in the main comparison in Table 1, or explain why such comparisons are unavailable?
5 comments

We thank the reviewer for the thorough and expert review. Your questions target exactly what the paper needs to establish, so we designed controlled experiments around them, each on all 1,000 HM3D v0.2 validation episodes with Qwen3-VL-32B-Instruct. Every variant changes one setting and is paired by episode with a re-run of the full method (SR 72.2 / SPL 38.6); 95% CIs come from 10,000 paired bootstrap resamples.

Q1/W1: What does execution-updated control add beyond prior topological navigation?

We agree that Node–Ghost graphs, cross-view candidate merging, failure marking, and backtracking are established ideas (NTS, ETPNav, FTM). The revised introduction now states this explicitly, and we do not claim these primitives as our contribution.

The difference lies in who makes the decisions and how the state reaches them:

  • In NTS, ETPNav, and FTM, the planner is trained with navigation supervision and can learn how to exploit the graph state.
  • In training-free ObjectNav, the decision-maker is a frozen VLM that can be influenced only through its input. Prior training-free VLM methods supply memory as soft context (WMNav: a value map; ReMemNav: retrieved memories; MSGNav: a scene graph), which the VLM remains free to ignore.
  • GA-Nav converts execution-grounded state (persistent candidate identities; Nodes, traversed edges, and Ghost states updated from measured motion) into hard constraints: the state defines the admissible direction set, with deterministic backtracking, and the VLM makes semantic choices only among the options that geometry and execution history leave open.

The capability this adds is that a frozen, untrained VLM makes long-horizon decisions that stay consistent with what the agent has actually executed, without having to learn how to use a map. We tested this directly:

VariantSRSPLΔSR [95% CI]ΔSPL [95% CI]Repeated Node–Node traversals / ep.
Full method (re-run)72.238.6––2.43
State as text only70.138.0−2.1 [−4.2, 0.0]−0.6 [−1.8, 0.6]3.14

In the text-only variant, directions toward failed Ghosts and repeatedly traversed Nodes are no longer removed: they stay selectable and are flagged in the prompt ("[Failed Ghost Frontier, failures=N]", "[WARNING: Path traversed N times]"). Each episode thus exposes the VLM to 1.65 failed-Ghost and 4.77 repeated-Node directions that the full method would remove. Despite the explicit warnings, the VLM re-enters used edges 29% more often, and SR drops by 2.1 pp.

Together with Table 3, where cross-step NGTM contributes far more than any other component (+7.5–8.8 SR and +4.5–5.9 SPL, vs. +1.5 and +1.2 SR for adding VerifierVLM and GrounderVLM to the full configuration), this indicates that GA-Nav's gains come mainly from its central idea, persistent execution-grounded topology used as a hard constraint, rather than from an incremental combination of equally important components. Accordingly, the revised introduction states that GA-Nav inherits Node–Ghost graphs, cross-view candidate merging, failure marking, and backtracking from this line of work, in which the planner is trained to exploit the graph state, whereas in TF-ZS ObjectNav the planner is a frozen VLM and prior training-free methods supply memory only as soft context; the remaining problem is to make the outcome of each executed action constrain the next decision of a frozen VLM. The first contribution now reads:

We introduce NGTM, which tracks depth-derived candidate waypoints across viewpoints, updates the topology and candidate states from measured motion, and uses them to restrict the directions available to VLM planning and to backtrack deterministically.

Q2/W2: Validity of Ghost state updates.

(1) What "reached" means. Our wording in Sec. 3.3 ("made sufficient progress along that direction") caused this concern, and we have rewritten it. "Reached" only ends the current attempt toward one specific candidate waypoint. It does not declare the direction explored, and it never removes a direction from the allowed set. Reached Ghosts are excluded from further association, so they neither block nor absorb new candidates, and the panorama at the new position generates new candidates further ahead. The allowed set shrinks only when:

  • (i) a direction matches a failed Ghost, i.e., one with two failed attempts (displacement < 0.25 m, no decrease in distance, or a non-translational action);
  • (ii) a Node–Node edge has been traversed at least twice while an unexplored Ghost is visible;
  • (iii) a target-memory Ghost is visible (only its direction is kept); or
  • (iv) ObserverVLM describes a direction as non-traversable.

When no Ghost direction remains, the agent deterministically backtracks toward the nearest unresolved frontier; if no direction is executable, all six directions are re-opened. Filtering therefore cannot trap the agent.

(2) Is completion premature? The full method makes 7,740 Ghost adjudications, 73.3% of which end as reached. In 88.8% of reached cases, the agent actually came within 0.65 m of the waypoint (86.7% satisfy both the distance and progress conditions, 2.1% only the distance condition); the remaining 11.2% satisfy only the progress condition, i.e., the agent covered at least half of the initial distance. Requiring the agent to get closer before completion (ρ_reach = 0.50 m) brings no improvement (ΔSR −1.2 [−3.6, 1.1], ΔSPL −0.1 [−1.4, 1.2]), indicating that the default does not end attempts prematurely.

(3) Association errors and their reach. We agree that spatial proximity does not guarantee the same navigable candidate; association is an approximation. Its effect is bounded by design: reached Ghosts are never matched, matching an unexplored or pending Ghost keeps the direction selectable, and among Ghost states only failed removes a direction. A wrong association can therefore remove a valid direction only through the failed state (matching an already failed Ghost, or a matched Ghost failing twice afterwards). Empirically:

  • Association radii of 0.60 m and 0.90 m give no detectable difference (ΔSR −0.3 [−2.7, 2.1] and −0.9 [−3.3, 1.5]).
  • Removals due to failed Ghosts are rare: 1.12 per episode by default and 0.85–1.25 across radii.
  • N_fail = 1, which nearly triples these removals (3.05 per episode), gives no detectable SR change (−1.1 [−3.5, 1.3]); N_fail = 3 gives −1.3 [−3.7, 1.1].
  • If the filtering often removed valid directions, disabling it should help; instead, SR drops by 2.1 pp (see Part 1/3).

(4) Association accuracy and failure cases. The simulator provides no ground-truth identities for depth-derived candidates, so association accuracy cannot be measured directly. The revised Appendix H now provides step-by-step visualizations of representative episodes (Figs. 7–10; full replays of six episodes at https://anonymous.4open.science/r/GA-Nav-visualization-E8AF/README.md) that list every removed direction and the reason for its removal (e.g., "Matched a failed Ghost (G18, failures=2)"), so that state-update errors can be inspected directly.

W2/Q3: Recent zero-shot baselines.

We have added MSGNav to the TF-ZS group of Table 1, together with ApexNav and CogNav. A key difference lies in the models: these methods build on different and stronger model stacks, whereas GA-Nav uses a single open-weight VLM for all modules, without any dedicated detector or segmenter. SR / SPL as reported in the original papers:

MethodModelsHM3D v0.1HM3D v0.2MP3D
CogNav (ICCV'25)GPT-4V + GPT-3 (proprietary) + OpenSEED72.5 / 26.2–46.6 / 16.1
ApexNav (RA-L'25)YOLOv7 + Grounding-DINO + Mobile-SAM + BLIP-2 + DeepSeek-V359.6 / 33.076.2 / 38.039.2 / 17.8
MSGNav (CVPR'26)GPT-4o (proprietary) + YOLO-World + SAM + CLIP–74.1 / 33.4–
WMNav (IROS'25)Gemini (proprietary)58.1 / 31.272.2 / 33.345.4 / 17.2
GA-Nav (ours)Qwen3-VL-32B only (open-weight)58.5 / 35.673.1 / 39.149.5 / 24.9

These model differences matter:

  • CogNav: its own ablation shows that replacing GPT-4V and GPT-3 with the open-source CogVLM2-19B and Llama3.1-8B lowers its HM3D SR from 72.5 to 62.6.
  • ApexNav: it recognizes targets with dedicated detectors (YOLOv7 and Grounding-DINO) and uses the 671B-parameter DeepSeek-V3 for semantic priors. YOLOv7 is trained with box-level supervision on COCO, whose label set covers all six HM3D goal categories. On MP3D, whose 21 goal categories go beyond COCO, GA-Nav outperforms ApexNav by 10.3 SR and 7.1 SPL.
  • MSGNav: it combines proprietary GPT-4o with three dedicated perception models (YOLO-World, SAM, CLIP) to build an explicit 3D scene graph.
  • WMNav: it uses proprietary Gemini; GA-Nav outperforms it on all six metrics.

Among the TF-ZS methods in the updated Table 1, GA-Nav achieves the best SPL on all three benchmarks (+2.6, +1.1, and +7.1 pp over the next best) and the best SR on MP3D (+2.9 pp), using only one open-weight VLM. On HM3D, methods built on these stronger model stacks report higher SR. Under an identical backbone, GA-Nav outperforms prior TF-ZS methods by a large margin (Table 2: +9.5 SR / +7.6 SPL over ReMemNav), and its performance improves directly with stronger VLMs (Table 4). The revised abstract, introduction, Sec. 4.2, and conclusion now state our claims precisely: best SPL on all three benchmarks and best SR on MP3D among TF-ZS methods.

Figures. Thank you for these suggestions. We have simplified Fig. 2 around its three-stage backbone (observation, search, approach), with shorter labels and more compact notation; the complete control flow of the six VLM functions is given in Algorithm 1 (App. A) and their prompts in App. D. Regarding your suggestion for Fig. 3: the former Fig. 6 in Appendix B illustrates, with an example, how Ghost states determine the allowed direction set. To make this easier to follow, we have moved it into the main text as Fig. 4, directly after Fig. 3. The revised Appendix H also adds step-by-step visualizations of successful and failed episodes (Figs. 7–10), which show how Ghost-state changes alter the allowed direction set from step to step.

Thank you again for the insightful comments, which helped us state our contribution more precisely. If our responses resolve your concerns, we would be grateful if you would consider updating your score, and we are glad to discuss further.

Reviewer_kpo5OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents GA-Nav, a training-free framework for zero-shot object navigation. The proposed approach introduces Node–Ghost Topology Memory (NGTM), which maintains exploration history and updates topological states based on actual action execution outcomes. Additionally, the Locate–Verify–Ground (LVG) procedure combines VLM-based target verification with RGB-D-based 3D grounding to support accurate target approach and stopping.

The method is evaluated on three benchmarks, including HM3D v0.1, HM3D v0.2, and MP3D, achieving strong performance in both success rate (SR) and success weighted by path length (SPL).

Critical strengths

Effective and practical framework. The proposed execution-updated topological memory provides an intuitive and effective mechanism for incorporating execution feedback into VLM-based navigation decisions.

Strong experimental performance. GA-Nav achieves leading results among the compared training-free zero-shot methods across three benchmarks, with particularly notable improvements in navigation efficiency.

Comprehensive experimental evaluation. The paper includes module ablations, comparisons under a common VLM backbone, and model sensitivity studies, providing convincing empirical evidence for the effectiveness of the framework.

Clear and well-structured methodology. The combination of NGTM and LVG provides a coherent solution for both target exploration and target approach. The overall pipeline is easy to understand, and the implementation details are well documented.

Critical weaknesses

Since GA-Nav involves multiple VLM modules, could the authors briefly discuss the average number of VLM calls per episode and the corresponding inference time?

NGTM relies on several predefined thresholds, such as Ghost association distances and failure counts. Could the authors provide some insights into how sensitive the navigation performance is to these hyperparameters?

Could the authors provide a few representative failure cases to better illustrate the current limitations of the proposed approach?

Questions for authors

Please follow the above questions.

1 comment

Thank you for the positive assessment and constructive questions. All new quantitative results use all 1,000 HM3D v0.2 validation episodes with Qwen3-VL-32B-Instruct. Variants are paired by episode with a full-method re-run (SR 72.2 / SPL 38.6); 95% CIs use 10,000 paired bootstrap resamples.

Q1: VLM calls and inference time per episode.

Each episode averages 21.6 macro steps and 51.1 VLM calls (~2.4 per step). Modules run as needed: ObserverVLM once per panoramic observation; PlannerVLM and ActorVLM during search; LocatorVLM and VerifierVLM only when a target direction is found; and GrounderVLM only during target approach.

ModuleCalls / episodeMean request (s)Request time / episode (s)
ObserverVLM14.6921.14310.67
PlannerVLM12.184.4854.55
ActorVLM13.311.9225.56
LocatorVLM2.523.448.66
VerifierVLM1.524.967.53
GrounderVLM6.923.2822.66
Total51.138.40429.62
  • Runtime. Active runtime averages 510.2 s/episode, from scene initialization to navigation loop completion (simulation, panorama rendering, VLM calls, and logging).
  • Measurement setup. One Qwen3-VL-32B-Instruct (BF16) replica on each of 7 GPUs served 40 concurrent Habitat workers, so request times include server-side queueing and reflect batch-evaluation throughput rather than single-robot latency.
  • Stability. All configurations use 51.1–52.1 calls per episode, with zero API failures.
  • Where the cost goes. ObserverVLM uses 72% of request time to describe six panoramic directions. Costs can be reduced by reusing descriptions at known Nodes, lowering panorama resolution, or using smaller models for short-output modules (ActorVLM, GrounderVLM). Smaller backbones are also viable: with Qwen3-VL-4B, GA-Nav reaches 65.7 SR / 29.4 SPL, still above all methods compared under the same backbone (Table 2).

This table and cost-reduction options are now in Appendix G (Table 7).

Q2: Sensitivity to NGTM thresholds. Each variant changes one parameter of the full method (Appendix G, Tables 5 and 6):

Parameter (default)ValueSRSPLΔSR [95% CI]ΔSPL [95% CI]
Ghost association radius (0.75 m)0.60 m71.938.7−0.3 [−2.7, 2.1]+0.1 [−1.1, 1.4]
0.90 m71.338.2−0.9 [−3.3, 1.5]−0.4 [−1.7, 1.0]
Failure count N_fail (2)171.137.7−1.1 [−3.5, 1.3]−0.9 [−2.2, 0.3]
370.938.2−1.3 [−3.7, 1.1]−0.5 [−1.7, 0.8]
  • All CIs include zero: within the tested ranges, SR changes by at most 1.3 pp and SPL by at most 0.9 pp, and the default setting gives the highest SR.
  • Mechanism statistics explain why: removals due to failed Ghosts are rare (1.12/episode by default; 0.85–1.25 across radii). Even N_fail = 1 nearly triples removals (3.05/episode) without a detectable SR change.

Q3: Representative failure cases.

In the re-run, 278/1,000 episodes fail by stopping at a wrong location or exhausting the 40-macro-step budget. Revised Appendix H analyzes three representative failures step by step (recorded with Qwen3-VL-8B; replays: https://anonymous.4open.science/r/GA-Nav-visualization-E8AF/README.md):

  1. Wrong stop after a category error (episode 651). Target recognition relies on the VLM: an armchair is taken for a sofa and accepted by VerifierVLM, GrounderVLM then anchors the wrong object, and the agent stops 13.6 m from the annotated sofa.
  2. Exhausted search budget (episode 147). The agent repeatedly explores bedrooms and corridors without confirming a chair; direction filtering and backtracking redirect the search, but the 40-step budget runs out first.
  3. Visible but unreachable target (episode 418). The toilet is partially visible, but the direct approach direction has no continuous traversable path; two approaches exit before success, and no detour is found within the remaining steps.

Ethics flag. The review flags ethics review without a stated concern. Our study uses only public simulated benchmarks (HM3D and MP3D in Habitat) under their academic licenses, and involves no human subjects, personal data, or physical robot deployment. Our generative AI use is disclosed in the AI Use Statement per ICLR policy. We are happy to address any specific concern.

Thank you for your support and suggestions.

Reviewer_6Td3OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes an agentic system for zero-shot object navigation. The key idea is to maintain a topological memory graph that provides the VLM with better global candidates to choose from. Each decision then follows a locate–verify–ground loop to select the best next waypoint. The method is evaluated on HM3D v1, HM3D v2, and MP3D, demonstrating superior performance.

Critical strengths
  1. The method is evaluated across three benchmarks, and a module-level ablation study is provided.

  2. The agentic system leverages recent advances in vision-language foundation models to address practical robotics problems.

  3. The overall pipeline is clearly presented and easy to follow.

Critical weaknesses
  1. The paper lacks comparison with more recent works that outperform GA-Nav, as summarized below (SR / SPL):
MethodHM3D v1HM3D v2MP3D
GA-Nav (this paper)58.5 / 35.673.1 / 39.149.5 / 24.9
LongNav-R1 [1]76.0 / 44.383.7 / 43.463.0 / 30.2
LightNav-0 [2]74.5 / 43.977.2 / 41.553.3 / 21.2
Uni-NaVid [3]73.7 / 37.1––

[1] LongNav-R1: Horizon-Adaptive Multi-Turn RL for Long-Horizon VLA Navigation, RSS 2026.

[2] LightNav-0: Eliciting VLM Spatial Intelligence for Generalist Embodied Navigation, arXiv 2026.

[3] Uni-NaVid: A Video-based Vision-Language-Action Model for Unifying Embodied Navigation Tasks, RSS 2025.

  1. The novelty is limited. Many agentic systems for navigation already exist. A clearer comparison with the literature that highlights what is genuinely new in this design would strengthen the paper.
Questions for authors
  1. Could you add more visualization to help understand the progress, such as the memory, waypoint candidates, selection, and VLM scores?
5 comments

We thank the reviewer for the review and for recognizing our evaluation on three benchmarks, the module-level ablation, and the clarity of the pipeline.

W1: Comparison with LongNav-R1, LightNav-0, and Uni-NaVid.

We thank the reviewer for pointing us to these strong methods. We would like to clarify, however, that they belong to a different setting from GA-Nav: they are trained navigation policies that learn from large-scale navigation supervision collected in HM3D/MP3D training scenes, whereas GA-Nav is training-free and zero-shot (TF-ZS).

MethodNavigation trainingNavigation data used for training
Uni-NaVid (RSS 2025)Imitation learning3.6M navigation samples, incl. 483k ObjectNav samples from HM3D training scenes
LongNav-R1 (RSS 2026)Imitation learning + multi-turn online RLHabitat-Web human demonstrations; online RL in 80 HM3D training scenes
LightNav-0 (arXiv, Aug. 31, 2026)SFT + DAgger + online RL4K+ hours of navigation data, incl. ObjectNav demonstrations from PIRLNav, HM3D, and MP3D
GA-Nav (ours)NoneNone (frozen, off-the-shelf Qwen3-VL-32B with fixed prompts)

The two settings answer different research questions. Trained policies show how far large-scale in-domain supervision can push performance on the benchmark distribution. TF-ZS methods ask how well a general-purpose VLM can navigate without any navigation data, rewards, or fine-tuning. This is the setting of the TF-ZS group in Table 1 (e.g., VLMNav, InstructNav, UniGoal, WMNav) and the focus of our paper; all our claims are made within it. In this setting, GA-Nav:

  • achieves the best SPL on all three benchmarks and the best SR on MP3D, also after adding the recent TF-ZS methods MSGNav, ApexNav, and CogNav (see our response to Reviewer SMD8);
  • outperforms prior TF-ZS methods under an identical backbone (Table 2, Qwen3-VL-4B: +9.5 SR / +7.6 SPL over the strongest baseline, ReMemNav);
  • improves directly with stronger general-purpose VLMs, without any retraining (Table 4: SR rises from 49.3 to 73.1 when moving from Qwen2.5-VL-3B to Qwen3-VL-32B).

We have added the three methods to the trained group of Table 1 with their training data stated, discussed them in the related work (Sec. 2.1), and noted that LightNav-0 is a concurrent arXiv preprint.

W2: Novelty.

We agree that many agentic VLM navigation systems exist, and the revised introduction now makes the distinction explicit.

Inherited from prior work (not claimed as ours): the Node–Ghost topology (NTS, ETPNav, FTM), cross-view candidate merging, failure marking, and backtracking.

What is new:

  1. Execution-grounded constraints for a frozen VLM planner (NGTM). Candidate identities persist across viewpoints, and Nodes, traversed edges (with traversal counts), and Ghost states are updated from measured motion rather than from VLM predictions. These states define the set of directions the VLM may choose, with deterministic backtracking. Prior TF-ZS methods give memory to the VLM only as soft context (e.g., WMNav's value map, ReMemNav's retrieved memories, MSGNav's scene graph), while trained topological planners (NTS, ETPNav, FTM) learn from navigation supervision how to use the graph. A frozen VLM can do neither, so GA-Nav enforces consistency with the executed history through the constraint itself.
  2. Target-memory Ghost. When a target approach is interrupted, the established target position is written back into the same topology as a prioritized candidate, linking approach failures to the subsequent search.
  3. LVG. LocatorVLM selects among annotated candidates, VerifierVLM independently checks the selection, and GrounderVLM provides a box and a point that, combined with depth, yield a 3D anchor fused across views; stopping is decided by the distance to this anchor.

Evidence.

  • Cross-step NGTM is the largest single factor in our ablation: +7.5–8.8 SR and +4.5–5.9 SPL (Table 3).
  • New controlled experiment: when the same state is given to the VLM only as text, with explicit warnings, SR drops by 2.1 pp (95% CI [−4.2, 0.0]) and repeated traversals of already-used edges increase by 29%. Even when warned, the frozen VLM re-enters used paths more often; the hard constraint prevents this.

Q1: More visualization.

The paper already shows a complete episode (Fig. 1), consecutive NGTM updates (Fig. 3), the construction of the allowed direction set with the Planner/Actor prompts and outputs (Fig. 4, now moved from App. B into the main text), LVG (Fig. 5), and a full Step 0–21 timeline (Fig. 6, App. A); figure numbers refer to the revised PDF. We agree that a full-episode view of how memory evolves helps. The revised Appendix H adds step-by-step visualizations of a successful and a failed episode (Figs. 7–10, recorded with Qwen3-VL-8B). Each row is one macro step and shows:

  • (a) the NGTM graph on the top-down map (Nodes, Ghosts, and their states);
  • (b) the decision view: the depth-derived candidate waypoints and the outputs of ActorVLM, LocatorVLM, VerifierVLM, and GrounderVLM (selected candidate, verification result, target box and point, 3D anchor);
  • (c) for each of the six directions, its NGTM element, whether it is allowed, the reason for each removal, and PlannerVLM's selected direction;
  • (d) NGTM update events, e.g., a target waypoint saved as a Ghost, paused and resumed updates, and the changing numbers of Nodes, Ghosts, and edges.

Appendix H also analyzes three successful and three failed episodes, and GIF replays of these six episodes are available at the anonymous link https://anonymous.4open.science/r/GA-Nav-visualization-E8AF/README.md (an interactive replay page, viewable macro step by macro step, can be downloaded there). GA-Nav's VLM modules output discrete JSON decisions rather than per-candidate numeric scores, so the visualizations show these decisions together with the reason NGTM gives for each removed direction.

We hope these clarifications address your concerns, in particular the setting difference from trained VLA policies and the new mechanism-level evidence. We would be grateful if you would take them into account in your assessment, and we are happy to discuss further.

Reviewer_6Td3OpenReview ↗

Official Comment of Reviewer

Thank you for the detailed response and the additional clarifications. However, two concerns remain.

(i) Performance. The method’s zero-shot results fall well short of two kinds of baselines. First, it trails data-driven approaches such as trained VLM-based navigators (e.g., LightNav), which, once trained, generalize zero-shot to both indoor and outdoor environments. Second, it trails very large general-purpose models such as GPT Astra and Fable, which have shown superior navigation performance.

(ii) Novelty. The contribution relative to existing agentic navigation systems is still not clearly established. I appreciate the new context design, but on its own it does not seem a strong enough contribution.

I will therefore keep my current score.

Comments on the paper

1 comment

We thank all reviewers for their careful reading and constructive feedback. Reviewers found GA-Nav practical and effective (kpo5, SMD8), valued the evaluation on three benchmarks with module ablations (6Td3, kpo5, SMD8), and found the pipeline clear (6Td3, kpo5). We have uploaded a revised PDF in which all changes are shown in blue. Below we summarize the main clarifications, new results, and revisions; details are given in the individual responses.

1. Setting and additional baselines

GA-Nav addresses training-free zero-shot (TF-ZS) ObjectNav: it uses no navigation demonstrations, no rewards, and no parameter updates, and all modules query a single off-the-shelf, open-weight VLM (Qwen3-VL-32B-Instruct) with fixed prompts.

  • Trained navigation policies (6Td3). LongNav-R1, LightNav-0, and Uni-NaVid learn from large-scale navigation supervision (imitation learning and/or online RL) collected in HM3D/MP3D training scenes. They address a different question from ours. We have added them to the trained group of Table 1, with their training data stated.
  • Recent TF-ZS methods (SMD8). MSGNav, ApexNav, and CogNav build on different and stronger model stacks: proprietary GPT-4o/GPT-4V and/or multiple dedicated detection and segmentation models. We have added them to the TF-ZS group. Using a single open-weight 32B VLM, GA-Nav still achieves the best SPL on all three benchmarks and the best SR on MP3D among the TF-ZS methods in the updated Table 1.

2. New experiments

All runs use the full HM3D v0.2 validation set (1,000 episodes) with Qwen3-VL-32B-Instruct. Each variant changes exactly one setting. For paired comparisons, we re-ran the full method together with the variants on the same episodes; 95% CIs come from 10,000 paired bootstrap resamples. Results and mechanism statistics are in the revised Appendix G (Tables 5 and 6).

VariantSRSPLΔSR [95% CI]ΔSPL [95% CI]
Full method (re-run)72.238.6––
Ghost association radius 0.75 → 0.60 m71.938.7−0.3 [−2.7, 2.1]+0.1 [−1.1, 1.4]
Ghost association radius 0.75 → 0.90 m71.338.2−0.9 [−3.3, 1.5]−0.4 [−1.7, 1.0]
Failure count N_fail 2 → 171.137.7−1.1 [−3.5, 1.3]−0.9 [−2.2, 0.3]
Failure count N_fail 2 → 370.938.2−1.3 [−3.7, 1.1]−0.5 [−1.7, 0.8]
Reach distance 0.65 → 0.50 m71.038.5−1.2 [−3.6, 1.1]−0.1 [−1.4, 1.2]
State given to the VLM as text only (no hard filtering)70.138.0−2.1 [−4.2, 0.0]−0.6 [−1.8, 0.6]
  • Robust to NGTM thresholds. Within the tested ranges, no threshold variant differs detectably from the full method, and the default setting gives the highest SR.
  • Hard constraints outperform textual memory. When the same execution-grounded state is only described to the VLM as text, SR drops by 2.1 pp and repeated traversals of already-used Node–Node edges increase by 29% (2.43 → 3.14 per episode).
  • Cost. An episode uses 51.1 VLM calls on average (about 2.4 per macro step), with zero API failures in all runs.

3. Revisions to the paper (in blue)

  • Table 1: adds the six methods above, with their training data and model stacks; the abstract, introduction, Sec. 4.2, and conclusion now state our claims precisely (best SPL on all three benchmarks and best SR on MP3D among TF-ZS methods).
  • Introduction: now separates what we inherit from prior topological navigation (Node–Ghost graphs, cross-view merging, failure marking, backtracking) from our contributions, with a rewritten contribution list.
  • Sec. 3.3: clarifies that "reached" only ends the current attempt toward one waypoint and never removes a direction.
  • Appendix G: the new experiments and mechanism statistics (Tables 5 and 6) and a per-module cost table (Table 7).
  • Appendix H: step-by-step visualizations of a successful and a failed episode (Figs. 7–10), and a case and failure analysis of three successful and three failed episodes; GIF replays of all six episodes are available at the anonymous link https://anonymous.4open.science/r/GA-Nav-visualization-E8AF/README.md (an interactive replay page can be downloaded there).
  • Figures: Fig. 2 is simplified. The former Fig. 6 (App. B), which illustrates with an example how Ghost states determine the allowed direction set, is now Fig. 4 in the main text, directly after Fig. 3.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.