acceptodds
Under review as a conference paper at ICLR 2027

Grounding Visual Evidence for Long-Range Language-Conditioned Navigation

Abstract

Long-range language-conditioned visual navigation (LCVN) requires embodied agents to follow natural-language instructions across environments spanning hundreds of meters. Topology-based LCVN relies on visual appearance for target retrieval, map construction, and navigation, but appearance becomes an unreliable proxy for task relevance at long range: similar landmarks confuse retrieval, visually redundant observations can be instruction-critical, and appearance variation causes waypoint errors to accumulate. We present **GRONav**, a unified grounded topology-based navigation framework that makes visual evidence decision-specific: retrieval is grounded in action and trajectory context, map compression in semantic composition, and navigation in geometric structure distilled from monocular depth. Experiments in CARLA and on a real quadrupedal robot demonstrate consistent gains in retrieval accuracy, navigation success, and robustness over strong baselines, while maintaining the lowest inference latency. GRONav achieves 91.7% target retrieval accuracy, reduces topology size by 68.0%, and improves real-world navigation performance by more than 25.0% on 60–200m tasks.

Primary area:
applications to robotics, autonomy, planning
Keywords:
visual navigation, long-range navigation

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_r458OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents GRONav, a unified topology-based framework for long-range language-conditioned visual navigation. It consists of three main components: (1) action-grounded target retrieval, which combines visual-semantic similarity with trajectory-action matching; (2) semantic-grounded keyframe selection, which considers visual changes and semantic entropy to construct compact topological maps; and (3) spatially grounded navigation, which employs depth-feature alignment and gated attention to improve waypoint prediction.

Critical strengths
  1. The motivation is clear, and the problem is meaningful.
  2. The reported results are strong. GRONav achieves high retrieval accuracy, substantial map compression, and improved real-world navigation performance.
  3. The paper reports inference latency and provides useful ablation studies on the navigation policy, supporting the effectiveness of its design choices.
Critical weaknesses
  1. Although the problem is interesting, the individual technical contributions appear relatively incremental. The paper would benefit from a clearer discussion of its methodological novelty compared with existing approaches, such as LM-Nav, DynaNav, and related works.
  2. Although several recent baselines are included, the strongest baseline in Table 2 remains ViNT (CoRL 2023). Comparisons with more competitive recent methods would further strengthen the evaluation.
  3. The action-grounded retrieval module is a key contribution, but its individual design choices are not sufficiently ablated.
Questions for authors
  1. Why do all diffusion-based baselines achieve 0% success in Table 2? The authors mainly attribute these failures to inference latency, but this explanation lacks sufficient experimental evidence. Could the authors provide further analysis?
  2. Could the authors provide a more controlled comparison of visual backbones? Some ablated variants already achieve performance comparable to competing methods. Although the appendix explores different encoders, backbone-matched comparisons would help disentangle the contributions of pretrained visual representations from the proposed architectural designs.
  3. Could the authors clarify the SPL values in Table 11? Several reported SPL values exceed the corresponding success rates, which is theoretically impossible under the standard SPL definition.

I am willing to raise my score if the authors can adequately address most of my concerns.

4 comments
AuthorsOpenReview ↗

Response to Reviewer r458 - Part 1

W1: Novelty clarify and comparsion with existing approaches.

Although the problem is interesting, the individual technical contributions appear relatively incremental. The paper would benefit from a clearer discussion of its methodological novelty compared with existing approaches, such as LM-Nav, DynaNav, and related works.

Thank you for this constructive comment. We agree that the methodological novelty of GRONav and its distinctions from existing approaches should be articulated more explicitly. Our contribution is not to claim that individual techniques such as visual-language matching, keyframe selection, or attention mechanisms are new in isolation. Rather, we identify decision-specific visual evidence as a central consideration in long-range LCVN and develop three complementary mechanisms to address the distinct visual evidence requirements of target retrieval, map construction, and navigation.

(1) Action-grounded retrieval. Unlike LM-Nav [4], which primarily matches language-described landmarks to graph nodes using CLIP similarity, GRONav explicitly aligns instructed action directives with motion signatures recovered from candidate graph trajectories using DPVO and LCS matching. This introduces trajectory-level evidence to disambiguate visually similar targets.

(2) Spatially grounded navigation. Unlike DynaNav [1], EffoNAV [2], and NoMaD [3], which focus on navigation efficiency, foundation-model-based visual representations, and diffusion-based control, respectively, GRONav transfers geometric priors from a frozen DepthAnythingV2 model through feature alignment and gated attention, improving spatial awareness and efficient waypoint prediction.

(3) Semantic-grounded map compression. Unlike conventional visual-change-based keyframe selection methods, such as ORB-SLAM and AKS, GRONav combines adaptive visual variation with object-level semantic entropy from SAM 2, preserving instruction-relevant landmarks even when visual changes are small.

Beyond these individual mechanisms, GRONav integrates them into a unified long-range LCVN framework. We support this principle with qualitative results (Figures 1A, 1C, 4, 5, and 6) and quantitative results (Tables 1–6). To our knowledge, no prior work studies how visual evidence should be organized and selected at each stage of the LCVN pipeline. We have clarified these distinctions further in the revised Introduction and Related Work (page 1-3).


[1] Wang, Jiahui, and Changhao Chen. "DynaNav: Dynamic Feature and Layer Selection for Efficient Visual Navigation." Advances in Neural Information Processing Systems 38 (2026): 81684-81711.

[2] Shen, Wangtian, et al. "EffoNAV: An effective foundation-model-based visual navigation approach in challenging environment." IEEE Robotics and Automation Letters 10.7 (2025): 6824-6831.

[3] Sridhar, Ajay, et al. "Nomad: Goal masked diffusion policies for navigation and exploration." 2024 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2024.

[4] Shah, Dhruv, Błażej Osiński, and Sergey Levine. "Lm-nav: Robotic navigation with large pre-trained models of language, vision, and action." Conference on robot learning. pmlr, 2023.

AuthorsOpenReview ↗

Response to Reviewer r458 - Part 2

W2&Q1: Weak performance on diffusion-based baselines and stronger baselines.

Although several recent baselines are included, the strongest baseline in Table 2 remains ViNT (CoRL 2023). Comparisons with more competitive recent methods would further strengthen the evaluation.

Thanks for your question, Our evaluation includes several recent visual navigation methods. Table 2 in the main text includes several recent methods (EffoNAV, NaviBridge, and NaviDiffusor, all 2025) covering both transformer-based and diffusion-based policies. Recent visual navigation work mainly either incorporating additional input modalities beyond RGB (e.g., depth, lidar) or using diffusion models to predict multimodal trajectory distributions.

Our work addresses a different bottleneck, which is the robustness and efficientness of visual navigation over long ranges in topological environments. Thanks to its generalizable architectural design, VINT remains the best performing baseline. Furthermore, since DynaNav is not open source, we cannot compare it directly; the comparison results in Table 15 are quoted directly from the paper of DynaNav.

Why do all diffusion-based baselines achieve 0% success in Table 2? The authors mainly attribute these failures to inference latency, but this explanation lacks sufficient experimental evidence. Could the authors provide further analysis?

This is a very interesting question. In fact, similar phenomena have been observed in many recent studies on end-to-end visual navigation, where diffusion-based methods perform worse than transformer-based methods [1–3]. We attribute this to the following two points.

(1) Task setting. All methods in our work use the same velocity limits (1.5 m/s and 0.8 rad/s), which are higher than the common setting in prior image-goal navigation work (0.2 m/s and 0.4 rad/s) because our tasks span much longer distances. This setting requires the model to correct itself promptly when it deviates, but the longer latency of diffusion methods hinders this process. To illustrate, we re-evaluate the diffusion-based baselines in Town02 (SR/SPL) with the limits reduced to 0.2 m/s and 0.4 rad/s, scaling the time budget accordingly. As shown in the table, although the success rate increases after adjusting the setting, it remains far behind the SR of the transformer-based method.

Method1.5m/s, 0.8rad/s0.2m/s, 0.4rad/s
NoMaD0.0/17.26.7/38.2
NaviBridge0.0/19.813.3/41.5
NaviDiffusor0.0/22.318.3/45.7

(2) Limited generalization of diffusion-based action generation. Recent work on manipulation found that diffusion-based action heads perform well on specific tasks but generalize worse than MLP- or transformer-based heads [4]. We believe this conclusion also applies to navigation tasks. In our task, we only finetune the model on a small dataset (30 minutes in Town01), and there remains a large gap between training and testing scenarios (Figure 8). With limited data, diffusion-based methods may quickly fit to the specific scenario bias while losing general navigation skills.

We have added the relevant discussion to the main text (page 8). Thank you again for your question, which has prompted us to think further about this phenomenon.


[1] Zhou, Changqing, et al. "UniNav: A Unified World-Action Diffusion Model for Visual Navigation." arXiv preprint arXiv:2608.03244 (2026).

[2] Guerrier, Maeva, et al. "Can vision foundation models navigate? zero-shot real-world evaluation and lessons learned." arXiv preprint arXiv:2603.25937 (2026).

[3] Shen, Wangtian, et al. "EffoNAV: An effective foundation-model-based visual navigation approach in challenging environment." IEEE Robotics and Automation Letters 10.7 (2025): 6824-6831.

[4] Ye, Jinhui, et al. "StarVLA-α: Reducing complexity in vision-language-action systems." European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2026.

AuthorsOpenReview ↗

Response to Reviewer r458 - Part 3

W3:Ablation on action-grounded retrieval.

The action-grounded retrieval module is a key contribution, but its individual design choices are not sufficiently ablated.

Thank you for this question. We further refined the ablation experiments related to action-grounded retrieval and added the relevant results to following Table. The table shows retrieval accuracy under different combinations.

First, Dynamic Programming (DP) enhance retrieval accuracy by selecting nodes that are reachable and spatially nearby.

Second, action grounding is essential, it leverages directional cues from the instruction to disambiguate semantically similar landmarks, which DP alone cannot guarantee—matching the retrieved node to the location described in the instructions. However, action grounding in isolation may converge to locally optimal solutions for individual sub-instructions, losing sight of the global instruction goal.

GRONav unifies the two by combining semantic matching, spatial consistency, and action grounding, thereby balancing local disambiguation with global coherence and achieving the best performance. The ablation experiments and discussion for this section have been added to the appendix (page 18).

MethodTown02Town03Town10Avg.
CLIP only50.037.537.541.7
CLIP+DP&Regulation (used in LM-Nav)66.758.350.058.3
CLIP+action grounding91.783.379.284.7
GRONav (ours)100.087.587.591.7

Q2: More controlled comparison of visual backbones.

Could the authors provide a more controlled comparison of visual backbones? [...] Although the appendix explores different encoders, backbone-matched comparisons would help disentangle the contributions of pretrained visual representations from the proposed architectural designs.

Thank you for your question. We believe we have made every effort to ensure a fair comparison and to rule out the possibility that the performance gains stem from a stronger visual encoder. Below we clarify this point.

GRONav and EffoNAV share the same backbone. The 33% improvement in Table 2 is therefore attributable entirely to our architectural designs, not to a stronger visual backbone. Moreover, the ablation study in Table 3 isolates architectural contributions under a fixed backbone. The "Baseline" variant in Table~3 uses the same DINOv2+EfficientNet encoder as GRONav but removes both the gating mechanism and spatial alignment. The progression from Baseline (36.1%) to +Gate (56.4%) to +Spatial Align (60.8%) to Full GRONav (89.4%) holds the backbone fixed and disentangles the contribution of each design, which is exactly the controlled comparison the reviewer suggests.

ViNT backbone swap is structurally infeasible. ViNT fuses the current and goal images by concatenating them along the channel dimension (6-channel input) before encoding, which is incompatible with the DINOv2 encoder (pretrained on 3-channel RGB). Changing this fusion mechanism would deviate from ViNT's own architecture. We therefore keep ViNT's original EfficientNet-B0 backbone, and we believe the comparison is as fair as the model designs allow.


Q3: Metric clarify.

Could the authors clarify the SPL values in Table 11? Several reported SPL values exceed the corresponding success rates, which is theoretically impossible under the standard SPL definition.

Thank you for your careful review. We would like to clarify that there is a typo in Table 11: SPL is not suitable in this paper, since the navigation models follow trajectories in the topological map. The correct metric should be Progress Rate (PR), which is used throughout this work. We believe PR better reflects the models' navigation ability even when they do not reach the final goal. We have revised this in the updated version (page 19).

Thanks for your response. I will raise my score to 3.

Reviewer_NMZdOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes GRONav, a topology-based framework for long-range language-conditioned visual navigation that integrates action-grounded retrieval, semantic-guided keyframe selection, and spatially grounded navigation. Experiments in CARLA and real-world environments demonstrate improvements in retrieval accuracy, navigation success, and map compression efficiency.

Critical strengths
  1. Empirical performance and real-world validation. Within the reported experimental settings, GRONav achieves substantial performance gains in both simulation and real-world deployment. The real-robot evaluation strengthens the practical relevance of the system.
  2. Overall readability and discussion of limitations. The manuscript is generally clear, and Appendix E presents a balanced and objective discussion of the current limitations.
Critical weaknesses
  1. The framing and presentation of the contributions could be more rigorous. The Introduction contains numerous evaluative descriptions of GRONav, but the first three paragraphs do not establish a sufficiently clear causal connection to the discussion beginning in the fourth paragraph. The opening descriptions in the Abstract (lines 011–013), Introduction (lines 031–034), and the “Topology-based language-conditioned visual navigation” subsection of Related Work (lines 107–111) are also repetitive. Furthermore, the contribution statements are not clearly differentiated: contributions 2 and 3 largely divide the three complementary mechanisms introduced in lines 075–084, while all three contributions focus on components of GRONav. Finally, the discussion of prior work in “Visual appearance in long-range navigation” could be more objective and coherent. Its tone and logical organization differ noticeably from the preceding subsection, making it difficult to obtain a clear picture of the current research landscape.
  2. The central methodological contribution and its supporting evidence need clearer emphasis. SKS appears to be the most distinctive methodological component, and targeted experiments would better demonstrate how its design supports the central principle proposed in the Introduction. By comparison, the high-level planner and low-level controller largely follow the established hierarchical paradigm of high-level planning and low-level execution in VLN, with their contributions appearing primarily engineering-oriented. Overall, the novelty seems to lie mainly in the incremental integration of these components.
Questions for authors
  1. SKS implementation: SAM 2 is a category-agnostic segmentation model, whereas SKS describes its outputs as semantic categories. How are semantic category labels assigned, and how are overlapping masks handled?
  2. Qwen3 configuration: Please specify the parameter scale (in billions) of the Qwen3 model used and whether it was run in instruct or thinking mode.
  3. Evaluation setting: What is the rationale for training on four real-world datasets and evaluating in three selected CARLA towns? Please clarify the distribution of semantic categories across these scenes and whether they exhibit the appearance gaps described in the Introduction. If such gaps are present, visual examples would help convey the motivation and the value of GRONav.
5 comments
AuthorsOpenReview ↗

Response to Reviewer NMZd - Part 1

W1: Adjustment to writing and presentation.

Thank you for pointing these out. We provide the following explanations for each point, and will make uniform changes to these sections in the revised version.

The framing and presentation of the contributions could be more rigorous. The Introduction contains numerous evaluative descriptions of GRONav, but the first three paragraphs do not establish a sufficiently clear causal connection to the discussion beginning in the fourth paragraph.

The first three paragraphs using examples listed in Figure 1 to show our central principle that visual appearance should be grounded according to the decision it supports, rather than treated as a universal similarity signal. GRONav is proposed based on this principle, which is also the core of our paper: for long-range LCVN tasks, visual appearance should be grounded to different representations in different parts of the framework.

We rewrote paragraphs 3 and 4 so that each mechanism follows from the failure it addresses. For each stage, we first identify a signal that stays informative where appearance does not: trajectory-level action for retrieval, semantic composition for mapping, and geometry for control. We then introduce the corresponding mechanism. We also removed evaluative descriptions of GRONav from the Introduction (page 1-2).

The opening descriptions in the Abstract, Introduction, and the “Topology-based language-conditioned visual navigation” subsection of Related Work are also repetitive.

The similarity in these three sections is to provide readers with a basic understanding of LCVN. In the abstract, we introduce LCVN to clarify the scope of our task, and the same applies in the introduction, leading to the example in Figure 1. The reiteration in the related works section is to distinguish it from VLN-type tasks. We have adjusted the relevant content, the Related Work subsection starts directly from LM-Nav and its follow-ups (page 3).

Furthermore, the contribution statements are not clearly differentiated: contributions 2 and 3 largely divide the three complementary mechanisms introduced in lines 075–084, while all three contributions focus on components of GRONav.

Thank you for this helpful comment. We agree that the original contribution statements did not clearly distinguish their respective roles. We have revised them to better reflect the progression of our work. The first contribution highlights our key insight: different stages of long-range LCVN require different types of visual evidence, and these requirements are interdependent. The second describes how we translate this insight into GRONav through three complementary grounding mechanisms. The third demonstrates the effectiveness of the overall framework through comprehensive simulation and real-world evaluations. In this way, the contributions separately emphasize the underlying insight, the proposed methodology, and its experimental validation. The revised contribitons are as below (also revised in page 2).

  1. We identify the distinct yet interdependent requirements for visual evidence in long-range LCVN and propose GRONav, a unified topology-based navigation framework that grounds visual evidence according to the decision-specific retrieval, mapping, and navigation.
  2. We develop three complementary mechanisms: action-grounded retrieval that leverages trajectory context to disambiguate targets, semantic-grounded keyframe selection that preserves instruction-critical landmarks, and spatially grounded navigation that exploits geometric features for robust waypoint prediction.
  3. We conduct comprehensive evaluations in CARLA and in real-world quadrupedal robot deployments, demonstrating consistent improvements over strong baselines in target retrieval accuracy, topological map compression, navigation performance, and inference efficiency.

Finally, the discussion of prior work in “Visual appearance in long-range navigation” could be more objective and coherent. [...]

We reorganized "Visual appearance in long-range navigation" into three lines of prior work: retrieval disambiguation, map construction, and image-goal navigation policies. For each line, we describe what existing methods do and what they leave open (see page 3). Thank you for your advise.

AuthorsOpenReview ↗

Response to Reviewer NMZd - Part 2

W2: Central contribution and novelty of the work.

Overall, the novelty seems to lie mainly in the incremental integration of these components.

Thank you for this comment. We first address the concern about incremental integration, then reply to the two points above.

We agree that the methodological novelty of GRONav and its distinctions from existing approaches should be articulated more explicitly. Our contribution is not to claim that individual techniques such as visual-language matching, keyframe selection, or attention mechanisms are new in isolation. Rather, we identify decision-specific visual evidence as a central consideration in long-range LCVN and develop three complementary mechanisms to address the distinct visual evidence requirements of target retrieval, map construction, and navigation.

(1) Action-grounded retrieval. Unlike LM-Nav [1], which primarily matches language-described landmarks to graph nodes using CLIP similarity, GRONav explicitly aligns instructed action directives with motion signatures recovered from candidate graph trajectories using DPVO and LCS matching. This introduces trajectory-level evidence to disambiguate visually similar targets.

(2) Spatially grounded navigation. Unlike DynaNav [2], EffoNAV [3], and NoMaD [4], which focus on navigation efficiency, foundation-model-based visual representations, and diffusion-based control, respectively, GRONav transfers geometric priors from a frozen DepthAnythingV2 model through feature alignment and gated attention, improving spatial awareness and efficient waypoint prediction.

(3) Semantic-grounded map compression. Unlike conventional visual-change-based keyframe selection methods, such as ORB-SLAM and AKS, GRONav combines adaptive visual variation with object-level semantic entropy from SAM 2, preserving instruction-relevant landmarks even when visual changes are small.

Beyond these individual mechanisms, GRONav integrates them into a unified long-range LCVN framework. We support this principle with qualitative results (Figures 1A, 1C, 4, 5, and 6) and quantitative results (Tables 1–6). To our knowledge, no prior work studies how visual evidence should be organized and selected at each stage of the LCVN pipeline. We have clarified these distinctions further in the revised Introduction and Related Work (page 1-3).


[1] Shah, Dhruv, Błażej Osiński, and Sergey Levine. "Lm-nav: Robotic navigation with large pre-trained models of language, vision, and action." Conference on robot learning. pmlr, 2023.

[2] Wang, Jiahui, and Changhao Chen. "DynaNav: Dynamic Feature and Layer Selection for Efficient Visual Navigation." Advances in Neural Information Processing Systems 38 (2026): 81684-81711.

[3] Shen, Wangtian, et al. "EffoNAV: An effective foundation-model-based visual navigation approach in challenging environment." IEEE Robotics and Automation Letters 10.7 (2025): 6824-6831.

[4] Sridhar, Ajay, et al. "Nomad: Goal masked diffusion policies for navigation and exploration." 2024 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2024.


[...] SKS appears to be the most distinctive methodological component, and targeted experiments would better demonstrate how its design supports the central principle proposed in the Introduction.

SKS applies the central principle to map construction. Visual-change selection keeps frames where appearance changes sharply, such as turns and junctions, and treats frames on straight segments as redundant. The landmarks named in instructions, such as a bus stop or a gas station along a straight road, often appear in exactly these "redundant" frames. Visual salience therefore does not imply navigational relevance. The map is built before any instruction is given, so the selection criterion cannot depend on the instruction. SKS instead uses semantic entropy as instruction-agnostic evidence of landmark richness, and combines it with the visual-change criterion.

Existing keyframe selection methods come from SLAM and 3D reconstruction, and they measure visual novelty. Topological navigation systems such as ViNT and LM-Nav usually subsample the traversal at fixed intervals. In contrast, SKS is the method that brings semantic evidence into topology compression for LCVN. It runs online during map construction and works with any navigation policy.

To further illustrate, We added an ablation, SKS w/o semantics, which removes only the semantic-entropy criterion and keeps the compression ratio matched, the results showing that semantic evidence is what preserves retrieval-critical landmarks. The ablation experiments for this section have been added to the appendix (page 18-19).

SceneAKSORBUniformSKS (w/o semantic)SKS
Town0262.562.587.562.587.5
Town0375.050.062.570.887.5
Town1087.575.075.087.575.0
Avg.75.062.575.073.683.3
AuthorsOpenReview ↗

Response to Reviewer NMZd - Part 3

By comparison, the high-level planner and low-level controller largely follow the established hierarchical paradigm of high-level planning and low-level execution in VLN, with their contributions appearing primarily engineering-oriented.

We thank the reviewer for raising this point. GRONav does share the two-level structure of hierarchical VLN, with planning above and control below. The two paradigms differ in how the levels are built, and this difference is why we study visual evidence at each level.

Difference from VLN. VLN models are typically trained end to end: language and observations are fused inside one network, and the plan stays implicit in its features. Such models are usually trained and evaluated on prebuilt indoor scans [1-2], which makes them hard to scale to environments spanning hundreds of meters (Appendix D). GRONav instead plans explicitly on a topological graph built online from one traversal. Each retrieval decision can be traced to the landmark and action evidence behind it, and the image-goal policy receives no language input. Because the modules are decoupled, each can be replaced and evaluated on its own, which lets us compare retrieval, mapping, and navigation module by module (Tables 1–5).

Novelty of the architecture. The explicit graph and the language-free policy give GRONav a property that end-to-end VLN lacks: each level makes its decision from visual evidence alone. This lets the architecture match the evidence to the decision at each level, using trajectory-level action evidence on the graph for retrieval and geometric evidence for control, together with semantic evidence for mapping. Our novelty is therefore an architecture that varies the visual evidence it relies on by stage, which no prior LCVN or VLN framework does, rather than the two-level split itself.

We have incorporated the relevant content into the first paragraph of Related works to help readers better understand the differences between the two approaches (page 3). Thank you for your suggestion.


Q1: Category labels and overlapping masks in SKS.

SKS implementation: SAM 2 is a category-agnostic segmentation model, whereas SKS describes its outputs as semantic categories. How are semantic category labels assigned, and how are overlapping masks handled?

Thank you for your question. (1) SKS does not use category labels. We use SAM 2 only to partition the image into regions and use the pixel area of each mask. Semantic entropy measures how evenly the image splits into object-level regions. The criterion is label-free and independent of the instruction, since the map is built before any instruction is given. (2) Each pixel is assigned to at most one mask, so masks do not overlap; pixels not covered by any mask form the background term in Eq. (8).

Thank you again for your question. the relevant content mentioned above has been added to Appendix A (page 15) and we have revised Section 3.3 to replace "object categories" with "segmented regions" and state that masks are non-overlapping.


Q2: Details on LLM model.

Qwen3 configuration: Please specify the parameter scale (in billions) of the Qwen3 model used and whether it was run in instruct or thinking mode.

Thank you for your question. We use Qwen3-VL-4B-Instruct (non-thinking mode) for both instruction decomposition and the CLIP+Qwen3 baseline. We have added these details to Appendix A (page 15).

Reviewer_j4ngOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes GRONav, a unified topology-based framework for long-range language-conditioned visual navigation (LCVN). The framework aims to address three challenges: ambiguous landmark retrieval, loss of important semantic information during map compression, and accumulated navigation errors caused by appearance variations. GRONav introduces three components: (1) action-grounded retrieval, which combines visual-semantic similarity with trajectory-based action matching; (2) spatially grounded navigation, which incorporates depth feature alignment and gated attention to improve waypoint prediction; and (3) semantic-grounded keyframe selection, which preserves visually distinctive and semantically informative observations in compressed topological maps. Experiments in CARLA and on a real-world quadrupedal robot show improvements in retrieval accuracy, navigation success rate, and map compression efficiency over the evaluated baselines.

Critical strengths
  • This paper presents a well-motivated framework for long-range language-conditioned visual navigation. By addressing the limitations of appearance-based representations across retrieval, mapping, and navigation, GRONav provides a unified approach to several important challenges in long-range navigation. This integration is practically meaningful for real-world robotic applications.

  • This paper provides promising experimental results in both simulation and real-world environments. The evaluation covers target retrieval, navigation performance, map compression, and component ablation. The reported improvements over the evaluated baselines, together with the real-world quadrupedal robot experiments, provide encouraging evidence of the framework's practical effectiveness.

Critical weaknesses
  • The contribution of action-grounded retrieval is not sufficiently validated. In Section 4.2, Table 1 shows substantial improvements over CLIP and CLIP+Qwen3, but does not isolate the effect of trajectory-based action matching from graph search and distance regularization. Since action-grounded retrieval is a central contribution of the paper, the absence of a controlled ablation makes it difficult to determine how much of the reported improvement comes from the proposed action-matching mechanism.

  • The effectiveness of semantic entropy in keyframe selection is not sufficiently established. In Section 3.3, the authors introduce semantic entropy to preserve informative landmarks during map compression. However, high semantic entropy does not necessarily indicate relevance to navigation instructions. Although Table 4 demonstrates the effectiveness of the overall SKS method, it does not isolate the contribution of semantic entropy from visual-change-based selection. This limits the evidence supporting the necessity of the proposed semantic criterion.

Questions for authors

In Section 3.1, Equation (2) considers only the highest-scoring node from the previous phase, whereas Appendix A specifies a beam width of five nodes. Could the authors clarify how multiple candidate states are maintained during the search?

Could the authors provide an ablation comparing visual-change-based keyframe selection with and without semantic entropy under the same compression ratio? This would help establish the independent contribution of the semantic criterion.

Could the authors provide an ablation study to isolate the contribution of trajectory-based action matching from graph search and distance regularization in target retrieval? This would help clarify the effectiveness of the proposed action-grounded retrieval mechanism.

2 comments
AuthorsOpenReview ↗

Response to Reviewer j4ng

Q1: Clarification of Equation (2).

In Section 3.1, Equation (2) considers only the highest-scoring node from the previous phase, whereas Appendix A specifies a beam width of five nodes. Could the authors clarify how multiple candidate states are maintained during the search?

Thank you for this question. the implementation details of High-level Retrieval are as below.

For each sub-instruction iiin the multipart language instructions, we first select the 20 nodes with highest CLIP similarity to the landmark description did_i, compute retrieval scores via Equation (1), and retain the top b=5b=5 nodes as the beam B(i)\mathcal{B}^{(i)} for the next phase. The dynamic programming then considers all transitions from any node in the previous beam B(i−1)\mathcal{B}^{(i-1)} to each candidate node vv in the current phase using Equation (2), selecting the best transition. Finally, we backtrack to recover the complete goal-node trajectory. We revise Equation (2) to explicitly reflect this multi-node beam transition: Q(i)(v)=max⁡u∈B(i−1)[Q(i−1)(u)+R(v,di,ai)−α⋅D(v,u)],Q^{(i)}(v) = \max_{u \in \mathcal{B}^{(i-1)}} \left[ Q^{(i-1)}(u) + R(v, d_i, a_i) - \alpha \cdot D(v, u) \right], where D(v,u)D(v, u) is the graph distance between vv and each beam node uu using Dijkstra algrothim. We have clarified this process and modified the main equation in the main text (page 4) and the appendix (page 15). Thanks again for your question.


W1&Q3: Ablation on action-grounded retrieval.

The contribution of action-grounded retrieval is not sufficiently validated. [...], the absence of a controlled ablation makes it difficult to determine how much of the reported improvement comes from the proposed action-matching mechanism.

Thank you for this question. We further refined the ablation experiments related to action-grounded retrieval and added the relevant results to following Table. The table shows retrieval accuracy under different combinations.

First, Dynamic Programming (DP) enhance retrieval accuracy by selecting nodes that are reachable and spatially nearby.

Second, action grounding is essential, it leverages directional cues from the instruction to disambiguate semantically similar landmarks, which DP alone cannot guarantee—matching the retrieved node to the location described in the instructions. However, action grounding in isolation may converge to locally optimal solutions for individual sub-instructions, losing sight of the global instruction goal.

GRONav unifies the two by combining semantic matching, spatial consistency, and action grounding, thereby balancing local disambiguation with global coherence and achieving the best performance. The ablation experiments and discussion for this section have been added to the appendix (page 18).

MethodTown02Town03Town10Avg.
CLIP only50.037.537.541.7
CLIP+DP&Regulation (used in LM-Nav [1] )66.758.350.058.3
CLIP+action grounding91.783.379.284.7
GRONav (ours)100.087.587.591.7

W2&Q2: Ablation on semantic-grounded keyframe selection.

The effectiveness of semantic entropy in keyframe selection is not sufficiently established. [...] Although Table 4 demonstrates the effectiveness of the overall SKS method, it does not isolate the contribution of semantic entropy from visual-change-based selection. [...]

Thank you for this question. Semantic-Grounded Keyframe Selection (SKS) jointly considers both visual-change criterion and semantic entropy. The visual-change-based methods (AKS and ORB) represent the case where only the visual criterion is retained. We further supplemented our ablation experiments with the SKS without semantic groudning, and its retrieval success rate compared to other baselines at the same compression ratio is shown in the table below. In terms of average Acc, SKS without semantics shows a significant drop in retrieval performance, indicating that semantics are crucial for task retrieval. The ablation experiments and discussion for this section have been added to the appendix (page 18-19).

SceneAKSORBUniformSKS (w/o semantic)SKS
Town0262.562.587.562.587.5
Town0375.050.062.570.887.5
Town1087.575.075.087.575.0
Avg.75.062.575.073.683.3

[1] Shah, Dhruv, Błażej Osiński, and Sergey Levine. "Lm-nav: Robotic navigation with large pre-trained models of language, vision, and action." Conference on robot learning. pmlr, 2023.

The authors have addressed most of my concerns through additional experiments and clarifications, so I will maintain my original score as 3.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response to All Reviewers

Dear all,

We thank all reviewers for their constructive feedback and for clear motivation and meaningful problem (j4ng, r458), the overall readability (NMzd, r458), and the strong performance across each component of our work (j4ng, NMzd, r458). We have addressed all concerns raised by the reviewers. In response to this feedback, we summarize our main clarifications, additional experiments, and implementation details below:

  1. (j4ng, NMZd, r458) More ablation studies. We supplemented our ablation experiments on the Action-Grounded Retrieval and Semantic-Grounded Keyframe Selection components to further demonstrate the effectiveness of our improvements.
  2. (j4ng, NMZd) Implementation details. We clarified the deployment details of Action-Grounded Retrieval, the usage of SAM and Qwen3, and the necessity of deploying them in the CARLA simulation.
  3. (NMZd, r458) Clarification of novelty. We emphasize our innovation at the framework level—better integrating visual input with different representations at different stages to achieve long-range LCVN tasks.
  4. (NMZd) Writing and presentation. We revised several statements based on the reviewer' suggestions to make them easier for readers to understand.
  5. (r458) Comparison on baselines. We have tried our best to ensure fair comparisons and eliminate potential problems caused by different visual encoders. For some baseline performance issues, we have provided possible reasons from an experimental perspective.

We have revised the manuscript, with all changes highlighted in blue throughout the main text and appendix. We hope these revisions address the reviewers' concerns and clarify our contributions. A detailed point-by-point response follows. We welcome further suggestions and appreciate the opportunity to improve our work.

Best Regards,

Submission 942 authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.