acceptodds
Under review as a conference paper at ICLR 2027

SeeNav-Agent: Enhancing Vision-Language Navigation with Visual Prompt and Step-Level Policy Optimization

Abstract

Existing Vision-Language Navigation (VLN) agents based on Large Vision-Language Models (LVLMs) often suffer from errors of perception, reasoning, and planning, which significantly hinder their navigation performance. To address these limitations, a novel VLN agent framework, named SeeNav-Agent, is proposed. First, to reduce hallucinations of the visual module of the VLN agent, a dual-view Visual Prompt (VP) technique is introduced, which can also improve the agent’s understanding of current spatial states. Subsequently, a novel step-level Reinforcement Fine-Tuning (RFT) method, Step Reward Group Policy Optimization (SRGPO), is designed for post-training. In SRGPO, we first define verifiable process rewards for navigation, and then perform efficient step-level advantage estimation by randomly grouping navigation steps. SRGPO provides dense reward signals for the RFT of the VLN agent and enhances its planning capability. Experiments on both the EmbodiedBench Navigation (Embench-Nav) and VLN-CE validate the effectiveness of the proposed method. By introducing the zero-shot VP module, the GPT-4.1 achieves a navigation success rate of 86.7%, surpassing the current best LVLM by approximately 20 percentage points (pp) on Embench-Nav and achieves an 11.1 pp improvement on VLN-CE compared to the base model. With VP+SRGPO, the Qwen2.5-VL-3B model reaches a navigation success rate of 72.3%, outperforming the best existing LVLM model by 5.6 pp on Embench-Nav and achieves 8 pp improvement on VLN-CE compared to the baseline method. Moreover, compared to GRPO and GiGPO, the proposed SRGPO demonstrates significant improvements in training stability, convergence efficiency, and generalization capability.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Reinforcement Learning, Vision-Language Navigation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_s5yGOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes SeeNav-Agent, combining dual-view visual prompting with step-level reinforcement fine-tuning for vision-language navigation. Visual prompts add target bounding boxes, navigation lines, and action markers to frontal and bird’s-eye views, while SRGPO normalizes step rewards within randomly sampled cross-task groups and combines them with episode-level advantages for policy optimization. GPT-4.1+VP achieves 86.7% SR on Embench-Nav, and Qwen2.5-VL-3B+VP+SRGPO achieves 72.3%/39.1% SR on Embench-Nav/R2R-CE.

Critical strengths

1. The visual-prompt ablations demonstrate how presentation affects the use of additional spatial information.

In Table 2, adding dual views and view alignment alone reduces SR from 65.0% to 45.0%, whereas the complete prompting configuration achieves 86.7%. This provides useful empirical insights into navigation interface design in settings where privileged information is available.

2. SRGPO substantially outperforms a baseline using the same process rewards.

Compared with VP+GiGPO using the same VPR, VP+SRGPO improves SR by 15.1/11.0 percentage points on Embench-Nav/R2R-CE. Grouping-strategy ablations and cross-scene evaluations further support the training scheme’s effectiveness in the evaluated settings.

Critical weaknesses

1. The method relies on privileged spatial and target information at test time, without establishing its practical availability or comparison fairness.

The authors directly use simulator ground truth to construct bird’s-eye views and target boxes, together with agent poses to draw target-direction lines, providing target-location cues even when the target leaves the frontal view. On the one hand, this substantially limits its practical applicability, as such information is often unavailable in real-world settings. On the other hand, since the main baselines are not shown to receive equivalent information, the reported gains cannot separate the contribution of visual prompting from the advantage of additional ground-truth information, limiting the conclusions for conventional VLN settings.

  1. SRGPO’s central justification conflates consistent reward semantics with comparable action values across states.

Equation (1) rewards immediate distance reduction or restored target visibility, but necessary detours may receive zero reward while repeatedly looking away and reacquiring the target may receive repeated positive rewards, so these signals do not directly represent actions’ long-term value across states. The paper neither analyzes these biases nor compares against policy optimization using the same dense rewards with ordinary batch normalization, leaving the algorithmic contribution of random grouping beyond reward shaping unestablished.

Questions for authors

1..How much benefit does visual prompting provide after controlling for access to ground-truth information? Please compare against baselines receiving equivalent spatial and target information, and evaluate performance when predicted BEVs, detected boxes, or noisy poses replace ground truth.

2.How does the process reward handle necessary detours and repeated recovery of target visibility? Please specify clarify whether the step-level reward conflicts with eventual task success in these cases.

3.What is the independent contribution of random grouping over ordinary reward normalization? Under identical inputs, process rewards, and environment-interaction budgets, does SRGPO consistently outperform full-batch reward normalization or standard return estimation?

Reviewer_agpTOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes SeeNav-Agent, a vision-language navigation framework combining dual-view visual prompting (VP) and Step Reward Group Policy Optimization (SRGPO). The visual prompting module enhances spatial understanding through front-view and bird’s-eye-view observations with additional visual annotations. SRGPO introduces step-level process rewards and random step grouping to improve reinforcement fine-tuning. Experiments on Embench-Nav and VLN-CE demonstrate substantial improvements over several baselines.

Critical strengths
  1. The paper addresses important perception and planning challenges in vision-language navigation.
  2. The experimental results show substantial improvements across two simulation benchmarks.
  3. The authors provide useful ablations and comparisons with GRPO and GiGPO, demonstrating the empirical effectiveness of SRGPO.
Critical weaknesses

W1. Limited novelty of visual prompting and reliance on privileged information. The proposed VP module mainly combines existing techniques, including BEV representations, bounding boxes, navigation arrows, and action projection. Similar visual prompting ideas have been explored in prior works such as PIVOT and VLMnav. The methodological novelty of this combination appears limited.More importantly, the BEV and target bounding boxes are obtained directly from the simulator’s internal state. Navigation arrows further provide explicit target-direction information. These signals are not directly available to a real robot without additional perception and localization modules. Consequently, the substantial performance improvements may largely result from privileged spatial information rather than improved visual reasoning capabilities. The paper lacks experiments using realistically estimated visual prompts or controlled comparisons under equivalent information access.

W2. Insufficient theoretical justification for cross-state advantage estimation in SRGPO. The central motivation of SRGPO is that state-independent process rewards allow navigation steps from different states to be randomly grouped for advantage estimation. However, a consistent reward definition across states does not imply that normalized immediate rewards provide valid estimates of state-dependent action advantages. In particular, the proposed reward encourages reducing the Euclidean distance to the goal or recovering target visibility. Such local progress does not necessarily align with long-horizon navigation success, especially when obstacle avoidance requires temporarily moving away from the target. Although the empirical improvements are encouraging, the theoretical justification for cross-state grouping remains insufficient. Additional comparisons with batch-level reward normalization and alternative advantage estimators would help establish the necessity of the proposed design.

W3. Lack of real-world validation. All experiments are conducted in simulation. Given the reliance on privileged visual information, it remains unclear whether the proposed framework can maintain its effectiveness in realistic robotic navigation settings with imperfect perception and localization.

Questions for authors
  1. How much of the VP improvement comes from privileged spatial information rather than visual prompting itself?
  2. Can the authors justify why randomly normalized step rewards provide meaningful action advantages across different states?
  3. How does SRGPO perform on navigation tasks requiring temporary movement away from the goal?
  4. Can the authors evaluate VP using predicted BEV and object detections instead of simulator ground truth?
Reviewer_QLUUOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

To address perception, reasoning, and planning errors in visual-language navigation, this paper proposes SeeNav-Agent, which combines dual-view visual cues with step-level reinforcement fine-tuning. The former assists navigation through target labeling, action projection, and viewpoint alignment, while the latter randomly groups navigation steps using SRGPO and optimizes the strategy by combining process rewards and final result rewards. Significant performance improvements are achieved on Embench-Nav and VLN-CE.

Critical strengths

The methodology is clearly structured, combining visual input assistance with step-level strategy optimization to improve navigation for different error sources.

It achieves significant improvements on two navigation benchmarks and provides component ablation, GiGPO comparisons of process rewards, and cross-scenario generalization experiments.

Critical weaknesses
  1. The innovation of dual-view visual cues is not yet prominent. The method mainly combines target bounding boxes, navigation lines, action projections, and viewpoint alignment, lacking direct comparisons with existing visual cues methods, as well as single-view cues that demonstrate the necessity of dual-view functionality.

  2. Additional environmental information affects performance attribution and the fairness of comparisons. BEV and target bounding boxes are directly derived from the simulator, and it is unclear whether the baseline receives equivalent information; how target markers and navigation lines are constructed in VLN-CE, and whether endpoint information is used, also needs clarification.

  3. The basis for cross-state advantage comparisons is insufficient. A unified meaning of rewards does not mean that action advantages in different states can be directly compared; necessary detours may not yield local progress rewards, and the paper has not fully verified the consistency of such rewards with long-term navigation goals.

  4. The independent contributions of random grouping are not adequately isolated. Although grouping range ablation exists, there is a lack of comparisons with simple schemes such as full-batch normalization and direct use of step-level rewards under the same rewards, making it difficult to determine whether the gains truly come from random grouping.

  5. There is a lack of direct quantitative evidence for improvements in core capabilities. The improved success rate and individual cases are insufficient to fully demonstrate the reduction of visual illusions and the enhancement of spatial reasoning and planning capabilities. Further statistical analysis of error types and diagnostic experiments targeting complex detours and long-distance navigation are needed.

Questions for authors

Please refer to the weaknesses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.