- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces RG-VLN, a new benchmark for Vision-Language Navigation (VLN) where agents must navigate based on underspecified everyday user requests rather than detailed step-by-step route instructions. The benchmark contains over 6,000 episodes across MP3D and HM3D environments, featuring both VLM-generated and human-written requests. Through matched-trajectory experiments, the authors demonstrate that replacing detailed instructions with underspecified requests causes a severe performance drop (68.5%–74.2%) for existing VLN methods. To address this, they propose HiNav, a hierarchical framework consisting of a Request Reasoner (which generates short-horizon textual subtasks from observations and the request) and a VLN Executor (which follows these subtasks). HiNav achieves state-of-the-art results on all RG-VLN splits and is also deployed zero-shot on a physical robot.
Critical strengths
-
Well-designed benchmark and diagnostic analysis: The formulation of "Request Grounding VLN" addresses a realistic and underexplored gap between standard VLN and real-world human-robot interaction. The matched-trajectory intervention (Table 3) and the request-detail density analysis (Table 1) are particularly strong. They convincingly isolate reduced navigation specification—rather than mere linguistic variation—as the primary source of difficulty. This provides valuable insights for the community.
-
Practical and extensible method design: The HiNav framework elegantly decouples high-level reasoning from low-level execution using natural language as an interface. The plug-and-play experiments (Table 4), where the same trained Reasoner improves frozen executors like StreamVLN and DualVLN by large margins, strongly validate the architectural choice. The zero-shot deployment on a Unitree Go2 robot further demonstrates practical relevance.
Critical weaknesses
-
Heavy reliance on VLM-generated data for both benchmark and training: The benchmark requests and the training supervision for HiNav are both synthesized using Qwen3-VL models. Although human evaluation shows comparable quality, this creates a potential bias: HiNav may be implicitly optimized for the specific linguistic patterns and failure modes of the generator VLM. The human-written split (HM3D-Test-Human) is relatively small (243 episodes), making it hard to fully rule out this bias.
-
Questionable proxy metric for navigation information: The "request-detail density" is defined simply as the character/word length of the request divided by the geodesic path length (D(I, τ) = |I|/L(τ)). Text length is a very crude proxy for actual semantic navigation information. A long but vague request could have high density, while a short but highly informative one could have low density. Basing key analytical conclusions (Section 5.2) on this metric weakens the soundness of the diagnosis.
Questions for authors
-
Regarding the request-detail density metric: Did you consider measuring semantic information content (e.g., counting unique landmarks or spatial relations mentioned) instead of relying purely on text length? How sensitive are your conclusions in Table 1 to the choice of this metric?
-
Since both the benchmark construction and the HiNav training data synthesis rely heavily on Qwen3-VL, how much of HiNav's success is due to the model learning the specific generation biases of the VLM? Could you provide an analysis of HiNav's performance specifically on the HM3D-Test-Human split compared to the VLM splits?
-
In the hierarchical learning ablation (Table 5), vanilla fine-tuning performs worse than the frozen baseline. Is this degradation solely due to the task shift disrupting pretrained priors, or could it be related to the limited size/quality of the synthesized training data?
-
For the real-world deployment on the Unitree Go2, what was the inference latency per step, and did the serial coupling between the Reasoner and Executor cause noticeable delays or stuttering during physical navigation?
4 comments
Response to Reviewer ynDh
Q3. Understanding the Degradation of Vanilla Fine-Tuning
Thank you for this insightful question. We agree that the degradation of vanilla fine-tuning may be attributed to both the task shift and the limited scale of synthesized training data.
1. Impact on Pretrained Instruction-Following Capabilities. To investigate whether fine-tuning on underspecified requests interferes with pretrained navigation capabilities, we additionally evaluate the frozen and fine-tuned Uni-NaVid models using the original R2R instructions corresponding to our evaluation trajectories.
| Model | SR↑ | SPL↑ | OSR↑ | NE↓ |
|---|---|---|---|---|
| Frozen Uni-NaVid | 52.1 | 48.1 | 57.0 | 5.08 |
| Vanilla Fine-tuning | 47.7 | 43.2 | 51.8 | 6.54 |
Vanilla fine-tuning reduces SR by 4.4 percentage points and SPL by 4.9 points even on the original R2R instructions, providing evidence that adapting to underspecified request-action supervision can interfere with pretrained instruction-following capabilities.
2. Limited Supervision and Hierarchical Adaptation. Notably, Uni-NaVid was originally trained on approximately 2.4M VLN samples, whereas our task-specific training set contains only 420K samples. Moreover, RG-VLN requires additional request grounding and spatial reasoning beyond conventional instruction following, making direct adaptation particularly challenging under limited supervision. Meanwhile, HiNav achieves 36.4% SR using the same underlying synthesized dataset, compared with 26.7% for vanilla fine-tuning, suggesting that the degradation cannot be attributed to data quality alone. We will further explore scaling up the training data in future work.
Q4. Real-World Inference Latency and Navigation Smoothness
Thank you for this question. The Reasoner and Executor have average inference latencies of 478 ms and 192 ms, respectively. Considering their sequential execution and communication overhead, the overall system incurs an average latency of 457 ms per step.
Regarding navigation smoothness, we did observe occasional motion pauses due to the serial coupling between reasoning and execution. However, these interruptions were relatively minor and did not noticeably disrupt the overall navigation process. We acknowledge this limitation and will explore asynchronous reasoning to further improve navigation smoothness in future work.
Thank you to the authors for the detailed response and additional experiments.
The new evaluations using the semantic-aware density metric (counting unique landmarks and spatial relations) and the cross-generator comparisons (Human vs. Qwen vs. InternVL) address my concerns regarding metric soundness and potential VLM bias. The clarifications on vanilla fine-tuning degradation and real-world inference latency are also helpful.
Please make sure to incorporate these important analyses and the clarifications regarding the evaluation subsets (as discussed with other reviewers) into the final revision. My initial rating is already positive, so I will maintain my score.
We sincerely thank the reviewer for the positive feedback and acknowledgment of our additional experiments and clarifications. We will ensure that all additional analyses and clarifications, including the evaluation subset definitions, are incorporated into the revised manuscript before the discussion phase concludes.
Response to Reviewer ynDh
We sincerely thank the reviewer for the positive assessment of our benchmark design, diagnostic analyses, hierarchical framework, and real-world deployment. We also appreciate the constructive suggestions regarding data-generation bias, the validity of request-detail density, hierarchical learning, and inference efficiency. We address these concerns below.
W1 & Q2. Generalization Beyond VLM-Generated Data
We thank the reviewer for highlighting the potential bias arising from using Qwen3-VL for both benchmark construction and training data synthesis.
To address this concern, we conduct additional evaluations on matched trajectories between HM3D-Test-Human and HM3D-Test-VLM, comparing human-written, Qwen3-VL-generated, and InternVL-generated requests using bootstrap.
HiNav shows only a small performance difference between human-written and Qwen-generated requests, with no statistically detectable degradation. In addition, HiNav maintains comparable performance on Qwen- and InternVL-generated requests. These results provide no clear evidence of Qwen3-VL-specific bias in our evaluations. We will incorporate these additional analyses into the revised manuscript.
W2 & Q1. Semantic-Aware Request Density Analysis
Thank you for this valuable suggestion. We agree that word count is a coarse proxy for navigation information and may not fully capture the semantic content of a request. Following the reviewer's suggestion, we additionally define a semantic-aware density metric based on the numbers of unique landmarks and spatial relations in each request:
Dsem(I,τ)=L(τ)Nlandmark(I)+Nrelation(I),
where Nlandmark and Nrelation denote the numbers of unique landmarks and spatial relations extracted from each request using GPT-OSS-20B, respectively. We re-group the episodes according to this metric and re-evaluate all seven methods on MP3D-Val-VLM. The results are summarized below.
All seven methods consistently exhibit higher success rates as semantic-aware density increases, reproducing the trend observed in Table 1. This indicates that our original observation is robust to the choice of density metric and does not solely depend on request length. We will revise the density definition and update Table 1 accordingly to incorporate this semantic-aware analysis.