acceptodds
Under review as a conference paper at ICLR 2027

RG-VLN: Benchmarking Underspecified Request Grounding in Vision-Language Navigation

Abstract

Vision-language navigation (VLN) and embodied navigation benchmarks span specified goals, route instructions, target descriptions, and user intentions. In everyday interaction, users may express an underspecified request that combines intent, multiple entities, and partial spatial cues while providing incomplete navigation information. Fulfilling such requests jointly requires target grounding, spatial reasoning, and route planning from the request and online observations. We formulate this complementary setting as Request Grounding VLN. To bridge this research gap, we introduce RG-VLN, a benchmark with over 6,000 episodes across MP3D and HM3D scenes, with both VLM-generated and human-written requests. RG-VLN associates trajectories with detailed route instructions, route-preserving paraphrases, and underspecified requests containing progressively less route information, thereby enabling controlled analysis of navigation specification. Replacing detailed instructions with underspecified requests reduces the success rates of representative VLN methods by 68.5%–74.2%, whereas route-preserving paraphrases cause only minor changes. Performance is also consistently lower for requests with lower navigation-detail density, indicating that reduced navigation specification is a major source of difficulty. Building on these findings, we further propose HiNav, a hierarchical framework that connects request reasoning with navigation execution through closed-loop, scene-grounded textual guidance. This interface resolves request ambiguity from observations and enables existing VLN policies to serve as plug-and-play executors. It achieves the highest success rate across all RG-VLN splits, outperforming the strongest baseline by 7.2%–21.2% relatively. Code and datasets will be publicly released.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Vision-Language Navigation, Embodied Navigation, Request Grounding, Underspecified Requests

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_ynDhOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces RG-VLN, a new benchmark for Vision-Language Navigation (VLN) where agents must navigate based on underspecified everyday user requests rather than detailed step-by-step route instructions. The benchmark contains over 6,000 episodes across MP3D and HM3D environments, featuring both VLM-generated and human-written requests. Through matched-trajectory experiments, the authors demonstrate that replacing detailed instructions with underspecified requests causes a severe performance drop (68.5%–74.2%) for existing VLN methods. To address this, they propose HiNav, a hierarchical framework consisting of a Request Reasoner (which generates short-horizon textual subtasks from observations and the request) and a VLN Executor (which follows these subtasks). HiNav achieves state-of-the-art results on all RG-VLN splits and is also deployed zero-shot on a physical robot.

Critical strengths
  1. Well-designed benchmark and diagnostic analysis: The formulation of "Request Grounding VLN" addresses a realistic and underexplored gap between standard VLN and real-world human-robot interaction. The matched-trajectory intervention (Table 3) and the request-detail density analysis (Table 1) are particularly strong. They convincingly isolate reduced navigation specification—rather than mere linguistic variation—as the primary source of difficulty. This provides valuable insights for the community.

  2. Practical and extensible method design: The HiNav framework elegantly decouples high-level reasoning from low-level execution using natural language as an interface. The plug-and-play experiments (Table 4), where the same trained Reasoner improves frozen executors like StreamVLN and DualVLN by large margins, strongly validate the architectural choice. The zero-shot deployment on a Unitree Go2 robot further demonstrates practical relevance.

Critical weaknesses
  1. Heavy reliance on VLM-generated data for both benchmark and training: The benchmark requests and the training supervision for HiNav are both synthesized using Qwen3-VL models. Although human evaluation shows comparable quality, this creates a potential bias: HiNav may be implicitly optimized for the specific linguistic patterns and failure modes of the generator VLM. The human-written split (HM3D-Test-Human) is relatively small (243 episodes), making it hard to fully rule out this bias.

  2. Questionable proxy metric for navigation information: The "request-detail density" is defined simply as the character/word length of the request divided by the geodesic path length (D(I, τ) = |I|/L(τ)). Text length is a very crude proxy for actual semantic navigation information. A long but vague request could have high density, while a short but highly informative one could have low density. Basing key analytical conclusions (Section 5.2) on this metric weakens the soundness of the diagnosis.

Questions for authors
  1. Regarding the request-detail density metric: Did you consider measuring semantic information content (e.g., counting unique landmarks or spatial relations mentioned) instead of relying purely on text length? How sensitive are your conclusions in Table 1 to the choice of this metric?

  2. Since both the benchmark construction and the HiNav training data synthesis rely heavily on Qwen3-VL, how much of HiNav's success is due to the model learning the specific generation biases of the VLM? Could you provide an analysis of HiNav's performance specifically on the HM3D-Test-Human split compared to the VLM splits?

  3. In the hierarchical learning ablation (Table 5), vanilla fine-tuning performs worse than the frozen baseline. Is this degradation solely due to the task shift disrupting pretrained priors, or could it be related to the limited size/quality of the synthesized training data?

  4. For the real-world deployment on the Unitree Go2, what was the inference latency per step, and did the serial coupling between the Reasoner and Executor cause noticeable delays or stuttering during physical navigation?

4 comments
AuthorsOpenReview ↗

Response to Reviewer ynDh

We sincerely thank the reviewer for the positive assessment of our benchmark design, diagnostic analyses, hierarchical framework, and real-world deployment. We also appreciate the constructive suggestions regarding data-generation bias, the validity of request-detail density, hierarchical learning, and inference efficiency. We address these concerns below.

W1 & Q2. Generalization Beyond VLM-Generated Data

We thank the reviewer for highlighting the potential bias arising from using Qwen3-VL for both benchmark construction and training data synthesis.

To address this concern, we conduct additional evaluations on matched trajectories between HM3D-Test-Human and HM3D-Test-VLM, comparing human-written, Qwen3-VL-generated, and InternVL-generated requests using bootstrap.

MethodComparison (A − B)ΔSR (pp)95% CI (pp)
HiNavHuman − Qwen+1.7[−9.70, +13.26]
HiNavQwen − InternVL+1.6[−5.67, +8.73]

HiNav shows only a small performance difference between human-written and Qwen-generated requests, with no statistically detectable degradation. In addition, HiNav maintains comparable performance on Qwen- and InternVL-generated requests. These results provide no clear evidence of Qwen3-VL-specific bias in our evaluations. We will incorporate these additional analyses into the revised manuscript.

W2 & Q1. Semantic-Aware Request Density Analysis

Thank you for this valuable suggestion. We agree that word count is a coarse proxy for navigation information and may not fully capture the semantic content of a request. Following the reviewer's suggestion, we additionally define a semantic-aware density metric based on the numbers of unique landmarks and spatial relations in each request:

Dsem(I,τ)=Nlandmark(I)+Nrelation(I)L(τ),D_{\mathrm{sem}}(I,\tau)=\frac{N_{\mathrm{landmark}}(I)+N_{\mathrm{relation}}(I)}{L(\tau)},

where NlandmarkN_{\mathrm{landmark}} and NrelationN_{\mathrm{relation}} denote the numbers of unique landmarks and spatial relations extracted from each request using GPT-OSS-20B, respectively. We re-group the episodes according to this metric and re-evaluate all seven methods on MP3D-Val-VLM. The results are summarized below.

MethodLow DensityMedium DensityHigh Density
SROSRSPLNESROSRSPLNESROSRSPLNE
NaVid15.0721.1412.8811.4321.1630.4318.198.0930.1045.1823.126.15
Uni-NaVid29.0234.3325.4410.2041.8348.5337.306.9048.7562.2039.515.57
StreamVLN20.0126.5818.1711.2229.1238.0126.017.8933.7352.3228.066.55
NaVILA17.6423.9516.2711.4823.8632.0621.978.1333.5448.5029.346.52
JanusVLN20.7628.4619.0710.6529.8139.3226.937.7739.6755.6935.105.91
DualVLN30.7734.2728.009.4744.5249.4140.316.1752.2563.9544.884.50
Ours39.7143.5332.759.5752.3556.4842.366.4662.2068.7745.114.78

All seven methods consistently exhibit higher success rates as semantic-aware density increases, reproducing the trend observed in Table 1. This indicates that our original observation is robust to the choice of density metric and does not solely depend on request length. We will revise the density definition and update Table 1 accordingly to incorporate this semantic-aware analysis.

AuthorsOpenReview ↗

Response to Reviewer ynDh

Q3. Understanding the Degradation of Vanilla Fine-Tuning

Thank you for this insightful question. We agree that the degradation of vanilla fine-tuning may be attributed to both the task shift and the limited scale of synthesized training data.

1. Impact on Pretrained Instruction-Following Capabilities. To investigate whether fine-tuning on underspecified requests interferes with pretrained navigation capabilities, we additionally evaluate the frozen and fine-tuned Uni-NaVid models using the original R2R instructions corresponding to our evaluation trajectories.

ModelSR↑SPL↑OSR↑NE↓
Frozen Uni-NaVid52.148.157.05.08
Vanilla Fine-tuning47.743.251.86.54

Vanilla fine-tuning reduces SR by 4.4 percentage points and SPL by 4.9 points even on the original R2R instructions, providing evidence that adapting to underspecified request-action supervision can interfere with pretrained instruction-following capabilities.

2. Limited Supervision and Hierarchical Adaptation. Notably, Uni-NaVid was originally trained on approximately 2.4M VLN samples, whereas our task-specific training set contains only 420K samples. Moreover, RG-VLN requires additional request grounding and spatial reasoning beyond conventional instruction following, making direct adaptation particularly challenging under limited supervision. Meanwhile, HiNav achieves 36.4% SR using the same underlying synthesized dataset, compared with 26.7% for vanilla fine-tuning, suggesting that the degradation cannot be attributed to data quality alone. We will further explore scaling up the training data in future work.

Q4. Real-World Inference Latency and Navigation Smoothness

Thank you for this question. The Reasoner and Executor have average inference latencies of 478 ms and 192 ms, respectively. Considering their sequential execution and communication overhead, the overall system incurs an average latency of 457 ms per step.

Regarding navigation smoothness, we did observe occasional motion pauses due to the serial coupling between reasoning and execution. However, these interruptions were relatively minor and did not noticeably disrupt the overall navigation process. We acknowledge this limitation and will explore asynchronous reasoning to further improve navigation smoothness in future work.

Thank you to the authors for the detailed response and additional experiments.

The new evaluations using the semantic-aware density metric (counting unique landmarks and spatial relations) and the cross-generator comparisons (Human vs. Qwen vs. InternVL) address my concerns regarding metric soundness and potential VLM bias. The clarifications on vanilla fine-tuning degradation and real-world inference latency are also helpful.

Please make sure to incorporate these important analyses and the clarifications regarding the evaluation subsets (as discussed with other reviewers) into the final revision. My initial rating is already positive, so I will maintain my score.

We sincerely thank the reviewer for the positive feedback and acknowledgment of our additional experiments and clarifications. We will ensure that all additional analyses and clarifications, including the evaluation subset definitions, are incorporated into the revised manuscript before the discussion phase concludes.

Reviewer_YYpqOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces Request Grounding Vision-Language Navigation (RG-VLN), a task in which embodied agents navigate from underspecified everyday requests rather than detailed route instructions. The authors construct the RG-VLN benchmark with over 6,000 episodes across MP3D and HM3D, incorporating VLM-generated and human-written requests. They further propose HiNav, a hierarchical framework that uses a VLM-based Request Reasoner to generate scene-grounded textual subtasks, which are executed by existing VLN policies through closed-loop navigation. Experiments across multiple VLN baselines demonstrate substantial performance degradation under underspecified requests, while HiNav achieves the highest success rates across all benchmark splits. Additional ablations, human evaluations, and real-world robot demonstrations are conducted to validate the proposed benchmark and framework.

Critical strengths

1. Comprehensive benchmark construction. The authors introduce RG-VLN with over 6,000 evaluation episodes across MP3D and HM3D, incorporating both VLM-generated and human-written requests. The matched-trajectory design, which pairs detailed instructions, paraphrases, and underspecified requests with the same navigation paths, provides a useful foundation for studying the effects of incomplete navigation information.

2. Effective hierarchical navigation framework. HiNav separates high-level request reasoning from low-level navigation execution through closed-loop textual subtask guidance. This is a practical design that allows existing VLN executors to be reused without architectural modifications. The substantial improvements obtained with frozen StreamVLN and DualVLN demonstrate the usefulness and transferability of this interface.

3. Extensive experimental evaluation. The paper evaluates six competitive VLN baselines across multiple benchmark splits and includes controlled comparisons, component ablations, and a real-world robot demonstration. The experiments show substantial performance degradation under underspecified requests and consistent success-rate improvements with HiNav, providing empirical support for both the benchmark motivation and the proposed method.

Critical weaknesses

1. Insufficient validation of benchmark solvability. The benchmark construction and verification processes primarily establish consistency between generated requests and their source trajectories, rather than demonstrating that the intended destinations can be independently recovered from the requests and available environmental observations. Both VLM and human request writers have access to the reference trajectories, while the human evaluation measures request validity rather than actual grounding success. Consequently, some requests may admit multiple plausible destinations, potentially penalizing navigation models for selecting reasonable alternatives. An independent human grounding evaluation would be important for establishing the reliability of the benchmark.

2. Limited independence in request quality assurance. The same Qwen3-VL-235B model is used for request generation, verification, and self-correction, introducing the possibility of correlated errors that remain undetected during automated verification. Furthermore, only 16.7% of generated samples undergo targeted human review, and the error rate among automatically accepted samples is not reported. Independent verification using a different VLM, together with random human auditing of accepted requests, would provide stronger evidence of annotation quality.

3. Limited architectural novelty and incomplete discussion of related work. Although HiNav is effective, its core design combines established ideas, including hierarchical language-based planning, visual memory selection, and pretrained navigation executors. Closely related approaches such as VLN-Copilot (ECCV 2024), InstructNav (CoRL 2024), and IDEAL-VLN (ICML 2026) are not adequately discussed. A clearer comparison with these methods is necessary to establish what is technically novel about HiNav beyond its application to the proposed benchmark.

4. Inconsistent evaluation-set descriptions. Tables 5 and 6 describe a separate R2R-derived MP3D evaluation set, whereas Tables 7–10 refer to a subset of MP3D-Val-VLM. Nevertheless, the same HiNav performance values (36.4 SR, 29.5 SPL, and 43.5 OSR) appear throughout, with matching additional metrics where reported. Although these experiments may use the same evaluation subset, the relationship is not clearly explained. The authors should specify the episode counts, selection criteria, and whether these experiments share identical evaluation samples.

Questions for authors

I would appreciate the authors addressing the concerns raised in the Weaknesses section, particularly those that could affect the validity of the benchmark and the interpretation of the experimental results.

6 comments
AuthorsOpenReview ↗

Response to Reviewer YYpq

We sincerely thank the reviewer for the positive assessment of our benchmark construction, hierarchical navigation framework, and experimental evaluation. We appreciate the constructive concerns and address them individually below.

W1. Independent Validation of Benchmark Solvability

Thank you for this important concern. We agree that trajectory consistency alone does not guarantee that a navigation goal can be independently recovered from an underspecified request.

1. Destination disambiguation in benchmark construction. As described in Section 3.2 and Appendix D.1, our request generation pipeline explicitly preserves destination-specific spatial cues while removing detailed movement instructions. In particular, the generation prompt requires the VLM to describe the precise spatial orientation and landmark relationships of the final destination. This design aims to reduce ambiguity about the intended target while preserving the challenge of navigating without explicit route guidance. We have also provided 100 example episodes in Anonymous repository – Anonymous GitHub for the reviewer to inspect the generated requests and their destination-specific descriptions.

2. Independent human navigation evaluation. To directly assess benchmark solvability, we conducted an additional human navigation experiment in HabitatSim. Three human participants independently controlled the agent based on the given requests and online visual observations, without access to the reference trajectories or ground-truth destinations. We obtained 99 valid navigation trials, with the following results:

MethodSR ↑SPL ↑OSR ↑NE ↓
Human86.965.788.92.26

Without access to reference trajectories or ground-truth destinations, human participants successfully reached the annotated destinations in 86 out of 99 episodes, demonstrating that the intended goals can generally be recovered from the requests and online observations alone, rather than merely being consistent with their source trajectories.

We will incorporate this human navigation evaluation into the revised manuscript.

W2. Independent Verification of Request Quality

Thank you for this valuable suggestion. We acknowledge that using the same VLM for generation and verification may introduce correlated errors. To address this concern, we conducted additional independent verification using a different VLM and human annotators.

1. Independent VLM verification. We re-evaluated all automatically accepted requests in RG-VLN using InternVL3.5-241B-A28B, a different VLM family from Qwen3-VL. InternVL independently assessed 96.7% of the requests as valid, providing large-scale cross-model evidence of annotation quality.

2. Random human auditing. We further conducted an independent human audit of 400 randomly selected automatically accepted request. The results showed that 385 of 400 requests were valid, providing an independent estimate of the annotation quality of automatically accepted samples.

Together, these results provide complementary evidence of annotation quality through large-scale cross-model verification and independent human auditing. The high validation rates demonstrate that RG-VLN maintains reliable annotation quality and is suitable for benchmark evaluation. We will incorporate these additional verification results and corresponding statistics into the revised manuscript.

AuthorsOpenReview ↗

Response to Reviewer YYpq

W3. Architectural Novelty and Comparison with Related Work

Thank you for highlighting these closely related studies. We clarify the key distinctions below.

1. VLN-Copilot (ECCV 2024). VLN-Copilot introduces uncertainty-triggered LLM assistance, where historical scene descriptions are provided to an LLM to generate supplementary guidance for a VLN agent. In contrast, HiNav focuses on request grounding by iteratively inferring the intended destination and missing route cues from the original request and accumulated observations, generating actionable navigation subtasks rather than supplementary information.

2. InstructNav (CoRL 2024). InstructNav translates semantic plans into navigation decisions through an explicit map-based system. HiNav instead generates short-horizon textual guidance directly consumable by existing pretrained VLN policies, without requiring dedicated spatial maps or waypoint-planning modules.

3. IDEAL-VLN (ICML 2026). IDEAL-VLN decomposes given navigation instructions into semantic anchors and aligns them with actions. In contrast, HiNav addresses missing navigation information in underspecified requests, using historical visual evidence to progressively infer the intended target and generate scene-grounded subtasks.

4. Technical contributions. HiNav's main contributions lie in request-conditioned visual history selection (DS-HG), closed-loop textual subtask generation, and plug-and-play integration with pretrained VLN executors. Together, these task-specific designs distinguish HiNav from existing hierarchical navigation approaches and demonstrate its unique technical contributions.

We will expand the Related Work section to discuss these three studies, acknowledge their connections to HiNav, and more clearly articulate our technical contributions.

W4. Clarification of Evaluation Sets in Tables 5–10

We apologize for the inaccurate description of the evaluation subsets in the original manuscript, which may have caused confusion. Tables 5–10 use exactly the same evaluation set of 4,793 episodes, derived from the R2R-CE val-unseen split and retained after quality assurance. Therefore, the repeated HiNav results (36.4 SR, 29.5 SPL, and 43.5 OSR) are expected, as they correspond to the same model configuration evaluated on identical episodes.

We will correct the corresponding descriptions and table captions to consistently identify the shared R2R-derived evaluation set and explicitly report its episode count and selection criteria.

I thank the authors for their very detailed explanations. I find these acceptable. I just have one question: Why haven't the authors already incorporated these revisions into the revised manuscript?

Reviewer_ecHdOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces RG-VLN, a benchmark for "request grounding" in vision-language navigation, where an agent must fulfill underspecified everyday requests (intent plus entities plus partial spatial cues, without complete route information) in continuous environments. The benchmark contains 6,406 episodes across three splits: MP3D-Val-VLM (4,850 VLM-generated requests grounded to VLN-CE val-unseen trajectories), HM3D-Test-VLM (1,313 on ObjectNav shortest-path trajectories), and HM3D-Test-Human (243 independently human-written requests), constructed via a Qwen3-VL-235B-assisted pipeline with iterative verification and targeted human review (16.7% flagged, 21.4% of flagged corrected). Because each trajectory is paired with a detailed route instruction, a route-preserving paraphrase, and underspecified request variants, matched-episode interventions isolate the effect of navigation specification: paraphrases change success rates by only 1.0-3.5%, whereas requests reduce them by 68.5-74.2% relatively; performance also declines with lower request-detail density. The paper further proposes HiNav, a hierarchical framework in which a fine-tuned Qwen3-VL-8B Request Reasoner emits closed-loop, scene-grounded textual subtasks (every four steps) to a VLN executor, with a dual-similarity CLIP-based history grounding module. HiNav achieves the best SR on all three splits (51.5/38.2/43.2), lifts frozen executors substantially (StreamVLN 14.7->44.0; DualVLN 19.3->50.7), matches a strong baseline's per-episode runtime (15.0s vs 15.05s), and is demonstrated zero-shot on a Unitree Go2 robot.

Critical strengths
  1. The matched-trajectory controlled evaluation is the paper's most consequential contribution. By evaluating identical episodes under three language conditions (instruction / route-preserving paraphrase / underspecified request), the benchmark attributes the 68.5-74.2% SR collapse to navigation specification rather than surface wording (paraphrases change SR by only 1.0-3.5%). This is cleaner evidence than the cross-dataset comparisons typical of benchmark papers, and together with the three-split structure - including 243 human-written requests and a human study showing VLM-written requests are near-indistinguishable from human ones (45.2% discrimination) - it yields a reusable, validated testbed whose value does not depend on HiNav.
  2. The frozen-executor transfer result (Table 4: StreamVLN 14.7->44.0 SR, DualVLN 19.3->50.7 with official frozen weights) is the most robust evidence for the textual-subtask interface, because it cannot be attributed to extra executor training. It demonstrates that the reasoner-executor language interface is plug-and-play in practice, which is directly useful to practitioners and strengthens the methodological claim independent of the fairness concerns affecting Table 2.
Critical weaknesses
  1. The quantitative record is currently unauditable in places, and the headline comparison is budget-asymmetric. HiNav's reasoner and default executor are fine-tuned on 9,927 synthesized episodes (~420K pairs) derived from R2R-CE training trajectories, while all baselines run official pretrained weights (Appendix A.2), so Table 2 margins conflate framework with adaptation budget. More concretely, the analyses in Tables 3-10 run on unnamed subsets whose sizes are never stated: the same HiNav numbers (36.4/29.5/43.5/67.6) appear under both "a separate fixed R2R-derived evaluation set" (Table 5 text) and "subset of MP3D-Val-VLM" (Tables 7-10 captions), and StreamVLN's request-condition SR is 14.7 in Tables 3-4 but 29.9 for type-1 requests in Table 7 - a factor-of-two gap that is unexplained unless the subsets differ, which is never stated. These issues are fixable with documentation and a budget-matched control, but until resolved the main numbers cannot be independently interpreted, which caps my confidence.
  2. Two central claims rest on analyses with unresolved confounds. First, the request-detail density analysis (D = |I|/L(tau)) places long-path episodes in the low-density tercile by construction, and long paths are intrinsically harder regardless of language, so the 15.9-25.5-point SR decline cannot be attributed to reduced specification without length controls. Second, the benchmark requests, the reasoner, its training supervision, and the alignment scorer all come from the Qwen3-VL family, and HiNav's relative margin shrinks from +21.2%/+15.1% on VLM-written splits to +7.2% (43.2 vs 40.3 SR, 243 episodes, no CI) on the human-written split - a pattern consistent with stylistic coupling that the paper does not discuss. Together these bear directly on whether the benchmark measures underspecification (versus path difficulty) and whether HiNav's advantage extends to genuinely human language.
Questions for authors
  1. (Clarification; material) Please define every evaluation subset used in Tables 3-10: sizes, selection rules, and which request type(s) they contain. Are the sets behind Tables 5-10 identical despite the two different descriptions ("R2R-derived evaluation set" vs "subset of MP3D-Val-VLM")? How do you reconcile StreamVLN's request-condition SR of 14.7 (Tables 3-4) with 29.9 for type-1 requests (Table 7)? Which request variant is used for the main results in Tables 1-2?
  2. (Analysis from existing runs; material) Please report the path-length distribution across the three density terciles. Does the high->low SR decline survive matching or stratifying by geodesic path length?
  3. (Material) Does HiNav's advantage on HM3D-Test-Human (43.2 vs 40.3 SR over Uni-NaVid, n=243) remain significant under a bootstrap over episodes? Please report CIs.
  4. (Clarification) In Table 5, does "the same limited training set" mean the full 9,927-episode synthesized set? Is a budget-matched fine-tune of Uni-NaVid or DualVLN on the same request-action supervision available, or feasible to report during the rebuttal?
  5. (Clarification if available; otherwise future work) Is a one-shot expansion baseline available - expanding each request once into a detailed route-style instruction (NavRAG-style) and feeding it to a frozen executor? This would isolate the value of closed-loop replanning, the paper's core methodological claim, from instruction enrichment.
  6. (Material) Will an anonymous code/benchmark link be provided during the review period?
4 comments
AuthorsOpenReview ↗

Response to Reviewer ecHd

We sincerely thank the reviewer for recognizing the value of our matched-request controlled evaluation, validated benchmark, and transferable reasoner–executor interface. We appreciate the constructive suggestions, which have helped us identify several aspects requiring clearer documentation and additional analysis. We address the concerns below.

W1-(1) & Q4. Training Budget and Adaptation Fairness

We appreciate the reviewer for pointing out the difference in task-specific adaptation budgets. Table 2 primarily evaluates the off-the-shelf generalization capabilities of existing pretrained VLM-based VLN methods to underspecified requests. To further investigate the effect of adaptation budgets, we conduct an additional fine-tuning experiment in Table 5, where Uni-NaVid is directly fine-tuned using the same limited training set as HiNav. Specifically, this refers to the full 9,927 R2R-CE-derived synthesized episodes (approximately 420K image–action pairs) used to train HiNav.

MethodSR (%)
Uni-NaVid (official pretrained weights)30.7
Uni-NaVid (direct fine-tuning)26.7
HiNav (Ours)36.4

Despite using the same task-specific training data, direct fine-tuning of Uni-NaVid does not improve its performance, whereas HiNav achieves 36.4% SR.

We attribute this difference partly to the substantial task shift and the complexity of RG-VLN. Although Uni-NaVid benefits from large-scale pretraining with approximately 2.4M instruction-following samples, RG-VLN requires agents to go beyond conventional instruction following: they must infer implicit targets, reason about missing spatial information, and plan navigation actions from underspecified requests. Directly fine-tuning an end-to-end VLN model therefore requires jointly adapting request understanding, high-level reasoning, and low-level action prediction using relatively limited task-specific supervision. In contrast, HiNav explicitly decouples request reasoning from navigation execution. This hierarchical decomposition provides structured intermediate supervision and reduces the burden of jointly learning these capabilities, enabling more effective use of limited training data.

Overall, the results in Table 5 suggest that HiNav's hierarchical design provides an advantage over direct fine-tuning under the same task-specific data setting.

W1-(2) & Q1. Evaluation Subset Clarification

We apologize for the insufficient documentation of the evaluation subsets and the inconsistent descriptions in several table captions.

Our experiments use two R2R-CE-derived evaluation subsets. First, we generate requests from R2R-CE trajectories and retain all 4,793 episodes that pass our request quality verification, which constitute the fixed evaluation set used in Tables 5–10. Second, for the matched-request intervention experiments (Tables 3–4), we randomly sample 300 episodes from these 4,793 episodes. Each sampled trajectory is paired with its original detailed route instruction, a route-preserving paraphrase, and an underspecified request.

The evaluation settings are summarized below.

TablesEvaluation setEpisodesRequest types
3–4Matched-request intervention set300VLN / Paraphrase / RG-VLN
5–10R2R-CE-derived evaluation set4,793Three Request variants

Regarding request variants, Tables 1–2 and Tables 5–10 report results aggregated across the three RG-VLN request types, unless otherwise specified. Table 7 additionally provides a breakdown by individual request type. These three request types should not be confused with the three language conditions (detailed instruction, paraphrase, and underspecified request) used in Tables 3–4.

The repeated HiNav results in Tables 5–10 are obtained from the same fixed R2R-CE-derived evaluation set. Our previous references to a subset of MP3D-Val-VLM in Tables 7–10 were inaccurate and will be corrected. Regarding StreamVLN, the SR of 14.7% in Tables 3–4 and 29.9% for Type 1 in Table 7 are obtained from different episode samples. The discrepancy primarily reflects the difference in evaluation sets.

We will revise the manuscript to explicitly specify the evaluation sets, episode counts, request variants, and sampling criteria.

AuthorsOpenReview ↗

Response to Reviewer ecHd

W2-(1) & Q2. Request-Detail Density and Path-Length Confounding

We appreciate this important observation. We agree that our original density definition, (D(I,\tau)=|I|/L(\tau)), introduces a dependency on geodesic path length, potentially confounding the relationship between request specification and navigation performance.

1. Path-Length Distribution and Density Analysis. Following the reviewer's suggestion, we report the path-length distributions across the three original density terciles:

DensityMeanMedianP25–P75
Low16.58 m14.74 m10.86–21.27 m
Medium10.58 m9.89 m8.32–12.00 m
High7.65 m7.62 m5.92–9.15 m

The results confirm that lower-density episodes tend to have longer paths. This is partly related to the different characteristics of detailed VLN instructions and RG-VLN requests. While detailed VLN instructions typically provide route-level guidance that grows with path length, RG-VLN requests primarily describe navigation intent, key landmarks, and spatial relations, without specifying complete routes. Specifically, our analysis shows that RG-VLN request length increases by approximately 6.4 words per 10 m, compared with 70.8 words per 10 m for detailed VLN-CE instructions, representing a reduction of approximately 91% in the growth rate.

2. Length-Controlled Evaluation. To examine whether the density–performance relationship persists after controlling for path length, we conduct an additional length-stratified evaluation. We stratify episodes into 2 m path-length bins, divide each bin into three density groups, and calculate weighted SRs using identical bin weights across all groups and models.

MethodLow SRMedium SRHigh SR
NaVid15.018.818.0
Uni-NaVid27.229.433.2
StreamVLN13.518.518.7
NaVILA15.018.121.3
JanusVLN21.623.624.7
DualVLN26.330.727.7
HiNav (Ours)35.342.346.1

The results show that all seven methods achieve higher SR in the High-density group than in the Low-density group, indicating that the positive association between request-detail density and navigation performance persists after controlling for path-length distributions.

We will include the path-length statistics and the additional length-controlled results in the revised manuscript.

AuthorsOpenReview ↗

Response to Reviewer ecHd

W2-(2) & Q3. Generator-Specific Bias and Generalization to Human-Written Requests

We appreciate the concern regarding potential stylistic coupling between Qwen-generated supervision, benchmark requests, and the HiNav reasoner. To address this concern, we conduct additional analyses from three perspectives.

1. Independent request–trajectory alignment evaluation. To assess request quality independently of the Qwen family, we conduct an additional trajectory–request alignment evaluation using PE-AV-Large-16-frame, an independently developed video–text embedding model.

The average alignment scores are 0.146 for human-written requests and 0.156 for Qwen-generated requests. These results show that Qwen-generated requests achieve comparable trajectory-alignment quality to human-written requests under an independent evaluator, supporting the grounding quality of our VLM-generated benchmark requests.

2. Cross-generator robustness evaluation. We note that the relative performance margins across the three benchmark splits are not directly comparable, as they involve different environments, trajectory sources, and evaluation episodes. Thus, the reduced margin on HM3D-Test-Human alone does not necessarily indicate Qwen-specific stylistic coupling.

To directly investigate this concern, we evaluate HiNav on requests from different linguistic sources, including human-written, Qwen-generated, and InternVL-generated requests on the same trajectories.

ModelComparisonΔSR (pp)95% CI (pp)
HiNavHuman − Qwen+1.7[−9.70, +13.26]
HiNavQwen − InternVL+1.6[−5.67, +8.73]

HiNav achieves comparable performance across different request sources, with no statistically significant differences between Human and Qwen or Qwen and InternVL. These results suggest that HiNav generalizes beyond Qwen-specific linguistic patterns, without exhibiting a particular advantage on Qwen-generated requests.

3. Statistical evaluation on human-written requests. Following the reviewer's suggestion, we conduct a paired bootstrap comparison between HiNav and Uni-NaVid on HM3D-Test-Human, using 10,000 bootstrap resamples.

StatisticResult
HiNav SR43.2%
Uni-NaVid SR40.3%
ΔSR (HiNav − Uni-NaVid)+2.9 pp
95% paired bootstrap CI[−1.23, +7.00] pp

We further examine the consistency of this advantage across bootstrap resamples:

Bootstrap outcomeCountFrequency
HiNav achieves higher SR8,98089.80%
Tie3033.03%
Uni-NaVid achieves higher SR7177.17%

HiNav achieves the highest SR on the human-written split, outperforming Uni-NaVid by 2.9 pp. Although the 95% CI includes zero, 89.80% of bootstrap resamples favor HiNav, further indicating its consistent performance advantage over Uni-NaVid.

Overall, these results suggest that HiNav does not exhibit a specific bias toward Qwen-generated requests and maintains a performance advantage over Uni-NaVid on the human-written split. We will include these results in the revised manuscript.

AuthorsOpenReview ↗

Response to Reviewer ecHd

Q5. One-Shot Instruction Expansion Baseline

Thank you for this valuable suggestion. We add a one-shot instruction expansion baseline to the frozen-executor transfer experiment in Table 4.

Specifically, we use HiNav's Request Reasoner to expand each underspecified request into a detailed, NavRAG-style route instruction, taking only the initial visual observation and original request as inputs. The expanded instruction is then provided to the frozen VLN executor for the entire episode, without subsequent high-level replanning or instruction updates.

ExecutorConfigurationSRSPLOSRNE
StreamVLNOriginal14.712.628.78.77
+ One-shot expansion34.029.245.77.04
+ HiNav44.040.551.05.04
DualVLNOriginal19.316.529.37.11
+ One-shot expansion33.329.141.76.56
+ HiNav50.746.654.74.36

The table show taht one-shot expansion substantially improves both executors, indicating that instruction enrichment alone is beneficial. These results also demonstrate that HiNav's advantages extend beyond one-time instruction enrichment, highlighting the additional benefit of closed-loop, observation-grounded guidance over static instruction expansion. We will incorporate this baseline into the revised Table 4.

Q6. Anonymous Code and Benchmark

Yes. We have provided an anonymous repository containing HiNav's inference code and 100 representative benchmark episodes, allowing reviewers to inspect our closed-loop reasoning–execution framework and benchmark examples.

Anonymous repository: https://anonymous.4open.science/r/RG-VLN/

We plan to release the complete benchmark and full HiNav implementation upon publication.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Summary of Manuscript Revisions

We sincerely thank the reviewers for their constructive feedback and valuable suggestions. We have revised the manuscript to address their concerns and strengthen the experimental evidence and presentation. The main revisions are summarized below, with the corresponding reviewers indicated. All substantive changes are highlighted in blue in the revised manuscript.

  • Evaluation protocol clarification — Reviewers ecHd (W1-(2), Q1) and YYpq (W4).
    We clarified the evaluation-set construction, sample counts, and request variants. Tables 3–4 use 300 randomly sampled episodes, while Table 5 and Tables 7–11 share a fixed R2R-CE-derived evaluation set containing 4,793 episodes across three request variants. We also corrected the corresponding descriptions and captions. (Section 5.1.)

  • Hierarchical adaptation and fine-tuning degradation — Reviewers ecHd (W1-(1), Q4) and ynDh (Q3).
    We specified the shared training set of 9,927 episodes and expanded the analysis of direct fine-tuning degradation from two perspectives: interference with pretrained instruction-following capabilities and limited task-specific supervision. We added an evaluation on original R2R instructions, where fine-tuning reduces SR from 52.1% to 47.7%. (Section 5.5; Tables 5 and 12.)

  • Semantic-aware density analysis — Reviewer ynDh (W2, Q1).
    We replaced word-count-based density in the main analysis with a semantic-aware metric based on unique landmarks and spatial relations extracted using GPT-OSS-20B. Table 1 and the accompanying analysis have been updated. (Section 5.2; Table 1.)

  • Path-length confounding — Reviewer ecHd (W2-(1), Q2).
    We retained the original word-count-based analysis in the appendix and added path-length distributions and a length-controlled evaluation. After stratifying episodes into 2 m path-length bins and applying identical bin weights, all seven methods still achieve higher SR in the high-density group than in the low-density group. (Appendix C.5; Tables 14–16.)

  • Generator bias and human-request generalization — Reviewers ecHd (W2-(2), Q3) and ynDh (W1, Q2).
    We added matched-trajectory comparisons using human-, Qwen-, and InternVL-written requests, with confidence intervals showing no statistically significant cross-source differences. On HM3D-Test-Human, HiNav exceeds Uni-NaVid by 2.9 percentage points, and 89.8% of 10,000 paired bootstrap resamples favor HiNav. (Section 5.3; Table 13.)

  • Independent trajectory–request alignment — Reviewer ecHd (W2-(2), Q3).
    We added evaluation using PE-AV-Large-16-frame, independent of the Qwen family. Average alignment scores are 0.146 for human-written requests and 0.156 for Qwen-generated requests. (Section 5.4.)

  • Independent request-quality verification — Reviewer YYpq (W2).
    We added cross-model verification using InternVL3.5-241B-A28B, which judges 96.7% of automatically accepted requests as valid. An independent audit of 400 randomly selected automatically accepted requests finds 385 valid samples. (Section 5.4.)

  • Independent benchmark-solvability evaluation — Reviewer YYpq (W1).
    We added human navigation experiments in HabitatSim without access to reference trajectories or ground-truth destinations. Three participants achieve 86 successes across 99 valid trials, with 86.9% SR and 65.7% SPL. (Section 5.4; Table 6.)

  • One-shot instruction expansion baseline — Reviewer ecHd (Q5).
    We added a baseline that expands each request once using the initial observation and then supplies the fixed instruction to a frozen executor. One-shot expansion improves StreamVLN and DualVLN to 34.0% and 33.3% SR, while HiNav achieves 44.0% and 50.7%, supporting the additional benefit of closed-loop guidance. (Section 5.5; Table 4.)

  • Related work and technical distinctions — Reviewer YYpq (W3).
    We added discussions and citations for VLN-Copilot, InstructNav, and IDEAL-VLN, and clarified HiNav’s request-conditioned history selection, closed-loop textual subtask generation, and integration with pretrained VLN executors. (Section 2.)

  • Real-world latency and navigation smoothness — Reviewer ynDh (Q4).
    We added Reasoner, Executor, and system latency measurements, discussed occasional motion pauses caused by serial coupling, and identified asynchronous reasoning as future work. (Appendix A.3.)

  • Anonymous code and benchmark access — Reviewer ecHd (Q6).
    We added an anonymous repository link providing inference code and 100 representative benchmark episodes. (Final sentence of the abstract.)

Please let us know if you have any further questions or concerns.

Dear Area Chair,

We sincerely thank you for your time and effort in handling our submission. We would like to summarize the reviewers’ assessments and the outcomes of the rebuttal discussion.

We appreciate the reviewers’ recognition of the benchmark’s comprehensive construction and controlled matched-trajectory evaluation (ecHd, ynDh, YYpq), HiNav’s effectiveness and transferability (ecHd, ynDh, YYpq), the practical hierarchical framework and zero-shot real-world deployment (ynDh), and the extensive experimental evaluation (YYpq).

Regarding the reviewers' concerns, we have conducted additional analyses and provided clear clarifications to address them: independent human navigation and annotation-quality verification (YYpq), semantic-aware density and path-length-controlled analyses (ynDh, ecHd), cross-generator evaluation and bootstrap analyses (ecHd, ynDh), a one-shot expansion baseline (ecHd), budget-symmetric fine-tuning comparison and degradation analysis (ecHd, ynDh), deployment latency clarification (ynDh), and clearer evaluation protocols and related-work comparisons (ecHd, YYpq). These revisions are incorporated into the manuscript and highlighted in blue. Details are provided in “Summary of Manuscript Revisions.”

Reviewers ynDh and YYpq confirmed that our responses and revisions addressed their concerns, and reviewer YYpq increased the rating from 2 to 3. Reviewers ynDh and ecHd maintained a rating of 3. Thus, all three reviewers currently assign 3 (Weak acceptance).

We are very grateful to all reviewers for recognizing our work and helping us improve the paper. We also thank the Area Chair for the careful consideration and coordination of the review process.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.