acceptodds
Under review as a conference paper at ICLR 2027

ErrandVLN: A Benchmark for Intent-Driven Long-Horizon Continuous Vision-and-Language Navigation

Abstract

Existing continuous vision-and-language navigation (VLN) benchmarks primarily evaluate single-goal route following, whereas everyday household requests may require agents to visit multiple semantic landmarks in a prescribed order. This mismatch leaves **intent-driven, multi-stage navigation** insufficiently characterized. We introduce **ErrandVLN**, a benchmark that pairs continuous trajectories with diverse role-conditioned requests, allowing the same ordered landmark requirements to be expressed across varied activity contexts. Constructed on InteriorGS environments in NVIDIA Isaac Sim through an automated semantic-geometric pipeline, ErrandVLN comprises 913 scenes, 65,134 unique physical trajectories, and 502,233 instructions, with evaluation splits for unseen scenes, novel trajectory compositions, and unseen instruction styles. To address compounding errors over ordered stages, we propose **VLN-JEPA**: a continuous trajectory proposer (TP-Flow) generates candidate motions, an action-conditioned latent world model (NaviLat) predicts future latent states under these candidate motions without decoding future observations, and a compact 2.35B VLM controller uses these predictions together with the instruction and observation history to generate the next executable action. On ErrandVLN Test Scene-Unseen, VLN-JEPA achieves superior performance across multi-stage milestone completion (20.5% GCR, 16.1% GC-SPL, 29.2% OSR) and competitive task success (12.6% SR), while substantially reducing physical collisions by 40% compared with DualVLN. On a single NVIDIA RTX 4090, it achieves 94 ms per policy-inference step (10.7 FPS) and 5.3 FPS in complete closed-loop execution.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Vision-and-Language Navigation, Benchmark, Embodied AI

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 4 reviews, above 68% of the others.
Reviewer_G7xiOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces ErrandVLN, a continuous VLN benchmark built on InteriorGS scenes in Isaac Sim (913 scenes, 65k trajectories, 502k instructions). Each episode is an ordered sequence of up to four landmark goals, and an LLM rewrites the same skeleton into instructions framed by different household roles and activities. The paper also proposes VLN-JEPA: a flow-based proposer samples candidate trajectories on a depth-derived occupancy map, a latent world model rolls out future latent states for each candidate, and these are fed as tokens to a 2B VLM that outputs the next action. It reports 12.6 SR / 20.5 GCR on Test Scene-Unseen, and 62.5 SR on R2R-CE when the proposer and world model are attached to JanusVLN.

Critical strengths
  1. Ordered multi-goal navigation in continuous, physically executable 3DGS scenes is a reasonable target, and the dataset is large.
  2. Separate held-out splits for scene, path and instruction style are useful for diagnosing generalization.
  3. The proposer and world model are trained only on ErrandVLN and still improve JanusVLN on R2R-CE, which is a non-trivial cross-simulator transfer result.
Critical weaknesses
  1. The intent / role-conditioned aspect is what separates ErrandVLN from LH-VLN, GOAT-Bench and IVLN, but no experiment supports it. The underlying task is ordered multi-landmark navigation, and the roles only change the wording. There is no comparison against plain landmark-list instructions on the same trajectories, and Seen-Style-Unseen results are not in the main paper, so we cannot tell whether role framing adds any difficulty or requires different reasoning. Both the area graph and the instructions are produced by Qwen-3.5-Plus, with no human verification and no human performance reference. It is unclear how many episodes are ambiguous, e.g. when a landmark category has several instances but success is measured against one specific target position.

  2. Table 2 does not support the claims. All VLN baselines are run zero-shot from released checkpoints in a new simulator and rendering domain, while VLN-JEPA is trained on ErrandVLN, so the comparison mostly measures domain shift. Some baseline numbers look like interface failures rather than policy behavior (NaVILA: PL 446.5 m). Even in this setting, zero-shot DualVLN has higher SR than the trained method (15.1 vs 12.6). At least a few baselines (e.g. StreamVLN, NaVid) should be fine-tuned on the ErrandVLN train split. With SR around 12%, it is also impossible to tell whether the difficulty comes from the multi-stage structure or from the sim/visual gap. Single-stage results on the same scenes would separate the two.

  3. Sensor mismatch. VLN-JEPA uses depth to build the occupancy grid and to score candidates by traversability, while the compared VLMs are RGB-only. The 40% collision reduction highlighted in the abstract is largely what depth-based traversability filtering would give anyway. The same applies on R2R-CE: JanusVLN is RGB-only, and the augmented version adds depth (Table 3). JanusVLN + TP-Flow without NaviLat is needed to attribute the gain to latent prediction rather than to depth.

  4. The evidence that the world model matters is weak. Removing NaviLat lowers SR from 12.6 to 9.8, and zeroing the future tokens lowers it to 10.2. Zeroed or shuffled tokens at test time are out-of-distribution inputs for a controller trained with real ones, so these drops do not show that the predicted content is informative; shuffling hurting more than zeroing suggests reliance on slot position. The right control is a controller trained with uninformative futures. No seeds or confidence intervals are given and the test split size is not stated in the main text, so 1-3 point differences are hard to judge.

  5. The R2R-CE claim of "leading performance across all published continuous-navigation baselines" is not accurate. AwareVLN (CVPR 2026, arXiv 2605.22816) reports 65.4 SR / 73.5 OSR / 4.02 NE on R2R-CE Val-Unseen with monocular RGB only, above the 62.5 / 69.2 / 4.59 reported here with RGB + depth. The SPL here is higher (59.7 vs 55.1), which is worth discussing, but the claim should be corrected and the work compared. It is also relevant in substance, since it models task-progress awareness over multi-part instructions, which is exactly the failure mode of multi-stage errands.

  6. Minor: SR and OSR use 3.0 m while GCR and GC-SPL use 1.5 m. Please justify. 3.0 m is loose for indoor landmarks, and with different thresholds SR can exceed GCR for a model that stops early (DualVLN: 15.1 vs 12.9).

Questions for authors
  1. How does performance change on the same trajectories with role-free, landmark-list instructions? What are the results on Val Seen-Path-Unseen and Seen-Style-Unseen?
  2. How are landmark instances disambiguated when a category appears more than once? Was any subset human-checked, and what is human SR?
  3. Results for baselines fine-tuned on ErrandVLN train?
  4. On R2R-CE, what does JanusVLN + TP-Flow alone (no NaviLat) achieve? Results over multiple seeds?
  5. Test split size, and a breakdown by number of stages (M = 1..4) in the main text.
12 comments

Question for the Authors Q1 (original):

How does performance change on the same trajectories with role-free, landmark-list instructions? What are the results on Val Seen-Path-Unseen and Seen-Style-Unseen?

Response:

We thank the reviewer for requesting clarification of the task setting and results across evaluation splits.

Task definition and contribution.

ErrandVLN evaluates whether an agent can fulfill the multi-stage navigation requirements of a semantically coherent household errand from natural-language instructions and RGB observations. Stages serve a common activity purpose: the agent must infer destinations and their order from a unified request and navigate in a continuous environment. Roles express the speaker's perspective and motivation; the task contribution is the organization of navigation requirements around a common activity purpose.

Landmarks serve different roles in dataset construction and agent execution. We use landmarks to identify reachable target positions, connect consecutive goals using A* to construct reference trajectories, and generate natural-language instructions in an activity context. Target positions support evaluation of stage completion and visiting order. The agent receives only instructions and RGB observations, without landmark lists, instance identifiers, goal coordinates, or structured visiting sequences. Using landmarks internally to construct trajectories therefore does not make the agent's task landmark-list navigation.

This follows the standard VLN definition: R2R [1] requires agents to interpret and follow natural-language navigation instructions using visual observations, and VLN-CE [2] extends this paradigm to continuous 3D environments. ErrandVLN extends it to multi-stage requests with a shared activity purpose. LH-VLN [3] sequentially combines subtasks without requiring a common overarching activity; ErrandVLN connects stages semantically. GOAT-Bench [4] supplies successive goals through category names, descriptions, or instance images; ErrandVLN expresses interconnected requirements in one natural-language request. IVLN [5] emphasizes memory across episodes in a persistent environment; ErrandVLN emphasizes multi-stage execution within an episode.

Landmark-list comparison and cross-split results.

We have not conducted a comparison with plain landmark-list instructions on the same trajectories. Our setting evaluates understanding and executing navigation requirements expressed in a high-level request, whereas a landmark list explicitly enumerates destinations and their order. The table below reports cross-split results under the current instruction setting.

Table 1. VLN-JEPA navigation performance across ErrandVLN evaluation splits. Values are means [95% bootstrap confidence intervals].

SplitGCR (%) ↑SR (%) ↑OSR (%) ↑GC-SPL (%) ↑
Val Seen-Path-Unseen18.88 [17.3, 20.6]8.90 [7.2, 10.7]25.20 [22.5, 27.9]15.64 [14.3, 17.0]
Val Seen-Style-Unseen18.15 [16.4, 19.9]9.10 [7.4, 10.9]22.00 [19.5, 24.6]15.46 [14.1, 17.0]
Test Scene-Unseen20.54 [18.6, 22.5]12.60 [10.6, 14.8]29.20 [26.4, 32.0]16.11 [14.7, 17.6]

On Val Seen-Style-Unseen, GCR is 18.15%, SR is 9.10%, and GC-SPL is 15.46%, reporting navigation performance under unseen persona-conditioned expressions in familiar scenes.

References:

[1] Anderson et al. Vision-and-Language Navigation: Interpreting Visually-Grounded Navigation Instructions in Real Environments. CVPR, 2018. [2] Krantz et al. Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environments. ECCV, 2020. [3] Song et al. Towards Long-Horizon Vision-Language Navigation: Platform, Benchmark and Method. CVPR, 2025. [4] Khanna et al. GOAT-Bench: A Benchmark for Multi-Modal Lifelong Navigation. CVPR, 2024. [5] Krantz et al. Iterative Vision-and-Language Navigation. CVPR, 2023.

Question for the Authors Q2 (original):

How are landmark instances disambiguated when a category appears more than once? Was any subset human-checked, and what is human SR?

Response:

We thank the reviewer for asking how target instances are distinguished and how the data are verified.

Target descriptions and evaluation correspondence.

When a category has multiple instances, we describe the target in the generated instruction using its functional region, spatial relationships to surrounding objects, and visual attributes such as color and state. For example, “the chair beside the table in the dining room” provides language cues for distinguishing instances of the same category.

For annotation and evaluation, each stage uses an instance ID to link the reference trajectory and the corresponding reachable target position, keeping the instance used for instruction generation consistent with the evaluation target. Instance IDs are not supplied to the agent. Success is measured relative to that target position, using a 3 m threshold for SR and a 1.5 m threshold for each stage target in GCR. Evaluation thus measures arrival at the intended target, rather than proximity to any object of the same category.

Human verification and human performance.

The revision provides a human-verified subset of 1,000 trajectories, with trajectory–instruction pairs individually checked for physical reachability and multi-goal consistency. A dedicated audit of target-instance uniqueness across all instructions has not been conducted, and the proportion of ambiguous tasks has not been quantified. Given the dataset scale and the time and human effort required for navigation evaluation, human navigation SR/GCR results are not yet available in this revision.

W2 / Q3 (combined summary):

Please compare ErrandVLN-fine-tuned baselines to distinguish domain shift from multi-stage difficulty, and explain possible interface failures and DualVLN's higher SR.

Response:

We thank the reviewer for requesting matched-domain comparisons and stage-wise analysis.

NaViLA's 446.5 denotes mean collisions; its mean path length is 10.1 m.

We fine-tune NaViLA and StreamVLN on the official ErrandVLN training split with optimization schedules and supervision objectives matched to VLN-JEPA, evaluating all three on the same 1,000 human-verified trajectories.

Table 2. Overall comparison after matched in-domain fine-tuning. Δ denotes VLN-JEPA minus StreamVLN.

MethodSR (%) ↑OSR (%) ↑GCR (%) ↑GC-SPL (%) ↑PL (m)Col ↓
NaViLA9.6024.3011.519.933.5525.67
StreamVLN19.4037.0020.8818.635.8859.57
VLN-JEPA23.0046.3030.2122.179.1421.76
Δ (vs. StreamVLN)+3.60+9.30+9.33+3.54+3.26−37.81

After in-domain fine-tuning, VLN-JEPA leads both baselines on SR, OSR, GCR, and GC-SPL, with fewer collisions. Over StreamVLN, GCR and SR improve by 9.33 and 3.60 percentage points. Our advantage thus persists after adapting the external baselines to ErrandVLN.

Table 3. Stage-wise comparison after matched in-domain fine-tuning. Values are means [95% bootstrap CIs].

MethodMGCR (%) ↑GC-SPL (%) ↑SR (%) ↑
NaViLA110.94 [7.8, 14.6]8.08 [5.4, 11.2]16.72 [12.8, 21.3]
—211.04 [8.2, 14.4]9.72 [7.1, 12.9]5.71 [3.7, 8.4]
—312.98 [8.1, 18.6]12.76 [8.0, 18.4]5.23 [2.3, 9.3]
—412.76 [6.8, 20.1]12.01 [6.3, 19.3]9.38 [4.2, 16.7]
StreamVLN124.01 [19.6, 28.8]19.18 [15.1, 23.7]27.66 [22.8, 32.8]
—220.47 [16.6, 24.6]19.16 [15.4, 23.3]18.11 [14.4, 22.3]
—318.60 [13.2, 24.8]17.90 [12.6, 24.0]10.47 [6.4, 15.7]
—415.89 [9.1, 23.7]15.89 [9.1, 23.7]12.50 [6.3, 20.8]
VLN-JEPA138.60 [33.4, 44.1]22.59 [18.3, 27.3]32.83 [28.0, 38.3]
—228.29 [24.1, 32.8]23.61 [19.6, 28.0]19.60 [15.9, 23.8]
—324.61 [18.4, 31.4]20.82 [14.9, 27.3]18.02 [12.8, 24.4]
—419.53 [12.2, 27.9]17.15 [10.2, 25.3]12.50 [6.3, 20.8]

At M=3, VLN-JEPA exceeds StreamVLN in SR by 7.55 points. At M=4, SR ties at 12.50%, but our GCR and GC-SPL are 3.64 and 1.26 points higher. GCR and GC-SPL gains across all groups show consistently better ordered and path-efficiency-weighted completion. Lower multi-stage SR after adaptation also indicates remaining task difficulty.

Table 4. Stage-wise comparison with DualVLN on the main Test Scene-Unseen set. DualVLN uses public weights. OSR includes 95% bootstrap CIs; Δ denotes VLN-JEPA minus DualVLN.

MethodMGCR (%) ↑SR (%) ↑OSR (%) ↑
DualVLN126.8744.7856.72 [44.8, 68.7]
—216.0220.1829.08 [24.3, 34.1]
—310.419.4718.60 [14.0, 23.2]
—49.088.3613.50 [10.0, 17.4]
—Overall12.9915.1023.10 [20.5, 25.7]
VLN-JEPA137.3132.8462.69 [50.7, 74.6]
—224.7817.2136.80 [31.8, 41.8]
—316.739.4725.26 [20.4, 30.5]
—415.846.1117.36 [13.2, 21.5]
—Overall20.5412.6029.20 [26.4, 32.0]
Δ1+10.44−11.94+5.97
—2+8.76−2.97+7.72
—3+6.320.00+6.66
—4+6.76−2.25+3.86
—Overall+7.55−2.50+6.10

DualVLN's 44.78% single-stage SR demonstrates effective navigation in this simulator; its four-stage SR falls to 8.36%, further illustrating the challenge of multi-stage execution.

VLN-JEPA achieves 20.54% overall GCR and 29.20% OSR, exceeding DualVLN by 7.55 and 6.10 percentage points, respectively, with gains in every stage-count group. These results demonstrate higher ordered subgoal completion and all-target reachability. TP-Flow supplies traversability-informed candidate actions; NaviLat forecasts their consequences, providing future-state information for VLM action generation. Module ablations further support these gains (Table 6 in the revised manuscript).

DualVLN has higher overall SR (15.10% versus 12.60%); its training covers ScaleVLN and more extensive DAgger data, providing broader scene and correction experience. We will expand training-data coverage and interactive correction training to further improve whole-task success.

Reviewer_K6wbOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces ErrandVLN, a benchmark for activity-conditioned, multi-stage vision-and-language navigation in continuous indoor environments. Tasks require agents to visit semantic landmarks in a prescribed order, potentially revisiting previously explored areas. Although instructions describe household activities, evaluation concerns their navigation requirements rather than physical manipulation.

The paper also proposes VLN-JEPA, combining a continuous trajectory proposer, an action-conditioned latent world model, and a VLM controller that uses predicted future representations to generate executable actions. Experiments evaluate the system on ErrandVLN and investigate the proposed modules through ablations, component diagnostics, and adaptation to a JanusVLN backbone on R2R-CE.

Critical strengths
  1. Clear and meaningful task formulation. ErrandVLN studies activity-conditioned navigation with ordered semantic goals and revisits, extending evaluation beyond single-destination route following. The task definition clearly specifies the navigation requirements and distinguishes partial milestone completion from complete task success.

  2. Detailed benchmark construction pipeline. The paper thoroughly describes environment processing, task skeleton generation, trajectory planning, physical filtering, geometric deduplication, and instruction synthesis. The separate splits for unseen scenes, unseen trajectories, and scene-conditioned instruction styles provide useful settings for examining different aspects of generalization.

  3. Well-described method and component studies. The roles of TP-Flow, NaviLat, and the VLM controller, together with their training and execution procedures, are clearly presented. Architectural ablations, future-token perturbations, and component-level diagnostics of trajectory generation and latent prediction provide evidence beyond overall navigation scores.

Critical weaknesses
  1. The connection between the benchmark’s distinctive challenges and the proposed method remains insufficiently established.
    The paper motivates VLN-JEPA through error accumulation in multi-stage navigation. However, the proposed modules primarily address local motion proposal and short-horizon outcome prediction, whereas the benchmark also emphasizes activity-grounded instructions, ordered goal execution, and revisits. It remains unclear which of these challenges principally limits existing models and how the proposed mechanism specifically alleviates that limitation. The stratified results in Appendix C.2 characterize the full model’s behavior but do not compare it with matched controls within these task strata. Consequently, the results demonstrate overall improvements more directly than they establish the proposed explanation for those improvements.

  2. Comparative behavioral analysis on ErrandVLN is limited.
    Figure 12 provides useful failure cases for VLN-JEPA itself, while Figure 14 compares baseline and proposed trajectories on R2R-CE. However, the paper does not provide corresponding paired qualitative trajectory comparisons with existing navigation models on ErrandVLN. This makes it difficult to understand where existing models fail in the new task and what behavior VLN-JEPA changes—for example, landmark identification, goal ordering, revisit navigation, local motion, or stopping. Aggregate performance differences alone do not resolve this question.

  3. Validation of generated instructions could be clarified.
    Appendix A.3 provides instruction-generation prompts that explicitly require coverage of the waypoints in order. However, any post-generation checks on the final instructions are less clear. A brief clarification of whether such checks are performed would be helpful.

  4. The external baseline comparisons combine differences in adaptation and observation modalities.
    Section 5.1 and Appendix C.1 state that external baselines use released checkpoints without ErrandVLN adaptation, whereas VLN-JEPA is trained on ErrandVLN. Table 10 reports RGB-only inputs for the external baselines in this evaluation, while VLN-JEPA additionally consumes depth through TP-Flow. Thus, Table 2 does not isolate the architectural contribution from task-specific training and additional sensory information. The matched-data SFT baseline and depth-matched w/o NaviLat control are valuable and partly address this concern, but conclusions drawn from the external comparisons should distinguish these sources of improvement.

Questions for authors
  1. Task–method connection: Which failure mechanism in existing models most directly motivates latent future prediction for ErrandVLN? Could the authors compare the full model with existing matched controls across stage counts and revisit requirements to clarify whether the gains specifically address multi-stage difficulties?

  2. Comparative cases: Could the authors provide paired ErrandVLN trajectories for VLN-JEPA and a representative existing navigation model, identifying the point of divergence and the associated failure behavior? Please clarify whether the examples illustrate typical failures or selected instances.

  3. Annotation quality: Beyond the requirements specified in the generation prompts, are any post-generation checks performed to verify consistency with the intended landmarks and their visiting order?

  4. Comparison conditions: Could the authors clarify which conclusions concern transfer from released checkpoints and which isolate the proposed modules’ benefits under matched conditions? Please distinguish the evidence for gains from ErrandVLN training, depth-based trajectory proposal, and latent future prediction.

5 comments

Critical Weakness W1 (original):

The connection between the benchmark’s distinctive challenges and the proposed method remains insufficiently established. The paper motivates VLN-JEPA through error accumulation in multi-stage navigation. However, the proposed modules primarily address local motion proposal and short-horizon outcome prediction, whereas the benchmark also emphasizes activity-grounded instructions, ordered goal execution, and revisits. It remains unclear which of these challenges principally limits existing models and how the proposed mechanism specifically alleviates that limitation. The stratified results in Appendix C.2 characterize the full model’s behavior but do not compare it with matched controls within these task strata. Consequently, the results demonstrate overall improvements more directly than they establish the proposed explanation for those improvements.

Question for the Authors Q1 (original):

Task–method connection: Which failure mechanism in existing models most directly motivates latent future prediction for ErrandVLN? Could the authors compare the full model with existing matched controls across stage counts and revisit requirements to clarify whether the gains specifically address multi-stage difficulties?

Response:

We thank the reviewer for suggesting that we further explain the connection between task challenges and method design.

The method targets local execution errors that accumulate during ordered multi-stage navigation. Collisions can cause failed motion, detours, or repeated adjustments that delay goal progress; premature stopping terminates all remaining stages. When an earlier milestone remains incomplete, later milestones cannot advance the ordered completion count. TP-Flow and NaviLat address these execution problems at the current decision step.

TP-Flow provides candidates with better local traversability. NaviLat predicts the future visual states resulting from these candidates and supplies them as prospective information to the controller, allowing action generation to reference future information. After each execution, predictions are updated from new real observations, allowing sustained improvements in local decisions to support subsequent stage progress.

Table 1. GCR by number of task stages for matched ablation configurations on ErrandVLN Test Scene-Unseen. Values are means [95% bootstrap confidence intervals].

ConfigurationM=1M=2M=3M=4
SFT-only22.39 [13.4, 32.8]16.47 [13.5, 19.6]10.18 [7.8, 12.6]9.81 [8.0, 11.7]
w/o NaviLat31.34 [20.9, 43.3]20.18 [16.8, 23.7]13.68 [10.9, 16.5]10.93 [8.8, 13.2]
Full model37.31 [26.9, 49.3]24.78 [21.1, 28.5]16.73 [13.8, 19.8]15.84 [13.3, 18.4]

At M=4, the full model improves GCR by 6.03 and 4.91 percentage points relative to SFT-only and w/o NaviLat, respectively. Its GC-SPL is 12.10%, above the 8.68% of w/o NaviLat. The full model maintains higher GCR across M=1–4, reaching 15.84% at M=4, supporting sustained progress through receding-horizon local foresight on longer tasks.

Table 2. Comparison with DualVLN by revisit requirement on ErrandVLN Test Scene-Unseen. Values are means [95% bootstrap confidence intervals].

Revisit requirementModelGCR (%) ↑SR (%) ↑OSR (%) ↑
No revisitDualVLN13.62 [11.8, 15.5]16.50 [14.0, 19.1]24.19 [21.2, 27.3]
No revisitVLN-JEPA21.92 [19.7, 24.2]13.52 [11.2, 15.9]30.27 [27.0, 33.5]
RevisitDualVLN10.40 [7.5, 13.6]9.28 [5.7, 13.4]18.56 [13.4, 23.7]
RevisitVLN-JEPA14.82 [11.7, 18.1]8.76 [5.2, 12.9]24.74 [18.6, 30.9]

VLN-JEPA achieves higher GCR and OSR in both groups. Without revisits, the improvements over DualVLN are 8.30 and 6.08 percentage points, respectively; with revisits, they are 4.42 and 6.18 percentage points. DualVLN has higher SR in both groups. Both models show lower GCR, SR, and OSR on revisit tasks, indicating additional difficulty in returning to previously visited areas while continuing the ordered task. The comparison shows that VLN-JEPA retains higher ordered subgoal completion and oracle success on tasks requiring revisits, complementing the stage-wise results above.

Critical Weakness W2 (original):

Comparative behavioral analysis on ErrandVLN is limited. Figure 12 provides useful failure cases for VLN-JEPA itself, while Figure 14 compares baseline and proposed trajectories on R2R-CE. However, the paper does not provide corresponding paired qualitative trajectory comparisons with existing navigation models on ErrandVLN. This makes it difficult to understand where existing models fail in the new task and what behavior VLN-JEPA changes—for example, landmark identification, goal ordering, revisit navigation, local motion, or stopping. Aggregate performance differences alone do not resolve this question.

Question for the Authors Q2 (original):

Comparative cases: Could the authors provide paired ErrandVLN trajectories for VLN-JEPA and a representative existing navigation model, identifying the point of divergence and the associated failure behavior? Please clarify whether the examples illustrate typical failures or selected instances.

Response:

We thank the reviewer for suggesting trajectory comparisons on the same episodes.

The revised manuscript adds paired trajectory comparisons between DualVLN and VLN-JEPA on the same episodes in Appendix C.8, covering both successful and unsuccessful execution.

In Examples 2 and 7, VLN-JEPA continues toward the final goal and stops nearby, whereas DualVLN terminates earlier. Examples 1 and 11 show different regional choices by the two methods. These cases visually complement the goal-progress metrics.

Among VLN-JEPA's failure cases, Example 3 contains repeated traversals, while the stopping positions in Examples 9 and 12 still deviate from the final goals. The failure visualizations summarize three phenomena: goal confusion under repetitive visual structures, local loops when backtracking is required, and failure to find goals behind unobserved partitions — corresponding to challenges in goal discrimination, long-term spatial memory, and active exploration.

Examples 3, 9, and 12 illustrate observed failures involving repeated traversal and inaccurate stopping.

Critical Weakness W3 (original):

Validation of generated instructions could be clarified. Appendix A.3 provides instruction-generation prompts that explicitly require coverage of the waypoints in order. However, any post-generation checks on the final instructions are less clear. A brief clarification of whether such checks are performed would be helpful.

Question for the Authors Q3 (original):

Annotation quality: Beyond the requirements specified in the generation prompts, are any post-generation checks performed to verify consistency with the intended landmarks and their visiting order?

Response:

We thank the reviewer for reminding us to explain the post-generation verification of instructions.

Yes, post-generation checks are performed. The scope and method are as follows.

In this revision, we provide a human-reviewed subset of 1,000 test trajectories, checking physical reachability and multi-goal consistency for each trajectory, including whether instructions match the intended landmarks and their visiting order. On this same subset, we compare NaViLA, StreamVLN, and VLN-JEPA under matched in-domain fine-tuning settings.

Given the dataset size and the human cost of exhaustive review, we do not currently plan a complete manual audit of the entire dataset. The human-reviewed results in this revision cover the 1,000-trajectory subset described above.

Reviewer_rUepOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary
  1. Existing continuous VLN benchmarks mostly evaluate single-goal route following, whereas real household requests often require visiting multiple places in an intent-driven order. Motivated by this gap, this paper introduces ErrandVLN, which pairs ordered multi-stop trajectories with role-conditioned instructions on InteriorGS to evaluate generalization across scenes, paths, and language styles.
  2. To mitigate compounding errors over multiple stages, this paper proposes VLN-JEPA, TP-Flow first proposes continuous candidate trajectories, NaviLat then predicts future latent states for each candidate, and a VLM finally selects the next action using these Future Tokens together with the instruction.
  3. On the ErrandVLN scene-unseen split, the method performs better overall on stage completion and collision avoidance, and transferring the same trajectory proposal and latent prediction modules to another backbone also yields gains on R2R-CE.
Critical strengths
  1. ErrandVLN casts continuous VLN as ordered multi-stop navigation organized by household intent, and maps the same physical requirements to diverse role-conditioned phrasings. This differs from both single-goal route following and common multi-stage object-finding setups, and fills a separately evaluable task gap.
  2. Rather than reacting from history alone or decoding future frames, VLN-JEPA first proposes multiple continuous trajectories with TP-Flow, predicts candidate-specific latent futures with NaviLat, and lets the VLM compare these outcomes through Future State Tokens before selecting the next action, which better matches the need to keep multi-stage decisions consistent over a long horizon.
  3. Empirically, the 2.35B model outperforms larger baselines such as DualVLN on GCR, GC-SPL, and collisions. Removing TP-Flow or NaviLat, or perturbing Future Tokens, degrades the metrics, and attaching the same modules to JanusVLN still improves R2R-CE, indicating that the gains are not solely from fine-tuning on the new data.
Critical weaknesses
  1. Predictive navigation with action-conditioned future modeling is already explored by methods such as NavMorph, VISTAv2, PiJEPA, and LatentPilot. VLN-JEPA mainly differs in combining continuous flow proposals, JEPA-style latent rollouts without pixel decoding, and Future Tokens for a VLM controller. This looks more like module assembly, clean same-setting comparisons to these predictive VLN baselines are still missing.
  2. The method emphasizes GCR and collisions, and explains DualVLN’s premature stopping and short trajectories, yet DualVLN still has higher SR.
  3. On ErrandVLN, DualVLN, NaVid, StreamVLN, and similar baselines use public checkpoints without ErrandVLN fine-tuning, while VLN-JEPA is fully trained on this data in two stages. Same-distribution SFT (w/o Both) already reaches 13.0% GCR, versus only 5.2% for the zero-shot base model, so the main results do not directly show that the TP-Flow + NaviLat architecture is stronger.
Questions for authors
  1. TP-Flow strongly depends on depth-derived occupancy, whereas the compared RGB VLM baselines may not receive equivalent inputs, the gains may come from the local map and trajectory scoring rather than from the latent world model alone.
  2. The R2R-CE transfer freezes TP-Flow and NaviLat pretrained on ErrandVLN, fine-tunes only LoRA and the Projector, and attaches them to JanusVLN. This looks more like an add-on module boost than evidence that the full method stands on its own for standard VLN.
  3. NaviLat rolls out only J=5J=5 steps, while errands average about 137 steps and have at most 4 stages. It is unclear how such short-horizon latent foresight can support long-horizon ordered constraints, the effect looks closer to improving local traversability than true stage-level planning.
  4. The main text does not clearly specify the ordering of Future Tokens, how many tokens each candidate corresponds to, or whether the VLM is told the candidate ranking rules.
  5. What the latent ss actually encodes, how it aligns with the visual encoder, and how much the regularization beyond next-state prediction contributes, all lack intuitive analysis.
  6. In the full model, the VLM receives Future State Tokens: latents predicted by NaviLat, projected into special tokens, and fed together with images and text. In the w/o NaviLat setting, TP-Flow trajectories are instead written as text and given to the VLM. This makes it hard to tell whether the drop comes from removing future prediction or from text trajectories simply weaker than tokens.
9 comments

Critical Weakness W1 (original):

Predictive navigation with action-conditioned future modeling is already explored by methods such as NavMorph, VISTAv2, PiJEPA, and LatentPilot. VLN-JEPA mainly differs in combining continuous flow proposals, JEPA-style latent rollouts without pixel decoding, and Future Tokens for a VLM controller. This looks more like module assembly, clean same-setting comparisons to these predictive VLN baselines are still missing.

Response:

We thank the reviewer. The revised manuscript compares NavMorph and LatentPilot on R2R-CE Val-Unseen and includes matched-backbone ablations on ErrandVLN.

Table 1. R2R-CE Val-Unseen comparison. Methods follow their reported configurations. Δ is augmented minus original JanusVLN.

MethodNE (m) ↓OSR (%) ↑SR (%) ↑SPL (%) ↑
NavMorph5.7556.947.933.2
LatentPilot4.4166.362.058.0
JanusVLN4.7865.260.556.8
JanusVLN + TP-Flow + NaviLat4.5969.262.559.7
Δ (vs. JanusVLN)−0.19+4.0+2.0+2.9

Augmenting JanusVLN improves SR/SPL by 2.0/2.9 points. Our contribution concerns how future states are generated and supplied to language-conditioned action generation:

1. Differences from predictive navigation methods.

  • VISTAv2 [1] generates future views with a diffusion Transformer in VAE latent space and decodes RGB for an online value map. Generation and decoding add computation, and geometric errors can affect value estimation. NaviLat directly predicts visual latent states without image reconstruction.
  • PiJEPA [2] combines an Octo prior with MPPI planning in JEPA latent space, scoring predictions against a goal image. ErrandVLN supplies language-defined ordered subgoals, without goal images. Our VLM uses instructions, history, and predicted states to generate actions.
  • NavMorph [3] emphasizes online adaptation through compact latent states and CEM. Latent dynamics do not impose explicit geometric obstacle constraints. TP-Flow screens motion candidates using local traversability before NaviLat predicts their visual consequences.
  • LatentPilot [4] propagates a Pilot Token across decision steps along execution history. NaviLat independently rolls out multiple candidate futures from the same history at each decision step, preserving candidate identity and temporal order. These futures inform the VLM's action generation.

2. TP-Flow–NaviLat coupling.

TP-Flow uses conditional flow matching and ODE integration to generate 16 trajectories, retaining 5 by traversability, progress, and smoothness scores. Trajectories become candidate actions, not direct execution commands. NaviLat predicts 5 states per candidate (25 tokens total), which inform VLM action generation alongside RGB and language.

Table 2. Component ablations on ErrandVLN Test Scene-Unseen. Δ is the full model minus the specified ablation.

ConfigurationSR (%) ↑GCR (%) ↑PL (m)Col ↓
Without NaviLat9.816.219.927.0
Without TP-Flow11.218.112.538.5
Full model12.620.511.326.6
Δ (vs. w/o NaviLat)+2.8+4.3−8.6−0.4
Δ (vs. w/o TP-Flow)+1.4+2.4−1.2−11.9

Without NaviLat, mean path length rises from 11.3 to 19.9 m; without TP-Flow, mean collisions rise from 26.6 to 38.5. These results support complementary roles in candidate feasibility and future-state information. Visual encoding, TP-Flow, and NaviLat jointly take 22.1 ms; total policy inference takes 94 ms (Appendix C.7).

3. State representation beyond LeWM.

We retain LeWM's [5] next-state objective and SIGReg regularizer. NaviLat combines DINOv2 patches with 4 spatial queries instead of a single CLS token. Attention maps show distinct, overlapping regional responses. Across 1,000 validation states, dimensions exceeding variance 0.05 account for 17.8%, 45.2%, and 88.5% in Vanilla LeWM, LeWM (ViT + Spatial), and NaviLat, respectively.

References: [1] VISTAv2, arXiv:2512.00041; [2] PiJEPA, arXiv:2603.25981; [3] NavMorph, arXiv:2506.23468; [4] LatentPilot, arXiv:2603.29165; [5] LeWorldModel, arXiv:2603.19312.

Critical Weakness W2 (original):

The method emphasizes GCR and collisions, and explains DualVLN’s premature stopping and short trajectories, yet DualVLN still has higher SR.

Response:

We thank the reviewer for reminding us to present the comparison with DualVLN more accurately.

DualVLN has higher overall SR, while VLN-JEPA has higher GCR and fewer mean collisions. The stage-wise results clarify which aspects improve.

DualVLN achieves an overall SR of 15.10%, compared with 12.60% for VLN-JEPA. VLN-JEPA achieves 20.54% GCR, above DualVLN's 12.99%, and also has fewer mean collisions. Results stratified by the number of stages further illustrate the differences.

Table 3. Stage-wise comparison with DualVLN on the main ErrandVLN Test Scene-Unseen evaluation set. Cells report GCR / SR (%). DualVLN uses public weights. Δ denotes VLN-JEPA minus DualVLN. All differences are calculated from the displayed values.

MethodM=1M=2M=3M=4Overall
DualVLN26.87 / 44.7816.02 / 20.1810.41 / 9.479.08 / 8.3612.99 / 15.10
VLN-JEPA37.31 / 32.8424.78 / 17.2116.73 / 9.4715.84 / 6.1120.54 / 12.60
Δ+10.44 / −11.94+8.76 / −2.97+6.32 / 0.00+6.76 / −2.25+7.55 / −2.50

DualVLN has higher SR at M=1, M=2, and M=4, and ties our method at M=3; VLN-JEPA has higher GCR in every group. GCR and GC-SPL use a stricter 1.5 m threshold, measuring ordered subgoal completion in detail. SR follows the commonly used 3.0 m success radius in VLN, requiring all milestones and STOP near the final goal. They provide complementary evaluations of fine-grained goal progress and full-task success.

Critical Weakness W3 (original):

On ErrandVLN, DualVLN, NaVid, StreamVLN, and similar baselines use public checkpoints without ErrandVLN fine-tuning, while VLN-JEPA is fully trained on this data in two stages. Same-distribution SFT (w/o Both) already reaches 13.0% GCR, versus only 5.2% for the zero-shot base model, so the main results do not directly show that the TP-Flow + NaviLat architecture is stronger.

Question for the Authors Q1 (original):

TP-Flow strongly depends on depth-derived occupancy, whereas the compared RGB VLM baselines may not receive equivalent inputs, the gains may come from the local map and trajectory scoring rather than from the latent world model alone.

Response:

We thank the reviewer for suggesting that we further distinguish gains from domain fine-tuning and architectural components.

We fine-tune NaViLA and StreamVLN on the official ErrandVLN training split using optimization schedules and supervision objectives matched to VLN-JEPA. All three models are evaluated on the same human-verified subset of 1,000 trajectories.

Table 4. Overall comparison after matched in-domain fine-tuning. Δ denotes VLN-JEPA minus StreamVLN.

MethodSR (%) ↑OSR (%) ↑GCR (%) ↑GC-SPL (%) ↑PL (m)Col ↓
NaViLA9.6024.3011.519.933.5525.67
StreamVLN19.4037.0020.8818.635.8859.57
VLN-JEPA23.0046.3030.2122.179.1421.76
Δ (vs. StreamVLN)+3.60+9.30+9.33+3.54+3.26−37.81

VLN-JEPA exceeds StreamVLN by 3.60 points in SR, 9.33 in GCR, and 3.54 in GC-SPL, with mean collisions reduced from 59.57 to 21.76. The gains therefore persist after external baselines receive in-domain fine-tuning.

To separate domain fine-tuning from architectural contributions, we use a matched ablation sequence with the same Qwen3.5-2B backbone throughout.

Table 5. Core architectural ablations with the same Qwen3.5-2B backbone on ErrandVLN Test Scene-Unseen. Δ denotes the full model minus SFT-only. All differences are calculated from the displayed values.

ConfigurationSR (%) ↑GCR (%) ↑GC-SPL (%) ↑PL (m)Col ↓
Base VLM (zero-shot)3.65.24.05.490.6
w/o Both (SFT)5.213.011.46.356.7
w/o NaviLat (MLP)5.315.412.97.992.4
w/o NaviLat9.816.212.919.927.0
w/o TP-Flow11.218.114.312.538.5
Full model12.620.516.111.326.6
Δ (vs. SFT-only)+7.4+7.5+4.7+5.0−30.1

Relative to zero-shot performance, SFT improves GCR by 7.8 percentage points. Relative to SFT, the full system further improves GCR by 7.5 and SR by 7.4 percentage points. Retaining TP-Flow while removing NaviLat reduces mean collisions from 56.7 to 27.0, with 16.2% GCR. Adding NaviLat raises GCR to 20.5% and reduces mean path length from 19.9 m to 11.3 m.

The external visual input is RGB. Depth Anything estimates depth from RGB only for TP-Flow's internal candidate generation and scoring. Trajectories are converted into candidate actions for future rollouts rather than executed directly. Neither NaviLat nor the VLM receives depth maps or extra depth features. We have corrected the depth annotation. NaviLat takes RGB-encoded latent history and action parameters as input. The w/o TP-Flow configuration retains NaviLat but uses five fixed action branches, exceeding SFT by 5.1 GCR and 6.0 SR percentage points, demonstrating that latent rollouts remain effective when the candidate source changes. The capacity-matched MLP control further confirms the role of action-conditioned future rollouts.

Reviewer_QEasOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper makes two main contributions: ErrandVLN, a benchmark for multi-stage household navigation, and VLN-JEPA, a navigation framework that combines a trajectory proposer, a JEPA-style latent world model, and a VLM controller. The world model predicts future representations conditioned on candidate motions, which are then projected into tokens and provided to the VLM to support action generation.

Critical strengths

The benchmark has potential research value. ErrandVLN combines ordered goals, revisits, and diverse household instructions in continuous navigation, covering 913 scenes, 65,134 physical trajectories, and 502,233 instructions. Its evaluation splits address generalization across scenes, paths, and instruction styles, making it potentially useful for studying multi-stage navigation beyond the proposed model.

The ablations provide evidence of practical benefits from the proposed modules. With the same controller training examples and optimization settings, the full model improves GCR from 13.0% to 20.5% and SR from 5.2% to 12.6% over the fine-tuned VLM baseline (Table 4). Removing either the trajectory proposer or the world model reduces performance, indicating that the gains cannot be explained solely by task-specific VLM fine-tuning.

Critical weaknesses
  1. Unmatched comparison conditions limit conclusions about methodological superiority. On ErrandVLN, external baselines receive no dataset-specific fine-tuning, whereas the proposed method is trained on the benchmark. On R2R-CE, the augmented JanusVLN additionally uses depth, while the original baseline uses RGB only (Table 3). These results demonstrate system performance and suggest transfer potential, but do not cleanly attribute the improvements to the proposed architecture. Comparisons with matched training data, adaptation procedures, and observation inputs would strengthen this claim.
  2. The mechanism underlying the benefits of future prediction remains insufficiently established. Table 4 provides meaningful module ablations, but comparing textual trajectories with predicted latent tokens changes both the representation format and the information supplied to the controller. This does not fully separate the benefits of forecasting from those of additional feature representations. Likewise, zeroing and shuffling tokens in Table 5 demonstrate dependence on these inputs, but do not establish accurate use of candidate-specific future consequences. A non-predictive control with matched representation format and capacity, together with an analysis linking prediction quality to navigation gains, would better substantiate the central mechanistic claim.
Questions for authors
  1. Clarification of comparison conditions. Section 5.1 states that external baselines receive no ErrandVLN-specific adaptation, and Table 3 indicates that the augmented JanusVLN additionally uses depth. Are there comparisons that match training data, adaptation procedures, and observation inputs? Please clarify which existing results support architectural improvements independently of these differences. This would affect my assessment of the claimed advantages over prior methods.
  2. Clarification of the contribution of future prediction. Table 4 already includes useful controls using textual candidate trajectories and fixed action branches. However, the former also changes the representation format, while Table 5 primarily demonstrates sensitivity to token content and ordering. What evidence distinguishes the benefit of predicting future states from that of providing additional state/action features? Any existing analysis linking prediction accuracy to navigation performance would help assess the central claim about prospective reasoning.
3 comments

W1 / Q1 — Comparison fairness (summary).**

Do the improvements persist when training data, adaptation procedures, and observation inputs are matched? Please clarify which results isolate the architectural contribution from domain adaptation and additional depth input.

Response:

We thank the reviewer for pointing out the need to clarify the comparison conditions and for suggesting matched controls.

We have added in-domain fine-tuning results for StreamVLN and NaViLA to compare the systems after matched domain-specific adaptation. The matched-backbone ablations further examine the contributions of TP-Flow and NaviLat.

To match domain adaptation conditions, we fine-tune StreamVLN and NaViLA on the official ErrandVLN training split using optimization schedules and supervision objectives matched to VLN-JEPA, and evaluate them on the same human-verified subset of 1,000 trajectories.

Table 1. Overall comparison after matched in-domain fine-tuning on the same human-verified subset of 1,000 ErrandVLN trajectories. Δ denotes VLN-JEPA minus StreamVLN. Differences for percentage metrics are in percentage points; other differences retain the column units. All differences are calculated from the displayed values.

MethodSR (%) ↑OSR (%) ↑GCR (%) ↑GC-SPL (%) ↑PL (m)Col ↓
NaViLA9.6024.3011.519.933.5525.67
StreamVLN19.4037.0020.8818.635.8859.57
VLN-JEPA23.0046.3030.2122.179.1421.76
Δ (vs. StreamVLN)+3.60+9.30+9.33+3.54+3.26−37.81

After in-domain fine-tuning, VLN-JEPA exceeds StreamVLN by 3.60, 9.33, and 3.54 percentage points in SR, GCR, and GC-SPL, respectively, while reducing mean collisions per episode from 59.57 to 21.76. This shows that our system retains higher task completion and fewer collisions after the external baselines receive the same domain-specific training.

To further examine whether the gains arise only from single-stage tasks, we provide the following stratified results. Here, M denotes the number of ordered milestones in a task, rather than the conditional success rate of the M-th stage within an episode.

Table 2. Stage-wise comparison after matched in-domain fine-tuning on the same human-verified subset of 1,000 ErrandVLN trajectories. Values are means [95% bootstrap confidence intervals].

MethodMGCR (%) ↑GC-SPL (%) ↑SR (%) ↑
NaViLA110.94 [7.8, 14.6]8.08 [5.4, 11.2]16.72 [12.8, 21.3]
—211.04 [8.2, 14.4]9.72 [7.1, 12.9]5.71 [3.7, 8.4]
—312.98 [8.1, 18.6]12.76 [8.0, 18.4]5.23 [2.3, 9.3]
—412.76 [6.8, 20.1]12.01 [6.3, 19.3]9.38 [4.2, 16.7]
StreamVLN124.01 [19.6, 28.8]19.18 [15.1, 23.7]27.66 [22.8, 32.8]
—220.47 [16.6, 24.6]19.16 [15.4, 23.3]18.11 [14.4, 22.3]
—318.60 [13.2, 24.8]17.90 [12.6, 24.0]10.47 [6.4, 15.7]
—415.89 [9.1, 23.7]15.89 [9.1, 23.7]12.50 [6.3, 20.8]
VLN-JEPA138.60 [33.4, 44.1]22.59 [18.3, 27.3]32.83 [28.0, 38.3]
—228.29 [24.1, 32.8]23.61 [19.6, 28.0]19.60 [15.9, 23.8]
—324.61 [18.4, 31.4]20.82 [14.9, 27.3]18.02 [12.8, 24.4]
—419.53 [12.2, 27.9]17.15 [10.2, 25.3]12.50 [6.3, 20.8]

VLN-JEPA achieves higher point estimates of GCR and GC-SPL in every group from M=1–4. At M=3, its SR is 18.02%, compared with 10.47% for StreamVLN and 5.23% for NaViLA. At M=4, its GCR is 3.64 percentage points higher than StreamVLN's, while both achieve 12.50% SR. The 95% confidence intervals also report the uncertainty of each group's results.

Regarding depth, the augmented system still uses monocular RGB as its external visual input. Depth Anything's output is used only by TP-Flow to generate and score candidate trajectories. These trajectories are not directly executed; they are first converted into candidate actions for NaviLat to perform future rollouts based on RGB latent history. The VLM receives tokens mapped from the predicted states, rather than depth maps or additional depth features. We have corrected the depth annotation in the Observation columns.

Component ablations using the same backbone and matched training settings further distinguish the modules' roles. Retaining TP-Flow while removing NaviLat yields SR/GCR of 9.8%/16.2%, compared with 12.6%/20.5% for the full model. Removing TP-Flow and driving NaviLat with fixed action branches still achieves 11.2%/18.1% SR/GCR, above the SFT-only results of 5.2%/13.0%, showing that latent rollouts are also effective with fixed candidates. The capacity- and format-matched MLP control further controls candidate inputs, token count, and training conditions; its detailed results are provided in the next response.

W2 / Q2 — Contribution of future prediction (summary).**

What evidence distinguishes future-state prediction from additional features or changes in representation format? Please provide a capacity- and format-matched non-predictive control and analyze the relationship between prediction accuracy and navigation performance.

Response:

We thank the reviewer for the specific suggestions on validating the future prediction mechanism.

We provide a capacity- and format-matched non-predictive control, prediction-quality analysis, and rollout-horizon ablation to examine the contribution of future prediction.

First, we introduce a capacity- and representation-format-matched non-predictive MLP control. This control uses the same historical observations, candidate action inputs, token count, training settings, and training objective as the full model, with matched parameter count. The difference is that a static MLP feature mapping replaces NaviLat's action-conditioned future-state rollouts. The results are shown below.

Table 3. Capacity- and representation-format-matched non-predictive MLP comparison on ErrandVLN Test Scene-Unseen. Δ denotes the full model minus the MLP control. All differences are calculated from the displayed values.

MethodSR (%) ↑GCR (%) ↑GC-SPL (%) ↑PL (m)Col ↓
w/o NaviLat (MLP)5.315.412.97.992.4
VLN-JEPA (Full model)12.620.516.111.326.6
Δ+7.3+5.1+3.2+3.4−65.8

Relative to the MLP control, the full model improves SR and GCR by 7.3 and 5.1 percentage points, respectively, and reduces mean collisions per episode from 92.4 to 26.6. A non-predictive static feature mapping lacks modeling of continuous spatiotemporal physical dynamics and cannot explicitly represent the geometric and semantic evolution of the robot's view after candidate actions are executed. With parameter count, inputs, token count, and training conditions matched, the gains arise from the world model's prospective rollouts of physical transitions, rather than simply increased feature capacity.

Second, we analyze the relationship between prediction quality and navigation performance. Under the same model configuration and evaluation protocol, we stratify test episodes by NaviLat's prediction error and compare GCR and GC-SPL across groups.

Table 4. Prediction quality and navigation performance grouped by prediction error on ErrandVLN Test Scene-Unseen. 95% bootstrap confidence intervals are shown.

GroupMean prediction error ↓GCR (%) ↑GC-SPL (%) ↑
Low Error0.05923.7 [19.2, 27.9]18.6 [15.3, 21.6]
Medium Error0.09321.6 [17.6, 26.8]16.2 [13.6, 19.6]
High Error0.15016.3 [11.7, 20.8]13.5 [9.8, 17.0]
Overall0.10120.5 [18.6, 22.5]16.1 [14.7, 17.6]

As mean prediction error decreases from 0.150 to 0.059, GCR increases from 16.3% to 23.7%, and GC-SPL increases from 13.5% to 18.6%, indicating that more accurate latent future predictions correspond to greater goal progress and higher path efficiency.

Third, we use a rollout-horizon ablation to examine the effect of temporal prediction range. We fix the number of candidates at K=5 and set H to 1, 2, 3, 4, 5, and 10. The values correspond to Figure 4 in the paper.

Table 5. Rollout-horizon ablation on ErrandVLN Test Scene-Unseen with the number of candidates fixed at K=5. H denotes rollout steps per candidate. GCR and GC-SPL are percentages.

Rollout horizon H per candidateGCR (%) ↑GC-SPL (%) ↑
14.33.9
214.612.0
316.913.9
418.615.3
5 (default)20.516.1
1019.216.3

At H=1, GCR and GC-SPL are only 4.3% and 3.9%. Extending to H=5 increases them to 20.5% and 16.1%. Further extension to H=10 reduces GCR to 19.2%, showing that longer prediction horizons do not necessarily improve goal completion. We therefore adopt H=5 as the default.

Dear Reviewer QEas,

As the author-reviewer discussion period is drawing to a close within the next 24 hours, we would like to sincerely thank you again for your constructive and insightful review of our submission.

We hope our previous response and newly added experiments have properly addressed your concerns, particularly regarding:

  1. Fairness of Comparison (W1 / Q1): We conducted rigorous in-domain fine-tuning for external baselines (StreamVLN and NaViLA) under identical training schedules and objectives on the 1,000 human-verified benchmark subset. As shown in Table 1 & Table 2, VLN-JEPA consistently maintains substantial leads across all milestones (M=1M=1–44), achieving +3.60+3.60\\% SR and +9.33+9.33\\% GCR over StreamVLN while reducing collision rates by over 6363\\%. We also clarified that external visual inputs remain monocular RGB (depth is solely used inside TP-Flow for candidate generation, not fed into the VLM).

  2. Mechanism of Future Prediction (W2 / Q2): Our component ablations and capacity/format-matched MLP controls further decouple the benefits of predictive latent forecasting from mere capacity expansion or static feature representation.

We would be immensely grateful if you could let us know whether our response and the new matched-baseline results have adequately resolved your questions. If so, we respectfully and kindly ask if you might consider reflecting this in a raised rating.

Should you have any remaining questions or require any further clarification before the deadline, we remain fully committed and eager to address them promptly.

Sincerely,
The Authors of Submission 51452

Comments on the paper

1 comment

General Response to All Reviewers

We sincerely thank Reviewers QEas and K6wb for their positive assessments and for recognizing the value of ErrandVLN as a benchmark for continuous multi-stage navigation, the detailed benchmark construction and evaluation design, and the experimental evidence supporting the practical benefits of the proposed modules. We appreciate their recognition of both the task formulation and the methodological contribution.

We also thank all four reviewers for their careful reading and constructive feedback. The questions raised by Reviewers rUep and G7xi, together with the suggestions from Reviewers QEas and K6wb, have helped us identify where additional controlled evidence and clearer explanations were needed. In particular, the reviews highlight the importance of matching comparison conditions, establishing the contribution of future prediction, clarifying the relationship between local foresight and long-horizon task execution, and accurately describing the task inputs and evaluation protocol. We have revised the manuscript and supplemented the experiments accordingly.

For the benchmark contribution, we clarify that ErrandVLN evaluates the navigation requirements of semantically coherent, high-level household requests. Agents receive natural-language instructions and RGB observations; landmarks are used internally for trajectory construction, instruction generation, and evaluation, rather than supplied as a structured goal list. We further clarify the distinctions from related benchmarks and provide a human-reviewed subset of 1,000 test trajectories, checked individually for physical reachability and multi-goal consistency.

For the experimental comparisons, we add NaViLA and StreamVLN fine-tuned on the official ErrandVLN training split under matched optimization schedules and supervision objectives, and evaluate all three models on the same human-verified subset. We also provide results stratified by the number of task stages. To examine the role of future prediction, we include a capacity- and representation-format-matched non-predictive MLP control, an analysis relating prediction quality to navigation performance, and an ablation of the rollout horizon. These experiments complement the component ablations, with confidence intervals reported for the relevant comparisons and grouped results.

We also clarify the observation interface and information flow. The system's external visual input is monocular RGB. Depth Anything estimates depth from the current RGB image solely for traversability modeling and candidate trajectory generation and scoring within TP-Flow. The proposed trajectories are converted into action sequences for NaviLat's future-state prediction rather than executed directly. NaviLat takes RGB latent history and candidate actions as input, and the VLM receives the projected future-state tokens alongside RGB observations and language instructions. Neither NaviLat nor the VLM receives depth maps or additional depth features. We have corrected the depth checkmark in the Observation columns and clarified these interfaces in the manuscript.

Finally, we clarify that the R2R-CE experiment evaluates the transfer of the combined TP-Flow and NaviLat modules to JanusVLN, while the ErrandVLN ablations examine individual component contributions. We have corrected the overly broad performance claim on R2R-CE, included the comparison with AwareVLN, and discussed its relationship to our method in Related Work. We have also expanded the qualitative comparisons and explanations of the evaluation metrics. Detailed responses to each comment follow.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.