acceptodds
Under review as a conference paper at ICLR 2027

PoseVLN: Learning Geometric State Evolution for Vision-Language Navigation

Abstract

Vision-Language Navigation (VLN) requires an embodied agent to relate past experience, current spatial state, and future motion while following natural-language instructions. Recent approaches improve these capabilities through long-horizon memory, state-aware reasoning, and future prediction, yet these components are often modeled separately, leaving navigation state evolution insufficiently connected across time. We argue that camera pose provides a natural geometric representation for bridging past, present, and future states. Based on this insight, we propose PoseVLN, a pose-aware VLN framework that jointly learns navigation decisions and geometric state evolution. PoseVLN organizes visual memory and the current observation in an episode-relative coordinate frame, and jointly predicts the next navigation actions together with a local Pose over the same horizon. The predicted Pose is composed with the current trajectory-level pose to obtain the future pose state, which is recursively propagated throughout navigation, forming a continuous geometric state evolution from past memory to future motion. Experiments on the R2R-CE and RxR-CE Val-Unseen splits demonstrate the effectiveness of explicitly modeling geometric state evolution, with PoseVLN outperforming the strongest compared baselines by 2.6 and 2.9 percentage points in success rate, respectively, using only single RGB observations.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Vision-Language Navigation, Embodied Navigation, Camera Pose, Geometric Reasoning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_uYTFOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes PoseVLN, which builds upon prior works such as StreamVLN to address the issue of inadequate long-term spatial state maintenance in VLN by explicitly modeling camera poses, integrating pose information into visual memory, and jointly predicting navigation actions and local pose changes. While the paper introduces a geometrically grounded modeling framework with intuitive motivations and reports competitive navigation results, it suffers from significant shortcomings in both experimental setup and result analysis.

Critical strengths

The paper treats camera poses as unified geometric variables and uses them as references for historical memory and spatial states, which is a well-motivated design choice. Compared to relying solely on historical image features or action history, pose-aware memory provides clear spatial references for different observations. By integrating both into a unified navigation framework, the overall design exhibits strong interpretability. Furthermore, experimental results across multiple benchmarks demonstrate the effectiveness of the proposed method.

Critical weaknesses

I believe the authors have not adequately clarified the fundamental differences between their approach and existing geometry-aware navigation or pose-conditioned visual models, primarily in the following aspects:

  1. The current experimental results fail to sufficiently isolate and justify the individual contributions of geometric supervision and pose inputs.
  2. Although ground-truth camera poses provided by the Habitat simulator are utilized to construct local pose supervision during training, I doubt whether the learned geometric motion patterns can generalize to real-world scenarios across different visual domains.
  3. The primary training objective focuses predominantly on local pose regression, and such local supervision is insufficient to guarantee long-term geometric consistency.
Questions for authors

In addition to the points raised in the weaknesses, I would like the authors to address the following:

  1. In real-world scenarios, pose errors accumulate over time. Please analyze the relationship between accumulated pose errors and navigation success rate.
  2. How does the model maintain its geometric state when predicted actions are inconsistent with actual execution outcomes, such as collisions in Habitat or motion errors in real-world?
  3. I believe the Directional Progress Loss warrants a more detailed explanation. This loss constrains local translation regression under a specific coordinate parameterization and cannot be interpreted as ensuring that the camera has made sufficient progress along the true motion direction in physical space. The authors should provide further clarification and evidence to verify that the benefits of this loss are not merely an optimization bias stemming from a specific parameterization.
9 comments

We sincerely thank the reviewer for the thoughtful review, constructive feedback, and positive comments on the motivation and interpretability of PoseVLN. The reviewer's suggestions have helped us clarify our contributions and strengthen the experimental analysis. In the following responses, we further explain the role of pose-conditioned navigation and provide additional evidence on visual robustness, accumulated geometric errors, execution discrepancies, and the Directional Progress Loss.

W1: Separating the contributions of geometric supervision and pose inputs.

We thank the reviewer for highlighting the need to distinguish PoseVLN from existing geometry-aware navigation and pose-conditioned visual models. Existing approaches primarily focus on learning spatial representations [1-4]. In contrast, PoseVLN uses camera pose as an evolving geometric state, predicting ΔPose over the same horizon as the navigation actions and recursively feeding updated poses into memory and subsequent decisions.

To isolate the contributions of geometric supervision and pose inputs, we additionally trained an Auxiliary-only variant that retains the same geometric objectives, weights, and schedules as full PoseVLN but removes current and historical pose inputs during both training and inference. Under matched training settings and the same aggregated DAgger data, we obtain the following R2R-CE Val-Unseen results:

Training configurationSR (%) ↑SPL (%) ↑NE (m) ↓
Action-only57.053.44.86
Action + auxiliary geometric supervision57.554.24.92
Full PoseVLN64.659.44.01

Geometric supervision. Without pose inputs, auxiliary geometric supervision improves SR from 57.0% to 57.5% and SPL from 53.4% to 54.2%, showing a modest independent benefit. To further examine whether geometric supervision helps the model interpret pose-related information, we provide action-derived poses as text prompts at inference:

Training configurationSR (%) ↑SPL (%) ↑NE (m) ↓
Action-only47.142.85.83
Action + auxiliary geometric supervision52.448.95.45

Compared with their respective no-pose baselines, SR decreases by 9.9 and 5.1 points. The smaller degradation with auxiliary supervision suggests that geometric learning reduces the model's sensitivity to action-derived pose prompts.

Pose inputs and recursive state. Beyond auxiliary supervision, full PoseVLN jointly learns pose-conditioned navigation and recursive geometric state updates, improving SR from 57.5% to 64.6%. This advantage also holds on RxR-CE, where SR increases from 52.9% for Action-only to 56.0% for Auxiliary-only and 62.0% for full PoseVLN (Appendix L, Table 16).

Together, these results support the contribution of jointly learned geometric state evolution beyond auxiliary supervision or directly providing pose information. Geometric supervision provides a modest independent benefit and improves adaptability to pose inputs, while integrating pose-conditioned memory with recursive state updates yields substantially larger navigation gains. We have updated Tables 3 and 5 and their accompanying discussion accordingly.

[1] JanusVLN: Decoupling Semantics and Spatiality with Dual Implicit Memory for Vision-Language Navigation. ICLR, 2026.

[2] GA-VLN: Geometry-Aware BEV Representation for Efficient Vision-Language Navigation. CVPR, 2026.

[3] Uncertainty-Aware Gaussian Map for Vision-Language Navigation. ICLR, 2026.

[4] Bridging the 2D-3D Gap: A Hierarchical Semantic-Geometric Map for Vision Language Navigation. CVPR, 2026.

W2: Generalization of learned geometric motion across visual domains.

We appreciate the reviewer's concern. Although Habitat ground-truth camera poses are used to construct local pose supervision during training, PoseVLN predicts ΔPose recursively from RGB observations without ground-truth poses or external odometry at inference.

To evaluate whether the learned geometric motion patterns generalize across different visual domains, we conducted two controlled visual perturbation experiments: reducing image brightness to 0.5× and applying Gaussian blur with radius 2. SR decreases from 64.6% to 62.5% under reduced brightness and to 58.4% under blur, corresponding to drops of 2.1 and 6.2 points, respectively. Both results remain above the clean-input action-only baseline of 57.0%, indicating tolerance to the tested visual changes, particularly brightness variation (Appendix H, Table 10).

Real-world evaluation. We further evaluated PoseVLN on Unitree Go2 in an outdoor and an indoor environment. Each environment includes three navigation tasks, with three trials per task under slightly varied starting positions. PoseVLN successfully completes 11 of 18 trials (61.1%), with at least two successful trials in four of six task settings, covering doorway traversal, straight motion, and turning behaviors. These results provide quantitative evidence that PoseVLN can perform language-guided navigation in the evaluated real-world scenarios (Appendix M).

Together, the visual perturbation experiments and real-world trials support PoseVLN's robustness to the tested appearance changes and its feasibility for real-world navigation.

W3 & Q1: Long-term geometric consistency and the relationship between accumulated pose errors and navigation success.

We thank the reviewer for raising this issue. Although ΔPose regression uses local supervision, training on trajectory-sampled windows in a shared episode-relative frame helps the model learn geometric relationships across steps. These relationships support recursive geometric state evolution beyond individual pose predictions. The original multi-horizon results in Tables 4 and 5 already provide evidence that PoseVLN learns useful geometric states across successive updates (H∈{4,8,12,16}H\in\{4,8,12,16\}), with lower pose errors consistently observed in successful episodes.

To further evaluate long-term geometric consistency over complete navigation trajectories and its relationship with navigation success, we measure translational ATE and rotational RMS error and independently divide episodes into five error quantiles:

Error quantileMean ATE (m)SR (%)Mean rotational RMS (°)SR (%)
Lowest 20%0.33071.28.0073.1
20–40%0.62971.512.8873.4
40–60%1.00868.519.4566.6
60–80%1.70166.830.6363.9
Highest 20%8.12645.068.5546.0

The results show that larger accumulated pose errors are generally associated with lower navigation success. SR decreases from 71.2% to 45.0% across the ATE extremes and from 73.1% to 46.0% across rotational-error extremes, while lower-error groups maintain relatively high success rates (updated Table 6).

Together, the multi-horizon and full-trajectory results indicate that PoseVLN maintains useful geometric states across successive updates (mean ATE of 0.33–1.70 m in the lowest four error quintiles), and that lower accumulated pose errors are associated with higher navigation success. However, errors remain large in a subset of episodes (mean ATE of 8.13 m in the highest-error quintile, where SR drops to 45.0%), and drift-free propagation is not guaranteed. We therefore appreciate the reviewer’s suggestion that explicit long-term geometric constraints could further improve consistency, and we consider this a valuable direction for future work.

Q2: Maintaining geometric state when predicted motion differs from actual execution.

We thank the reviewer for raising this practical issue. Collisions in Habitat and motion errors in real-world scenarios can reduce navigation success by causing discrepancies between predicted actions and actual execution. PoseVLN maintains its geometric state through learned ΔPose predictions, incorporating newly acquired RGB observations and visual memory into subsequent predictions. This recursive process allows navigation decisions to adapt to newly observed conditions despite execution discrepancies.

To evaluate this capability under collisions in Habitat, we compare learned poses with poses derived from executed actions using the same pose-token interface. We group navigation tasks by collision counts in the action-derived pose rollouts and compare success rates on the same tasks:

CollisionsTasksAction-derived poses SR (%)PoseVLN SR (%)
094669.568.2
1–257563.366.6
3–516960.463.3
6–106844.150.0
>108112.323.5
All1,83963.264.6

PoseVLN achieves higher SR in every collision-containing group than using poses derived from executed actions, with the largest gain occurring above ten collisions (12.3% to 23.5%). Notably, on tasks with 0–5 collisions in Action-DR rollouts, PoseVLN maintains relatively high navigation success, achieving 68.2%, 66.6%, and 63.3% SR for the 0, 1–2, and 3–5 collision groups, respectively. These results, together with the scene-level geometric error analysis, support the robustness of learned geometric state maintenance under execution discrepancies (updated Tables 7 and 8).

Together, these results support the value of recursively maintained geometric states in mitigating the navigation impact of collision-related execution discrepancies. However, the substantial decline in success rate under frequent collisions highlights the importance of the reviewer's concern. We appreciate this observation and consider improving geometric state maintenance under severe execution discrepancies an important direction for future work.

Q3: Motivation and geometric interpretation of the Directional Progress Loss.

We thank the reviewer for highlighting the need for a clearer motivation and geometric interpretation. Directional Progress Loss addresses systematic translation underestimation during local pose regression. While the loss operates in the adopted inverse-extrinsic parameterization and does not guarantee physical camera progress, we provide additional translation-magnitude diagnostics and physical camera-displacement evaluations to examine whether its benefits extend beyond this parameterization.

Translation underestimation. During exploratory training on R2R-train and RxR-train, we observed increasingly underestimated local translations without the progress loss. We compared checkpoints trained with and without this objective:

ModelMean predicted / GT magnitude (m)Relative deficitPredictions <0.1 m
Without progress, intermediate0.3410 / 0.650147.55%18.40%
Without progress, final0.1356 / 0.652779.23%64.32%
With progress, final0.5778 / 0.63258.65%4.61%

Without the loss, the relative magnitude deficit reaches 79.23%, with 64.32% of predictions below 0.1 m. With the loss, these values decrease to 8.65% and 4.61%, respectively, demonstrating its effectiveness in mitigating systematic translation underestimation (Appendix G, Table 9).

Physical displacement accuracy. To examine whether these improvements extend beyond the specific coordinate parameterization, we convert predicted poses into physical camera-center displacements. On a supplementary subset of 10 episodes, the With-Progress checkpoint reduces mean displacement error from 0.352 to 0.174 m and RMSE from 0.415 to 0.201 m, compared with the Without-Progress checkpoint at the same training step. Mean directional error also decreases from 16.64° to 11.01° over 132 valid segments. These independently generated rollout results provide evidence that the benefits extend to physical displacement estimation, rather than merely improving translation magnitude in the adopted parameterization (Appendix G, Table 9).

Together with the navigation ablation, these findings support the effectiveness of Directional Progress Loss in mitigating translation underestimation and improving geometric motion prediction and navigation. Following the reviewer's suggestion, we have clarified its geometric interpretation as a motion-conditioned translation-underestimation regularizer, which constrains the predicted translation's projection onto the target direction in the inverse-extrinsic parameterization, rather than guaranteeing sufficient physical camera progress. The improved physical displacement accuracy further supports its benefits beyond parameterization-specific optimization (Appendix G, Table 9).

Reviewer_9RBuOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

PoseVLN extends StreamVLN with pose-aware memory and geometric state evolution for single-RGB-stream VLN. The initial observation defines an episode-relative coordinate frame. Historical and current poses are encoded as tokens, and the model jointly predicts four navigation actions and a local pose transition over the same horizon. After execution, the predicted transformation is composed with the current state and stored with the new observation. Training uses simulator pose supervision and a directional progress loss; evaluation recursively propagates predicted poses from the identity without step-wise ground-truth pose or external odometry. The method reports 64.6% SR, 59.4% SPL, and 4.01 m NE on R2R-CE Val-Unseen, and 62.0% SR, 53.5% SPL, and 70.5 nDTW on RxR-CE English Val-Unseen. SR exceeds the strongest compared baseline by 2.6 and 2.9 percentage points, respectively. The paper also includes training ablations, pose diagnostics, and a preliminary physical-robot demonstration.

Critical strengths
  • A clear geometric representation delivers consistent navigation results under single-RGB inference. Using a shared episode frame to connect memory with local future motion is a compact and sensible design. The paper explicitly distinguishes ground-truth pose supervision during training from recursive pose prediction during evaluation. SR, SPL, and NE are strong on both benchmarks, with improved nDTW on RxR-CE, supporting the practical value of the approach for navigation.

  • The matched-data closed-loop training comparison strengthens the empirical evidence. In Table 3, the action-only baseline reaches 56.99% SR and 53.38% SPL after DAgger, while the full model reaches 64.63% and 59.35%. The paper states that both use the same aggregated closed-loop rollout data, so the final improvement cannot simply be attributed to additional DAgger training. Disclosure of action-only cold-start degradation, accumulated drift, and training-only pose re-anchoring during expert intervention also helps clarify the experimental scope.

Critical weaknesses
  • The central mechanism is insufficiently isolated, and an action-derived pose baseline is missing. The cumulative loss ablation in Table 3 and inference-time masking in Table 5 do not provide a full factorial comparison of geometric supervision and state representation. Removing all pose inputs reduces SR from 64.63% to 62.89%, but this model still received geometric supervision, and masking changes its training input distribution. Consequently, the contribution of auxiliary geometric learning versus recursive state inputs remains unresolved. Forward, left, right, and stop actions already encode substantial motion information. A matched baseline that composes nominal poses from executed actions would help establish what learned ΔPose contributes beyond simple motion integration. Collisions and execution errors can create meaningful differences, but these should be quantified.

  • Pose comparisons and generalization evidence limit the stronger mechanistic claims. Table 4 independently aligns translation scale for each pose pair, so its position RPE does not validate the metric scale of the propagated state. PoseVLN also incorporates intermediate RGB observations and updated memory, whereas VGGT receives endpoint images only. Table 5 usefully adds raw position errors, but correlations between pose error and eventual navigation success do not establish causality. Main results cover two Val-Unseen splits without multiple training seeds or confidence intervals, and physical deployment is illustrated by a single qualitative example. These limitations do not negate the navigation improvements, but constrain claims about the mechanism, stability, and deployment readiness.

Questions for authors

The following questions seek clarification and validation of the central conclusions; the answers could change my rating.

  1. What are the separate contributions of geometric supervision and recursive state? With matched initialization, training steps, and DAgger data, could you retrain action-only, geometric-auxiliary-supervision-without-pose-inputs, and full recursive-pose models? Please report both benchmarks rather than relying only on inference-time masking.

  2. Does learned ΔPose outperform action integration? Could you compare against poses composed from nominal executed-action transforms, reporting navigation, unaligned translation error, and collision cases? Is the pose head explicitly conditioned on the final generated and executed actions, and how are inconsistencies handled?

  3. How different are training and inference pose contexts? Appendices B and D describe recorded simulator poses being used to construct training context, while rollout policies propagate predictions and are re-anchored during expert recovery. Please specify the proportions of ground-truth and predicted pose context during optimization, the intervention frequency, and the effects of predicted-context training or removing re-anchoring.

  4. How stable and efficient are the results? Please provide multiple-seed results or paired episode-level confidence intervals for key navigation comparisons and batch-one end-to-end decision latency. Could you report pose comparisons under matched observation conditions without per-pair scale alignment, and additional physical deployment episodes?

7 comments

We sincerely thank the reviewer for the insightful and constructive feedback, particularly for recognizing the geometric design, navigation performance, and matched-data DAgger comparison. The reviewer's suggestions have helped us improve the clarity and experimental validation of our work. Below, we provide detailed responses and additional evidence addressing the concerns regarding geometric supervision, recursive state inputs, pose estimation, statistical reliability, and real-world evaluation.

W1 & Q1: Separate contributions of geometric supervision and recursive state inputs.

We thank the reviewer for suggesting a clearer isolation of geometric supervision and recursive state inputs. To address the limitations of the original cumulative loss ablation (Table 3) and inference-time masking (Table 5), we additionally trained an Auxiliary-only variant that retains the same geometric objectives, weights, and schedules as full PoseVLN but removes current and historical pose inputs during both training and inference. Under matched training settings and the same aggregated DAgger data, the R2R-CE Val-Unseen results are:

Training configurationSR (%) ↑SPL (%) ↑NE (m) ↓
Action-only57.053.44.86
Action + auxiliary geometric supervision57.554.24.92
Full PoseVLN64.659.44.01

Auxiliary geometric supervision. Without pose inputs, geometric supervision improves SR from 57.0% to 57.5% and SPL from 53.4% to 54.2%, demonstrating a modest independent contribution beyond action supervision.

Recursive state inputs. Full PoseVLN further improves SR from 57.5% to 64.6%, SPL from 54.2% to 59.4%, and NE from 4.92 to 4.01 m. The improvement is also observed on RxR-CE English Val-Unseen, where the original action-only baseline achieves 52.9% SR, Auxiliary-only reaches 56.0%, and full PoseVLN achieves 62.0% (Appendix L, Table 16).

Together, these results distinguish the contribution of auxiliary geometric supervision from the additional benefits of the pose-conditioned recursive state framework. Geometric supervision alone provides a modest improvement, while integrating pose inputs with recursive geometric state evolution yields substantially larger navigation gains on both benchmarks. We have updated Table 3 and the corresponding discussion accordingly.

W1 & Q2: Learned ΔPose versus action integration, collision cases, and execution inconsistencies.

We thank the reviewer for suggesting a matched action-derived pose baseline. Since Forward, Left, Right, and Stop actions already encode substantial motion information, we introduce Action-based Dead Reckoning (Action-DR), which composes nominal poses from executed actions, to evaluate what learned ΔPose contributes beyond simple motion integration. We provide action-derived poses as text prompts for the Action-only and Auxiliary-only models without trained pose encoders, and through the existing pose-token interface for full PoseVLN.

Training configurationSR (%) ↑SPL (%) ↑NE (m) ↓
Action-only47.142.85.83
Action + auxiliary geometric supervision52.448.95.45
Full PoseVLN63.258.84.16

Compared with their respective no-pose baselines, Action-DR text prompts reduce SR by 9.9 and 5.1 points, suggesting that auxiliary geometric supervision reduces sensitivity to pose information. More importantly, using the same trained PoseVLN checkpoint and pose-token interface, learned ΔPose improves SR from 63.2% to 64.6%, demonstrating its additional navigation benefit beyond nominal action integration.

Collision-stratified navigation. To quantify execution discrepancies, we group the same 1,839 R2R-CE Val-Unseen tasks by collision counts in Action-DR rollouts and compare navigation success:

Action-DR collisionsTasksAction-DR SR (%)PoseVLN SR (%)
094669.568.2
1–257563.366.6
3–516960.463.3
6–106844.150.0
>108112.323.5
All1,83963.264.6

PoseVLN achieves higher SR in every collision-containing group, with the largest gain above ten collisions (12.3% to 23.5%). Under limited collisions (0–5), it maintains relatively high SR of 63.3%–68.2%, supporting the value of learned geometric state feedback under execution discrepancies (updated Table 7).

Unaligned translation errors. We further compare episode-level translational ATE without scale alignment in two representative scenes. Neither method consistently achieves lower overall ATE (4.662 vs. 4.840 m and 3.344 vs. 2.754 m, Action-DR versus learned ΔPose). However, for tasks with at least three Action-DR collisions, learned ΔPose achieves lower ATE in both scenes (16.061 vs. 18.249 m and 6.377 vs. 10.464 m, learned ΔPose versus Action-DR). These results provide further evidence of the benefits of learned pose feedback for geometric estimation under larger execution discrepancies (updated Table 8).

Action conditioning and execution inconsistencies. The pose head is not explicitly conditioned on the final generated or executed actions; it predicts ΔPose from the current pose-token hidden state, while action generation uses the shared multimodal representation. After execution, the predicted ΔPose is composed with the current state, and newly acquired RGB observations and visual memory inform subsequent predictions. This enables subsequent decisions to incorporate new visual evidence without explicitly correcting accumulated pose errors.

Together, these results support the additional navigation value of learned ΔPose beyond nominal action integration, particularly under collision-related execution discrepancies. However, the substantial decline in navigation success under frequent collisions highlights the importance of the reviewer's concern. We consider explicit action conditioning and execution-aware geometric state correction valuable directions for further improvement.

Q3: Training and inference pose contexts and expert re-anchoring.

We thank the reviewer for requesting this clarification. We clarify the differences between training and inference pose contexts, the proportions of GT and predicted pose inputs during optimization, expert intervention frequency, and the role of re-anchoring.

Training versus inference pose contexts. PoseVLN uses different pose contexts across three stages. During optimization, pose contexts are constructed from recorded simulator trajectories. During policy-controlled DAgger rollouts, the model recursively propagates predicted poses, with re-anchoring applied during expert recovery. During evaluation, the model starts from the identity pose and recursively maintains its predicted geometric state without GT pose inputs, expert intervention, or re-anchoring.

GT and predicted pose proportions during optimization. All valid current and historical pose inputs are 100% GT-derived and 0% predicted, with local ΔPose targets obtained from the same recorded trajectories. Thus, DAgger collects trajectories influenced by predicted-state errors, while optimization uses recorded camera poses as geometric context and supervision.

Expert intervention frequency and re-anchoring. Expert intervention is triggered when reference-path deviation exceeds 1.0 m, the normalized accumulated action-error score exceeds 0.5, the action-error count exceeds 8, or an invalid Stop occurs. Policy control resumes when deviation falls below 0.5 m. The audited intervention statistics are:

Collection runEpisodes with ≥1 interventionTakeovers per episode
R2R96.68%1.09
RxR88.50%1.01
Combined91.32%1.04

Expert recovery occurs in most collection episodes, averaging approximately one takeover per episode. At expert takeover and policy return, the internal pose is re-anchored to the recorded camera pose in the episode-relative frame, and unexecuted actions and stale caches are cleared to maintain consistency after expert-controlled motion.

Effects of predicted-context training and removing re-anchoring. Due to computational resource constraints, we have not separately quantified these two settings. Training with predicted pose contexts could help the model adapt to accumulated prediction errors, while removing re-anchoring during expert recovery could introduce geometric state inconsistencies after expert-controlled motion. These effects require further experimental evaluation.

Together, these clarifications distinguish the training and inference protocols and explain the role of expert recovery during DAgger collection. Importantly, the reported navigation gains are achieved with fully predicted pose contexts and no expert re-anchoring during evaluation, supporting the effectiveness of recursive geometric state evolution under the current training protocol. We have clarified the collection and re-anchoring procedures in Appendix B, training-sample construction in Appendix D, and pose-context proportions and intervention statistics in Appendix I (Tables 11 and 12).

W2 & Q4: Statistical reliability, efficiency, pose comparison, and real-world evaluation.

We thank the reviewer for suggesting these additional evaluations. We conducted paired statistical and latency analyses, added a fixed-scale pose comparison, clarified the different observation protocols and interpretation of pose errors, and extended the physical deployment experiments.

Statistical reliability. We compare StreamVLN and PoseVLN on the same 1,839 R2R-CE Val-Unseen episodes using 20,000 paired bootstrap resamples, preserving episode correspondence. We also measure batch-one end-to-end decision latency:

MethodSR (%) [95% CI]Mean median latency (ms)Mean P95 latency (ms)
StreamVLN56.2 [53.94, 58.40]546.97767.83
PoseVLN64.6 [62.43, 66.78]560.12781.09
Difference+8.4 pp [5.87, 11.09]+13.15 (+2.40%)+13.26 (+1.73%)

The paired SR improvement over the reproduced StreamVLN is 8.4 points [5.87, 11.09], with the confidence interval entirely above zero. On RxR-CE English Val-Unseen, PoseVLN achieves 62.0% SR [60.45, 63.58] across 3,669 episodes. These results provide statistical support for the R2R-CE improvement over StreamVLN and quantify episode-level uncertainty on RxR-CE for the evaluated checkpoints (Appendix J).

Batch-one efficiency. PoseVLN increases mean median decision latency by only 13.15 ms (2.40%) and mean P95 latency by 13.26 ms (1.73%) compared with StreamVLN. These results demonstrate that the navigation improvements are achieved with modest additional end-to-end decision latency (Appendix J).

Pose comparison and observation protocols. We agree that the per-pair scale alignment in the original Table 4 evaluates translation agreement up to scale rather than metric-scale accuracy. We also note that the two methods use different observation protocols: VGGT takes the endpoint image pair (frame i and frame i+H), whereas PoseVLN is causal and predicts ΔPose before the future frame is observed. These protocols cannot be matched: PoseVLN cannot take an endpoint image pair, and VGGT cannot recover relative pose from a single image. We therefore evaluate each method under its native protocol and supplement the original comparison with a fixed-scale evaluation.

Specifically, we calibrate VGGT using 103,300 moving H=4 pairs from 10,819 R2R training trajectories, obtaining a single global scale of s∗=1.777s^*=1.777 by minimizing translation error. This scale is fixed across H∈{4,8,12,16}H\in\{4,8,12,16\} and both R2R-CE and RxR-CE evaluations, without per-pair alignment. Calibration and evaluation use separate scenes, and evaluation GT is not used for scale fitting. PoseVLN retains its original predicted metric scale.

HorizonPoseVLN position error (m) ↓VGGT position error (m) ↓
H=40.25680.2547
H=80.50490.6772
H=120.78631.0563
H=161.10921.4348

The methods achieve comparable errors at H=4, while PoseVLN achieves lower errors at H=8, 12, and 16. These results provide additional evidence of longer-horizon geometric accuracy without per-pair scale alignment, under the respective observation protocols (Appendix K, Table 15).

Reviewer_mXsyOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes PoseVLN, a VLN agent that utilizes camera pose as a condition for action prediction. PoseVLN organizes visual memory and the current observation in an episode-relative coordinate frame, and jointly predicts the next K navigation actions together with a local ∆Pose over the same horizon. The predicted ∆Pose is composed with the current trajectory-level pose to obtain the future pose state, which is recursively propagated throughout navigation, forming a continuous geometric state evolution from past memory to future motion. Empirically, training with action prediction, pose regression, and a directional-progress objective with DAgger, PoseVLN achieves strong empirical performance on multiple navigation benchmarks. Besides, it also demonstrates generalization to real environments by deploying on a Unitree Go2 for real-world navigation.

Critical strengths
  1. The proposed approach of including camera pose information for decision making is intuitive and effective.
  2. Predicting a local pose transition over the same four-step horizon as the action chunk makes the geometric target directly relevant to the policy’s upcoming behavior. This is a sensible way to connect geometric learning with navigation, rather than adding a spatial prediction task with an unrelated temporal scope.
  3. Strong empirical results show improvements across benchmarks and evaluation metrics.
  4. The physical-robot demonstration provides preliminary evidence of deployability.
Critical weaknesses
  1. Missing test set results for R2R-CE.
  2. Table 5 suggests that PoseVLN benefits from using pose as input. But the improvement of PoseVLN seems to come mainly from learning with pose as an auxiliary task. Experiments are needed to demonstrate that maintaining and feeding back pose states is valuable.
  3. Recursive pose composition can accumulate translation and rotation errors, while the method lacks an explicit observation-based correction mechanism.
  4. The real-world evidence is only one example and might not be sufficient to demonstrate its performance when generalizing to real-world environments.
Questions for authors

N/A

5 comments

We sincerely thank the reviewer for the thoughtful assessment and constructive suggestions, as well as for recognizing the pose-conditioned design, horizon-matched ΔPose prediction, and empirical and real-robot results. The feedback has helped us strengthen our analysis and clarify the scope of our findings. Below, we address each concern with additional experimental evidence and acknowledge the remaining limitations.

W1: Missing R2R-CE Test Results

We appreciate the reviewer's suggestion and recognize the value of reporting held-out test results. However, official R2R-CE test evaluation is currently unavailable because the evaluation service was retired in January 2026, and the ground-truth test goals remain unreleased. The organizers therefore recommend Val-Unseen for future comparisons (see the official VLN-CE challenge page).

Following this recommendation, we evaluate PoseVLN on R2R-CE and RxR-CE Val-Unseen, achieving 64.6% and 62.0% SR, respectively, exceeding the strongest compared baselines by 2.6 and 2.9 points. These evaluations provide consistent evidence across two benchmarks under the currently recommended protocol. We have clarified this in Appendix N.

W2: Contributions of Pose Supervision and Recursive Pose Feedback

We thank the reviewer for raising this important distinction. The original inference-time masking results in Table 5 show that the jointly trained PoseVLN benefits from its predicted pose inputs: masking both current and historical poses reduces SR from 64.6% to 62.9%. To further distinguish this contribution from auxiliary geometric supervision, we conducted additional experiments with matched training settings.

Specifically, we trained an Auxiliary-only variant that retains the same geometric objectives, weights, and schedules as full PoseVLN but removes current and historical pose inputs during both training and inference. Using the same aggregated DAgger data, the R2R-CE Val-Unseen results are:

Training configurationSR (%) ↑SPL (%) ↑NE (m) ↓
Action-only57.053.44.86
Action + auxiliary geometric supervision57.554.24.92
Full PoseVLN64.659.44.01

Auxiliary geometric supervision. Without pose inputs, auxiliary supervision improves SR from 57.0% to 57.5%, showing a modest independent benefit. To further examine whether geometric supervision helps the model interpret pose-related information, we provide poses derived from executed actions as text prompts at inference:

Training configurationSR (%) ↑SPL (%) ↑NE (m) ↓
Action-only + pose text47.142.85.83
Auxiliary-only + pose text52.448.95.45

Compared with their respective no-pose baselines, SR decreases by 9.9 and 5.1 points. The smaller degradation with auxiliary supervision suggests that geometric learning reduces the model's sensitivity to pose-related information.

Maintaining and feeding back pose states. Beyond auxiliary supervision, full PoseVLN jointly learns pose-conditioned navigation and recursive geometric state evolution, improving SR from 57.5% to 64.6%. This advantage also holds on RxR-CE English Val-Unseen, where SR increases from 52.9% for Action-only to 56.0% for Auxiliary-only and 62.0% for full PoseVLN (Appendix L, Table 16).

Together, these results demonstrate that PoseVLN's navigation improvements extend beyond learning pose as an auxiliary task. Auxiliary geometric supervision provides a modest independent benefit and improves adaptability to pose inputs, while jointly learning pose-conditioned navigation and recursively maintaining and feeding back geometric states yields substantially larger improvements. We have updated Tables 3 and 5 and their accompanying discussion accordingly.

W3: Accumulated Pose Errors and State Correction

We thank the reviewer for highlighting this important issue. Accumulated translation and rotation errors, particularly under execution discrepancies, can affect navigation performance. Although PoseVLN lacks explicit pose correction, it maintains a degree of robustness to such errors by recursively maintaining predicted geometric states and incorporating newly acquired RGB observations and visual memory into subsequent predictions. To evaluate this robustness and its limitations, we extended our geometric evaluation to complete trajectories and conducted additional collision-stratified comparisons.

Accumulated errors and long-term geometric consistency**.** Building on the original multi-horizon results in Tables 4 and 5 (H∈{4,8,12,16}H\in\{4,8,12,16\}), we evaluate complete navigation trajectories using translational ATE and rotational RMS error, independently dividing episodes into five error quantiles:

Error quantileMean ATE (m)SR (%)Mean rotational RMS (°)SR (%)
Lowest 20%0.33071.28.0073.1
20–40%0.62971.512.8873.4
40–60%1.00868.519.4566.6
60–80%1.70166.830.6363.9
Highest 20%8.12645.068.5546.0

Larger accumulated errors are generally associated with lower navigation success, with SR decreasing from 71.2% to 45.0% across the ATE extremes and from 73.1% to 46.0% across rotational-error extremes. Meanwhile, the lower-error groups maintain relatively high success rates, indicating that PoseVLN can sustain effective navigation when accumulated geometric errors remain moderate (updated Table 6). Additional collision-stratified experiments further support this robustness, with PoseVLN achieving higher navigation success than action-derived pose integration in every collision-containing group (updated Tables 7 and 8).

Together, these results indicate that PoseVLN maintains useful geometric states and supports effective navigation under moderate accumulated errors and execution discrepancies. However, the substantial decline in navigation success under severe pose errors and frequent collisions highlights the importance of the reviewer's concern. We appreciate this suggestion and consider explicit observation-based pose correction a valuable direction for further improving long-horizon geometric consistency and navigation robustness.

W4: Real-World Generalization

We appreciate the reviewer's request for broader real-world evaluation. To extend the original single demonstration, we conducted additional experiments on Unitree Go2 across two real-world environments, including an outdoor scene and an indoor scene. Each environment includes three navigation tasks, with three trials per task under slightly varied starting positions. PoseVLN successfully completes 11 of 18 trials (61.1%), achieving at least two successes in four of six task settings, covering doorway traversal, straight motion, and turning behaviors. Detailed instructions, outcomes, and trajectory frames are provided in Appendix M.

To further examine robustness to visual appearance changes, we conducted controlled perturbation experiments. Under brightness reduction (×0.5) and Gaussian blur (radius 2), PoseVLN achieves 62.5% and 58.4% SR, respectively, compared with 64.6% on clean inputs. Both results remain above the clean-input action-only baseline of 57.0%, providing complementary evidence of robustness to the tested visual changes (Appendix H, Table 10).

Together, these results extend the original single demonstration to quantitative real-world evaluations across multiple environments and navigation tasks, providing additional evidence of PoseVLN's real-world navigation feasibility.

Dear Reviewer mXsy,

Thank you for your time and thoughtful feedback on our paper. As the discussion period draws to a close, we would appreciate knowing whether our rebuttal has adequately addressed your concerns. If you have any remaining questions, we would be happy to provide further clarification. We sincerely appreciate your time and consideration.

Best regards, Authors

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Rebuttal Discussion and Consensus

Dear Area Chair and Reviewers,

We thank you for your time and constructive feedback. The discussion helped us clarify how pose inputs contribute to closed-loop navigation, quantify PoseVLN’s robustness to accumulated pose errors, and broaden the real-world evaluation.

Following the rebuttal, Reviewers 9RBu and mXsy both maintained their ratings of 3 (Weak Acceptance). Reviewer 9RBu wrote, “The authors have addressed most of my concerns through additional experiments and clarifications,” and Reviewer mXsy stated, “The experiments address my concerns.” We also addressed Reviewer uYTF’s initial request to isolate the contribution of pose inputs through controlled experiments on both benchmarks. In the follow-up discussion, we clarified how pose errors accumulate during inference and how the use of ground-truth poses during training differs from deployment.

The revised paper addresses three main concerns raised in the reviews.

  • Contribution of pose inputs. To address the reviewers’ shared question about the contribution of pose inputs, we trained an auxiliary-only variant using the same geometric supervision and training data as full PoseVLN, with pose inputs removed during both training and inference. Adding pose inputs increased navigation success from 57.5% to 64.6% on R2R-CE Val-Unseen and from 56.0% to 62.0% on RxR-CE English Val-Unseen. These gains show that recursively maintaining a geometric state through pose updates benefits navigation beyond auxiliary geometric supervision alone.
  • Robustness to accumulated pose error and collisions. PoseVLN achieved 68.5% and 66.8% navigation success in episode groups with mean absolute trajectory errors of 1.01 m and 1.70 m, respectively. Success fell to 45.0% in the highest-error group (mean 8.13 m). In a comparison using the same checkpoint, learned pose updates also yielded higher navigation success than action-based dead reckoning in every group where the baseline encountered collisions. Together, these results indicate that the geometric state PoseVLN recursively maintains through pose updates remains useful under moderate drift and improves robustness to collision-related execution errors.
  • Real-world evaluation. We expanded the original demonstration to six indoor and outdoor navigation tasks. PoseVLN completed 11 of 18 trials and also succeeded on longer routes in a follow-up evaluation.

We appreciate the reviewers’ suggestions and thank the Area Chair for considering the additional evidence and post-rebuttal feedback.

Sincerely,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.