- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes PoseVLN, which builds upon prior works such as StreamVLN to address the issue of inadequate long-term spatial state maintenance in VLN by explicitly modeling camera poses, integrating pose information into visual memory, and jointly predicting navigation actions and local pose changes. While the paper introduces a geometrically grounded modeling framework with intuitive motivations and reports competitive navigation results, it suffers from significant shortcomings in both experimental setup and result analysis.
Critical strengths
The paper treats camera poses as unified geometric variables and uses them as references for historical memory and spatial states, which is a well-motivated design choice. Compared to relying solely on historical image features or action history, pose-aware memory provides clear spatial references for different observations. By integrating both into a unified navigation framework, the overall design exhibits strong interpretability. Furthermore, experimental results across multiple benchmarks demonstrate the effectiveness of the proposed method.
Critical weaknesses
I believe the authors have not adequately clarified the fundamental differences between their approach and existing geometry-aware navigation or pose-conditioned visual models, primarily in the following aspects:
- The current experimental results fail to sufficiently isolate and justify the individual contributions of geometric supervision and pose inputs.
- Although ground-truth camera poses provided by the Habitat simulator are utilized to construct local pose supervision during training, I doubt whether the learned geometric motion patterns can generalize to real-world scenarios across different visual domains.
- The primary training objective focuses predominantly on local pose regression, and such local supervision is insufficient to guarantee long-term geometric consistency.
Questions for authors
In addition to the points raised in the weaknesses, I would like the authors to address the following:
- In real-world scenarios, pose errors accumulate over time. Please analyze the relationship between accumulated pose errors and navigation success rate.
- How does the model maintain its geometric state when predicted actions are inconsistent with actual execution outcomes, such as collisions in Habitat or motion errors in real-world?
- I believe the Directional Progress Loss warrants a more detailed explanation. This loss constrains local translation regression under a specific coordinate parameterization and cannot be interpreted as ensuring that the camera has made sufficient progress along the true motion direction in physical space. The authors should provide further clarification and evidence to verify that the benefits of this loss are not merely an optimization bias stemming from a specific parameterization.
9 comments
W2: Generalization of learned geometric motion across visual domains.
We appreciate the reviewer's concern. Although Habitat ground-truth camera poses are used to construct local pose supervision during training, PoseVLN predicts ΔPose recursively from RGB observations without ground-truth poses or external odometry at inference.
To evaluate whether the learned geometric motion patterns generalize across different visual domains, we conducted two controlled visual perturbation experiments: reducing image brightness to 0.5× and applying Gaussian blur with radius 2. SR decreases from 64.6% to 62.5% under reduced brightness and to 58.4% under blur, corresponding to drops of 2.1 and 6.2 points, respectively. Both results remain above the clean-input action-only baseline of 57.0%, indicating tolerance to the tested visual changes, particularly brightness variation (Appendix H, Table 10).
Real-world evaluation. We further evaluated PoseVLN on Unitree Go2 in an outdoor and an indoor environment. Each environment includes three navigation tasks, with three trials per task under slightly varied starting positions. PoseVLN successfully completes 11 of 18 trials (61.1%), with at least two successful trials in four of six task settings, covering doorway traversal, straight motion, and turning behaviors. These results provide quantitative evidence that PoseVLN can perform language-guided navigation in the evaluated real-world scenarios (Appendix M).
Together, the visual perturbation experiments and real-world trials support PoseVLN's robustness to the tested appearance changes and its feasibility for real-world navigation.
W3 & Q1: Long-term geometric consistency and the relationship between accumulated pose errors and navigation success.
We thank the reviewer for raising this issue. Although ΔPose regression uses local supervision, training on trajectory-sampled windows in a shared episode-relative frame helps the model learn geometric relationships across steps. These relationships support recursive geometric state evolution beyond individual pose predictions. The original multi-horizon results in Tables 4 and 5 already provide evidence that PoseVLN learns useful geometric states across successive updates (), with lower pose errors consistently observed in successful episodes.
To further evaluate long-term geometric consistency over complete navigation trajectories and its relationship with navigation success, we measure translational ATE and rotational RMS error and independently divide episodes into five error quantiles:
| Error quantile | Mean ATE (m) | SR (%) | Mean rotational RMS (°) | SR (%) |
|---|---|---|---|---|
| Lowest 20% | 0.330 | 71.2 | 8.00 | 73.1 |
| 20–40% | 0.629 | 71.5 | 12.88 | 73.4 |
| 40–60% | 1.008 | 68.5 | 19.45 | 66.6 |
| 60–80% | 1.701 | 66.8 | 30.63 | 63.9 |
| Highest 20% | 8.126 | 45.0 | 68.55 | 46.0 |
The results show that larger accumulated pose errors are generally associated with lower navigation success. SR decreases from 71.2% to 45.0% across the ATE extremes and from 73.1% to 46.0% across rotational-error extremes, while lower-error groups maintain relatively high success rates (updated Table 6).
Together, the multi-horizon and full-trajectory results indicate that PoseVLN maintains useful geometric states across successive updates (mean ATE of 0.33–1.70 m in the lowest four error quintiles), and that lower accumulated pose errors are associated with higher navigation success. However, errors remain large in a subset of episodes (mean ATE of 8.13 m in the highest-error quintile, where SR drops to 45.0%), and drift-free propagation is not guaranteed. We therefore appreciate the reviewer’s suggestion that explicit long-term geometric constraints could further improve consistency, and we consider this a valuable direction for future work.
Q2: Maintaining geometric state when predicted motion differs from actual execution.
We thank the reviewer for raising this practical issue. Collisions in Habitat and motion errors in real-world scenarios can reduce navigation success by causing discrepancies between predicted actions and actual execution. PoseVLN maintains its geometric state through learned ΔPose predictions, incorporating newly acquired RGB observations and visual memory into subsequent predictions. This recursive process allows navigation decisions to adapt to newly observed conditions despite execution discrepancies.
To evaluate this capability under collisions in Habitat, we compare learned poses with poses derived from executed actions using the same pose-token interface. We group navigation tasks by collision counts in the action-derived pose rollouts and compare success rates on the same tasks:
| Collisions | Tasks | Action-derived poses SR (%) | PoseVLN SR (%) |
|---|---|---|---|
| 0 | 946 | 69.5 | 68.2 |
| 1–2 | 575 | 63.3 | 66.6 |
| 3–5 | 169 | 60.4 | 63.3 |
| 6–10 | 68 | 44.1 | 50.0 |
| >10 | 81 | 12.3 | 23.5 |
| All | 1,839 | 63.2 | 64.6 |
PoseVLN achieves higher SR in every collision-containing group than using poses derived from executed actions, with the largest gain occurring above ten collisions (12.3% to 23.5%). Notably, on tasks with 0–5 collisions in Action-DR rollouts, PoseVLN maintains relatively high navigation success, achieving 68.2%, 66.6%, and 63.3% SR for the 0, 1–2, and 3–5 collision groups, respectively. These results, together with the scene-level geometric error analysis, support the robustness of learned geometric state maintenance under execution discrepancies (updated Tables 7 and 8).
Together, these results support the value of recursively maintained geometric states in mitigating the navigation impact of collision-related execution discrepancies. However, the substantial decline in success rate under frequent collisions highlights the importance of the reviewer's concern. We appreciate this observation and consider improving geometric state maintenance under severe execution discrepancies an important direction for future work.
Q3: Motivation and geometric interpretation of the Directional Progress Loss.
We thank the reviewer for highlighting the need for a clearer motivation and geometric interpretation. Directional Progress Loss addresses systematic translation underestimation during local pose regression. While the loss operates in the adopted inverse-extrinsic parameterization and does not guarantee physical camera progress, we provide additional translation-magnitude diagnostics and physical camera-displacement evaluations to examine whether its benefits extend beyond this parameterization.
Translation underestimation. During exploratory training on R2R-train and RxR-train, we observed increasingly underestimated local translations without the progress loss. We compared checkpoints trained with and without this objective:
| Model | Mean predicted / GT magnitude (m) | Relative deficit | Predictions <0.1 m |
|---|---|---|---|
| Without progress, intermediate | 0.3410 / 0.6501 | 47.55% | 18.40% |
| Without progress, final | 0.1356 / 0.6527 | 79.23% | 64.32% |
| With progress, final | 0.5778 / 0.6325 | 8.65% | 4.61% |
Without the loss, the relative magnitude deficit reaches 79.23%, with 64.32% of predictions below 0.1 m. With the loss, these values decrease to 8.65% and 4.61%, respectively, demonstrating its effectiveness in mitigating systematic translation underestimation (Appendix G, Table 9).
Physical displacement accuracy. To examine whether these improvements extend beyond the specific coordinate parameterization, we convert predicted poses into physical camera-center displacements. On a supplementary subset of 10 episodes, the With-Progress checkpoint reduces mean displacement error from 0.352 to 0.174 m and RMSE from 0.415 to 0.201 m, compared with the Without-Progress checkpoint at the same training step. Mean directional error also decreases from 16.64° to 11.01° over 132 valid segments. These independently generated rollout results provide evidence that the benefits extend to physical displacement estimation, rather than merely improving translation magnitude in the adopted parameterization (Appendix G, Table 9).
Together with the navigation ablation, these findings support the effectiveness of Directional Progress Loss in mitigating translation underestimation and improving geometric motion prediction and navigation. Following the reviewer's suggestion, we have clarified its geometric interpretation as a motion-conditioned translation-underestimation regularizer, which constrains the predicted translation's projection onto the target direction in the inverse-extrinsic parameterization, rather than guaranteeing sufficient physical camera progress. The improved physical displacement accuracy further supports its benefits beyond parameterization-specific optimization (Appendix G, Table 9).
We sincerely thank the reviewer for the thoughtful review, constructive feedback, and positive comments on the motivation and interpretability of PoseVLN. The reviewer's suggestions have helped us clarify our contributions and strengthen the experimental analysis. In the following responses, we further explain the role of pose-conditioned navigation and provide additional evidence on visual robustness, accumulated geometric errors, execution discrepancies, and the Directional Progress Loss.
We thank the reviewer for highlighting the need to distinguish PoseVLN from existing geometry-aware navigation and pose-conditioned visual models. Existing approaches primarily focus on learning spatial representations [1-4]. In contrast, PoseVLN uses camera pose as an evolving geometric state, predicting ΔPose over the same horizon as the navigation actions and recursively feeding updated poses into memory and subsequent decisions.
To isolate the contributions of geometric supervision and pose inputs, we additionally trained an Auxiliary-only variant that retains the same geometric objectives, weights, and schedules as full PoseVLN but removes current and historical pose inputs during both training and inference. Under matched training settings and the same aggregated DAgger data, we obtain the following R2R-CE Val-Unseen results:
Geometric supervision. Without pose inputs, auxiliary geometric supervision improves SR from 57.0% to 57.5% and SPL from 53.4% to 54.2%, showing a modest independent benefit. To further examine whether geometric supervision helps the model interpret pose-related information, we provide action-derived poses as text prompts at inference:
Compared with their respective no-pose baselines, SR decreases by 9.9 and 5.1 points. The smaller degradation with auxiliary supervision suggests that geometric learning reduces the model's sensitivity to action-derived pose prompts.
Pose inputs and recursive state. Beyond auxiliary supervision, full PoseVLN jointly learns pose-conditioned navigation and recursive geometric state updates, improving SR from 57.5% to 64.6%. This advantage also holds on RxR-CE, where SR increases from 52.9% for Action-only to 56.0% for Auxiliary-only and 62.0% for full PoseVLN (Appendix L, Table 16).
Together, these results support the contribution of jointly learned geometric state evolution beyond auxiliary supervision or directly providing pose information. Geometric supervision provides a modest independent benefit and improves adaptability to pose inputs, while integrating pose-conditioned memory with recursive state updates yields substantially larger navigation gains. We have updated Tables 3 and 5 and their accompanying discussion accordingly.
[1] JanusVLN: Decoupling Semantics and Spatiality with Dual Implicit Memory for Vision-Language Navigation. ICLR, 2026.
[2] GA-VLN: Geometry-Aware BEV Representation for Efficient Vision-Language Navigation. CVPR, 2026.
[3] Uncertainty-Aware Gaussian Map for Vision-Language Navigation. ICLR, 2026.
[4] Bridging the 2D-3D Gap: A Hierarchical Semantic-Geometric Map for Vision Language Navigation. CVPR, 2026.