- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes Canvas-VLN, a trajectory-centered framework for vision-and-language navigation in continuous environments. The main idea is to use pixel trajectories as a unified representation for training supervision, future action planning, and historical execution memory. The proposed Gated Execution-Trajectory Memory (GETM) projects executed positions onto historical visual anchors and adaptively updates the memory based on visibility, trajectory-point budgets, and turning events. This creates a closed-loop predict–execute–write-back mechanism. Experiments on the Val-Unseen splits show competitive performance on R2R-CE and strong results on RxR-CE. The paper also includes component ablations, trajectory-length analysis, and qualitative real-world robot demonstrations.
Critical strengths
- An intuitive and practical execution-grounded memory design.
The proposed GETM provides a clear way to connect predicted future trajectories with actual execution history. Recording executed motion rather than simply storing predicted plans or raw observations is a meaningful design choice for long-horizon navigation. The bounded memory mechanism is also practical for controlling the size of historical context.
- Strong benchmark performance without additional navigation training data.
Canvas-VLN achieves competitive results on R2R-CE and particularly strong performance on RxR-CE, reaching 60.56% SR and 49.77% SPL. The component ablations further support the usefulness of execution-trajectory memory, especially for longer navigation tasks. These results suggest that the proposed memory representation can improve instruction-following navigation without relying on additional training data.
Critical weaknesses
- Dependence on accurate geometric information and insufficient robustness analysis.
GETM relies on accurate depth and camera poses to reproject executed positions onto historical observations. While depth information is standard in VLN-CE, the source of pose estimates and their reliability during real-world deployment are not sufficiently explained. It is unclear how localization errors, depth noise, or accumulated pose drift affect the consistency of the execution memory. Furthermore, GETM uses several fixed thresholds, including a 75° turning threshold, an 8-point trajectory budget, and 0.5 m sampling intervals, without sufficient sensitivity analysis. These design choices raise concerns about the robustness and generalizability of the proposed memory mechanism.
- Incomplete evaluation of generalization and real-world robustness.
The quantitative evaluation is limited to the R2R-CE and RxR-CE Val-Unseen splits, with no results on the official Test-Challenge splits. More importantly, the real-world experiments are primarily qualitative, showing selected successful navigation trajectories without reporting success rates, collision statistics, or path efficiency across repeated trials. Given the paper's emphasis on execution-grounded memory and practical navigation, stronger quantitative real-world evidence is needed to substantiate its robustness and deployment claims.
Questions for authors
Q1. Pose estimation and geometric robustness.
How are camera poses obtained during real-world deployment? Does GETM rely on ground-truth poses, external localization, or onboard odometry/SLAM? How sensitive is navigation performance to pose drift and depth estimation errors?
Q2. Generalization to Test-Challenge splits.
Can the authors provide results on the official R2R-CE and RxR-CE Test-Challenge splits? This would help establish whether the improvements observed on Val-Unseen generalize to the held-out evaluation sets.
2 comments
Response Reminder
Dear Reviewer,
As we approach the end of the discussion period, we invite you to review the authors’ response to your comments. Please update us on whether your concerns have been sufficiently addressed and whether you wish to pose further questions to the authors.
We greatly appreciate your contribution.
Response to Reviewer E47C
Thank you for the concrete questions. We can now specify the real-world pose source and provide controlled geometric-perturbation results and repeated-trial deployment measurements.
Q1. Pose estimation and geometric robustness.
Thank you for highlighting the need to specify the pose source and quantify sensitivity to geometric errors. We have added the deployment's onboard wheel–IMU odometry details in Appendix E. The chassis reports left/right wheel angular velocities, from which
diff_drive_controllercomputes wheel odometry. An EKF inrobot_localizationfuses planar linear velocity from wheel odometry with yaw angular velocity from a Hipnuc IMU. The resulting local pose is published as/odomand used by the bridge to update GETM. The configuration uses neither IMU absolute orientation nor acceleration, and uses no additional localization source. Camera poses are obtained by composing this estimated base pose with the calibrated camera extrinsics.We completed a controlled evaluation with fixed model weights and controller settings on 300 stratified Val-Unseen episodes per dataset (Appendix G, Figure 4). Pose drift and depth corruption are applied separately, with three noise seeds per perturbed condition and one clean run. Navigation metrics use simulator ground truth. Each entry below gives SR / SPL (%), averaged across noise seeds for perturbed conditions.
Pose-L/M/H use translation drift of 1/3/5%, yaw drift of 0.25/0.75/1.50 degrees per traveled meter, and turn-proportional drift of 0.5/1/2% of cumulative absolute rotation. Depth-L/M/H use relative Gaussian standard deviation and pixel-dropout probability of 1/5/10%.
The results identify accumulated pose drift as the stronger sensitivity: low drift reduces SR by 2.78/3.00 pp on R2R/RxR, whereas high drift reduces SR by 25.44/33.33 pp. Waypoint rejection rises from 1.50%/1.16% to 10.11%/15.34%. Depth corruption has a smaller effect; at the highest level, RxR SR/SPL decrease by 2.22/3.73 pp. These measurements evaluate the combined memory-projection and geometric-execution pipeline; waypoint rejection does not separately identify memory misalignment as the cause.
Real-world quantitative evaluation.
Thank you for requesting repeated-trial evidence beyond the qualitative examples. We now report 31/48 successes (64.6% SR) in Table 2, based on 16 routes across six environments—four office buildings and two outdoor environments—with three trials per route. Every successful trial ended with a model-issued STOP within 3 m of the goal and the target visible in the final camera image. One hardware-interrupted trial is counted as a failure; network-interrupted runs are excluded from the 48 evaluated trials. Figure 3 and Appendix E (Figures 6 and 7) provide successful and failed examples. Collision-rate and path-efficiency measurements are unavailable for this experiment.
Fixed memory parameters.
Thank you for raising the question of sensitivity to GETM's fixed parameters. To test this directly, we are running a 300-episode parameter-sensitivity evaluation without retraining, keeping model weights fixed. We will report the settings and results alongside the ViKeyMem-like comparison in a follow-up. The completed Recent Raw comparison (Table 4) evaluates gated-anchor organization under the same eight-image budget, while the 2×2 ablation (Table 3) separates trajectory overlays and turn transitions.
Q2. Official test-set evaluation.
Thank you for requesting this additional test of generalization. We investigated the official evaluation routes for both benchmarks. For R2R-CE, the organizers announced the retirement of the official test server in January 2026 and recommend Val-Unseen for future method comparisons (official announcement). For RxR-CE, Yao et al. (ICML 2026; their Appendix D.1, Table 5) also reported that the official test evaluation server was unavailable (SC²-WM). We could not obtain a working submission route for RxR-Habitat Test-Challenge, including after contacting the organizers. We therefore use Val-Unseen for quantitative benchmark comparisons on both datasets.