acceptodds
Under review as a conference paper at ICLR 2027

Canvas-VLN: From Future Plans to Execution-Grounded Memories for Vision-Language Navigation

Abstract

Vision-and-Language Navigation (VLN) requires an agent to make long-horizon, closed-loop decisions in continuous environments by following natural-language instructions. Recent methods translate the outputs of vision-language models into executable local plans by predicting action chunks, image-space goal points, or pixel trajectories. However, existing trajectory-based approaches typically treat trajectories as disposable action outputs, while representing historical context using sequences of raw observations or textual action chunks. Consequently, future planning and past execution experience reside in different representation spaces. We present Canvas-VLN, a unified canvas–trajectory navigation framework in which pixel trajectories serve three roles: supervision targets, online plans, and navigation memory. The model predicts a future pixel trajectory on the current navigation canvas, and a local controller executes a short prefix of the predicted trajectory. The waypoints actually traversed by the agent are then reprojected onto historical anchor images to construct a Gated Execution-Trajectory Memory. This memory adaptively commits local trajectory segments according to visibility, waypoint budget, and turning events, while explicit turn states connect consecutive historical anchors. It thereby maintains a long-horizon, execution-grounded navigation context under a fixed visual-memory budget. Canvas-VLN thus establishes a predict–execute–write-back loop, in which a predicted future trajectory becomes historical trajectory evidence after execution and subsequently informs the next planning step. Trained solely on the official R2R-CE and RxR-CE navigation demonstrations, without additional training data or DAgger augmentation, Canvas-VLN achieves a Success Rate (SR) of 53.83% on R2R-CE Val-Unseen and 60.56% on RxR-CE Val-Unseen, establishing a new state of the art on RxR-CE among methods trained exclusively on task-specific navigation data.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Vision-and-Language Navigation, Pixel-Trajectory Prediction, Execution-Trajectory Memory

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_E47COpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Canvas-VLN, a trajectory-centered framework for vision-and-language navigation in continuous environments. The main idea is to use pixel trajectories as a unified representation for training supervision, future action planning, and historical execution memory. The proposed Gated Execution-Trajectory Memory (GETM) projects executed positions onto historical visual anchors and adaptively updates the memory based on visibility, trajectory-point budgets, and turning events. This creates a closed-loop predict–execute–write-back mechanism. Experiments on the Val-Unseen splits show competitive performance on R2R-CE and strong results on RxR-CE. The paper also includes component ablations, trajectory-length analysis, and qualitative real-world robot demonstrations.

Critical strengths
  1. An intuitive and practical execution-grounded memory design.

The proposed GETM provides a clear way to connect predicted future trajectories with actual execution history. Recording executed motion rather than simply storing predicted plans or raw observations is a meaningful design choice for long-horizon navigation. The bounded memory mechanism is also practical for controlling the size of historical context.

  1. Strong benchmark performance without additional navigation training data.

Canvas-VLN achieves competitive results on R2R-CE and particularly strong performance on RxR-CE, reaching 60.56% SR and 49.77% SPL. The component ablations further support the usefulness of execution-trajectory memory, especially for longer navigation tasks. These results suggest that the proposed memory representation can improve instruction-following navigation without relying on additional training data.

Critical weaknesses
  1. Dependence on accurate geometric information and insufficient robustness analysis.

GETM relies on accurate depth and camera poses to reproject executed positions onto historical observations. While depth information is standard in VLN-CE, the source of pose estimates and their reliability during real-world deployment are not sufficiently explained. It is unclear how localization errors, depth noise, or accumulated pose drift affect the consistency of the execution memory. Furthermore, GETM uses several fixed thresholds, including a 75° turning threshold, an 8-point trajectory budget, and 0.5 m sampling intervals, without sufficient sensitivity analysis. These design choices raise concerns about the robustness and generalizability of the proposed memory mechanism.

  1. Incomplete evaluation of generalization and real-world robustness.

The quantitative evaluation is limited to the R2R-CE and RxR-CE Val-Unseen splits, with no results on the official Test-Challenge splits. More importantly, the real-world experiments are primarily qualitative, showing selected successful navigation trajectories without reporting success rates, collision statistics, or path efficiency across repeated trials. Given the paper's emphasis on execution-grounded memory and practical navigation, stronger quantitative real-world evidence is needed to substantiate its robustness and deployment claims.

Questions for authors

Q1. Pose estimation and geometric robustness.

How are camera poses obtained during real-world deployment? Does GETM rely on ground-truth poses, external localization, or onboard odometry/SLAM? How sensitive is navigation performance to pose drift and depth estimation errors?

Q2. Generalization to Test-Challenge splits.

Can the authors provide results on the official R2R-CE and RxR-CE Test-Challenge splits? This would help establish whether the improvements observed on Val-Unseen generalize to the held-out evaluation sets.

2 comments
AuthorsOpenReview ↗

Response to Reviewer E47C

Thank you for the concrete questions. We can now specify the real-world pose source and provide controlled geometric-perturbation results and repeated-trial deployment measurements.

Q1. Pose estimation and geometric robustness.

Thank you for highlighting the need to specify the pose source and quantify sensitivity to geometric errors. We have added the deployment's onboard wheel–IMU odometry details in Appendix E. The chassis reports left/right wheel angular velocities, from which diff_drive_controller computes wheel odometry. An EKF in robot_localization fuses planar linear velocity from wheel odometry with yaw angular velocity from a Hipnuc IMU. The resulting local pose is published as /odom and used by the bridge to update GETM. The configuration uses neither IMU absolute orientation nor acceleration, and uses no additional localization source. Camera poses are obtained by composing this estimated base pose with the calibrated camera extrinsics.

We completed a controlled evaluation with fixed model weights and controller settings on 300 stratified Val-Unseen episodes per dataset (Appendix G, Figure 4). Pose drift and depth corruption are applied separately, with three noise seeds per perturbed condition and one clean run. Navigation metrics use simulator ground truth. Each entry below gives SR / SPL (%), averaged across noise seeds for perturbed conditions.

ConditionR2R-CERxR-CE
Clean54.67 / 48.2758.00 / 48.70
Pose-L51.89 / 46.1655.00 / 45.44
Pose-M41.33 / 36.4537.89 / 31.80
Pose-H29.22 / 26.0224.67 / 21.19
Depth-L54.11 / 48.5461.44 / 51.03
Depth-M53.89 / 47.9860.00 / 49.24
Depth-H56.22 / 49.2055.78 / 44.97

Pose-L/M/H use translation drift of 1/3/5%, yaw drift of 0.25/0.75/1.50 degrees per traveled meter, and turn-proportional drift of 0.5/1/2% of cumulative absolute rotation. Depth-L/M/H use relative Gaussian standard deviation and pixel-dropout probability of 1/5/10%.

The results identify accumulated pose drift as the stronger sensitivity: low drift reduces SR by 2.78/3.00 pp on R2R/RxR, whereas high drift reduces SR by 25.44/33.33 pp. Waypoint rejection rises from 1.50%/1.16% to 10.11%/15.34%. Depth corruption has a smaller effect; at the highest level, RxR SR/SPL decrease by 2.22/3.73 pp. These measurements evaluate the combined memory-projection and geometric-execution pipeline; waypoint rejection does not separately identify memory misalignment as the cause.

Real-world quantitative evaluation.

Thank you for requesting repeated-trial evidence beyond the qualitative examples. We now report 31/48 successes (64.6% SR) in Table 2, based on 16 routes across six environments—four office buildings and two outdoor environments—with three trials per route. Every successful trial ended with a model-issued STOP within 3 m of the goal and the target visible in the final camera image. One hardware-interrupted trial is counted as a failure; network-interrupted runs are excluded from the 48 evaluated trials. Figure 3 and Appendix E (Figures 6 and 7) provide successful and failed examples. Collision-rate and path-efficiency measurements are unavailable for this experiment.

Fixed memory parameters.

Thank you for raising the question of sensitivity to GETM's fixed parameters. To test this directly, we are running a 300-episode parameter-sensitivity evaluation without retraining, keeping model weights fixed. We will report the settings and results alongside the ViKeyMem-like comparison in a follow-up. The completed Recent Raw comparison (Table 4) evaluates gated-anchor organization under the same eight-image budget, while the 2×2 ablation (Table 3) separates trajectory overlays and turn transitions.

Q2. Official test-set evaluation.

Thank you for requesting this additional test of generalization. We investigated the official evaluation routes for both benchmarks. For R2R-CE, the organizers announced the retirement of the official test server in January 2026 and recommend Val-Unseen for future method comparisons (official announcement). For RxR-CE, Yao et al. (ICML 2026; their Appendix D.1, Table 5) also reported that the official test evaluation server was unavailable (SC²-WM). We could not obtain a working submission route for RxR-Habitat Test-Challenge, including after contacting the organizers. We therefore use Val-Unseen for quantitative benchmark comparisons on both datasets.

Area_Chair_gpkxOpenReview ↗

Response Reminder

Dear Reviewer,

As we approach the end of the discussion period, we invite you to review the authors’ response to your comments. Please update us on whether your concerns have been sufficiently addressed and whether you wish to pose further questions to the authors.

We greatly appreciate your contribution.

Reviewer_WuujOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Canvas-VLN uses pixel trajectories for training supervision, local planning, and navigation memory. Its Gated Execution-Trajectory Memory (GETM) projects the actually executed motion onto the historical image and maintains a finite-length history based on visibility, trajectory point budget, and turning events.

Critical strengths
  1. GETM provides a concrete method for recording actual motion execution, with clear online update rules and a fixed memory budget.
  2. RxR-CE shows strong results without using additional navigation training data. In ablation with consistent anchor selection and memory budget, the complete method improves SR by 1.61 percentage points and SPL by 1.49 percentage points compared to Raw Anchor.
Critical weaknesses
  1. The methodological novelty is limited. Goal2Pixel already employs visibility-based keyframe selection and historical trajectory overlays, while Traj-VLN already uses pixel-trajectory prediction. Canvas-VLN’s remaining novelty appears to lie mainly in online trajectory segmentation and explicit turn-state organization. The paper lacks comparisons with the closest prior memory design to establish the value of these differences.
  2. The ablations do not isolate the contribution of trajectory overlays. Raw Anchor removes both trajectory overlays and transition text. Retaining overlays without transition text yields 56.34% RxR SR, below Raw Anchor’s 58.95%. Without a variant that preserves transition text while removing overlays, the results cannot distinguish the effects of trajectory overlays, turn descriptions, and their interaction. Please report the success rate on real-world deployment and compare with SOTA.
Questions for authors

See Weakness above

9 comments
AuthorsOpenReview ↗

Response to Reviewer Wuuj

Thank you for the constructive review. Below, we address the memory-cue ablation, comparisons of memory organization, and quantitative real-world evaluation.

1. Separating overlays and transition text (Table 3).

Thank you for identifying the missing control needed to separate overlays from transition text. We have independently trained the requested transition-only variant under the same protocol, completing the 2×2 comparison shown below and in Table 3.

Trajectory overlayTransition textR2R SR / SPL / nDTWRxR SR / SPL / nDTW
NoNo52.09 / 45.99 / 61.1858.95 / 48.28 / 66.01
YesNo54.00 / 47.99 / 62.1256.34 / 46.60 / 64.92
NoYes54.75 / 47.36 / 61.0559.30 / 48.76 / 66.65
YesYes53.83 / 48.03 / 62.1660.56 / 49.77 / 67.32

On RxR-CE, adding overlays with transition text present improves SR/SPL/nDTW by 1.26/1.01/0.67 pp. Without transition text, overlays reduce these metrics by 2.61/1.68/1.09 pp relative to Raw Anchor. Thus, the effect of overlays depends on the accompanying cross-anchor orientation information. Full GETM performs best on all three RxR metrics, supporting the joint use of execution traces and turn transitions. This pattern is consistent with transitions helping the policy relate trajectory segments expressed in different anchor frames.

On R2R-CE, transition-only has the highest SR; adding overlays reduces SR by 0.92 pp while improving SPL/nDTW by 0.67/1.11 pp. The benefit of combining both cues is therefore clearer on RxR-CE, rather than uniform across datasets and metrics.

2. Contribution and comparison with prior memory designs.

Thank you for asking us to establish the contribution beyond closely related memory and planning methods. Traj-VLN advances pixel-trajectory prediction, and Goal2Pixel introduces goal-pixel grounding with visibility-aware, trajectory-annotated keyframe memory. We recognize both contributions and discuss their methodological overlap with Canvas-VLN in Related Work (Section 2).

Our contribution is a common trajectory language across offline supervision, online prediction, and execution memory, together with GETM's organization of that memory for long-horizon navigation. The trajectory serves successively as an expert supervision target, a predicted local plan, and a record of realized motion. GETM maintains this execution record as bounded local segments connected by yaw transitions, closing the predict–execute–write-back loop (Sections 3.2–3.4, Figure 2).

Using only standard navigation demonstrations, Canvas-VLN achieves 60.6% SR on RxR-CE Val-Unseen, improving on Traj-VLN and Goal2Pixel by 8.0 and 12.5 percentage points, respectively. Its 53.8% R2R-CE SR remains comparable to task-data-only baselines (Table 1). This establishes state-of-the-art RxR-CE performance among the task-data-only methods compared in Table 1 and demonstrates the practical value of a simple execution-to-memory trajectory interface for long-horizon VLN.

The new controls examine this contribution within the same navigation framework. Raw Anchor outperforms the independently trained Recent Raw baseline by 10.11/9.53 pp SR/SPL on R2R-CE and 7.33/6.48 pp on RxR-CE (Table 4), while the complete 2×2 comparison identifies the joint contribution of overlays and turn transitions (Table 3).

For the closest prior memory design, we are training a ViKeyMem-like (v2-based) variant under Canvas-VLN's matched protocol. It follows Goal2Pixel v2, the version available at our submission deadline; the later v3 (October 27, 2026) revised the memory-construction description. We will report the implementation choices and results of this paper-based adaptation in a follow-up.

3. Real-world success rate and comparisons.

Thank you for requesting real-world success rates and comparisons with prior methods. Table 2 now reports a Canvas-VLN success rate of 64.6% (31/48 evaluated trials) over 16 routes across six environments, with three trials per route. The environments comprise four office buildings and two outdoor environments. Success requires a model-issued STOP within 3 m of the goal and target visibility in the final image. The evaluation counts one hardware interruption as a failure and excludes network-interrupted runs. For context, the table also includes published NaVid, NaVILA, and EvoNav results across different robots and protocols. Figure 3 and Appendix E (Figures 6 and 7) show qualitative examples. A same-platform baseline comparison has not been conducted.

Thanks for the authors’ rebuttal, which I will check later. For real world results, please submit the raw real world demos of your successful rollouts. Also, please report the inference time during the real world execution.

Thank you for requesting the real-world demonstrations and inference timing. We are preparing anonymized first-person videos of successful rollouts, showing the horizontal RGB view, trajectory canvas, predicted trajectories, and GETM. For a small subset of these routes, we will also provide supplementary third-person demonstrations, clearly identified as additional recordings. We will upload these videos as supplementary material and report inference latency during real-world execution. We will also provide the results and analyses of the two ongoing follow-up experiments: the ViKeyMem-like (v2-based) comparison and the parameter-sensitivity evaluation.

Reviewer_f8zNOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces Canvas-VLN, a framework for Vision-Language Navigation (VLN). By employing a "Gated Execution-Trajectory Memory" (GETM) mechanism—which integrates future trajectory prediction with a memory of past executions—the framework achieves excellent performance on the RxR-CE benchmark.

Critical strengths
  1. The paper addresses a critical challenge in long-horizon navigation by explicitly incorporating previously executed trajectories into the agent's historical memory.

  2. GETM offers a simple, interpretable memory mechanism that features adaptive trajectory segmentation and operates with a bounded (finite) memory capacity.

  3. The method delivers strong performance on RxR-CE without requiring additional task-specific training data. Furthermore, the authors provide component ablation studies and demonstrate the framework's application on a real-world robot.

Critical weaknesses
  1. The proposed framework appears to synthesize existing concepts from Goal2Pixel (which utilizes historical observations annotated with trajectories) and Traj-VLN (which focuses on trajectory prediction in pixel space). While the specific integration of these ideas and the design of the memory update mechanism represent the core contributions, the overall methodological advancement is relatively incremental.

  2. Although Canvas-VLN performs well on RxR-CE, it is less competitive on R2R-CE. It achieves an SR (Success Rate) of 53.83% and an SPL (Success weighted by Path Length) of 48.03%; for comparison, Traj-VLN achieves 58.4%/52.2%, and Goal2Pixel achieves 53.9%/52.7%. Moreover, the performance gain provided by GETM over a "raw anchor memory" is quite modest (an improvement of only 1.61% in SR on RxR-CE), which raises questions regarding the significance of its contribution.

  3. This method utilizes both a horizontal-view RGB observation and a trajectory-view perspective (tilted 35° downward), combined with depth-based geometric processing. Compared to baseline methods that rely on a single viewpoint, this approach likely incorporates additional information. Although the authors conducted ablation studies regarding the horizontal view, a more rigorous comparison—under identical observational conditions—is required to determine whether the performance gains are truly attributable to GETM itself.

  4. While the paper includes demonstrations using a real robot, the results are primarily qualitative. The absence of quantitative evaluations (such as navigation success rates across diverse routes and environments) makes it difficult to assess the method's robustness and generalization capabilities in the real world.

  5. The paper lacks detailed ablation studies on specific design choices within GETM, such as the gating strategy, memory capacity, and trajectory sampling. Furthermore, although "long-range navigation" is a primary motivation for this research, the performance improvement on the longest routes (Q4) is relatively minor and statistically uncertain.

Questions for authors

see weaknesses

4 comments
AuthorsOpenReview ↗

Response to Reviewer f8zN

Thank you for the detailed assessment.

1. Contribution relative to Goal2Pixel and Traj-VLN.

Thank you for asking us to distinguish our contribution from these related methods. Canvas-VLN unifies offline supervision, online prediction, and execution memory through a common trajectory language. Its GETM organizes realized motion into bounded memory for long-horizon navigation. We recognize Traj-VLN's contribution to pixel-trajectory planning and Goal2Pixel's contributions to goal-pixel grounding and visibility-aware trajectory memory, and explicitly discuss the overlap in Related Work (Section 2).

First, Canvas-VLN unifies offline supervision, online prediction, and historical memory through a common trajectory language. Expert trajectories serve as training targets; predicted trajectories guide local execution; and the motion actually executed is written back as memory for subsequent prediction. This closes the predict–execute–write-back loop between future plans and realized experience (Sections 3.2–3.4).

Second, GETM makes this loop practical for long-horizon navigation under a bounded visual-memory budget. It converts executed paths into local memory trajectories, adaptively commits segments through visibility, point-budget, and turning gates, and connects successive anchors with explicit yaw transitions (Section 3.2, Figure 2).

Trained only on standard R2R/RxR-CE navigation demonstrations, Canvas-VLN achieves 60.6% SR on RxR-CE Val-Unseen, exceeding Traj-VLN (52.6%) and Goal2Pixel (48.1%) by 8.0 and 12.5 percentage points, respectively (Table 1). This is state-of-the-art RxR-CE performance among the task-data-only methods compared, demonstrating an intuitive, simple, and effective approach to long-horizon VLN through execution-grounded trajectory memory.

Table 4 supports gated anchor retention: Raw Anchor exceeds independently trained Recent Raw by 10.11/9.53 pp SR/SPL on R2R-CE and 7.33/6.48 pp on RxR-CE.

We are also training a ViKeyMem-like (v2-based) variant under the same Canvas-VLN protocol, based on the version available at our submission deadline. The later v3 (October 27, 2026) revised the memory description. We will report this paper-based adaptation's implementation choices and results in a follow-up.

2. R2R performance.

Thank you for raising the R2R comparison. GETM was designed from the outset for long instructions and long navigation paths, where tracking execution progress under bounded memory is crucial. RxR-CE directly tests this setting. Canvas-VLN also remains competitive in R2R-CE SR: 53.8%, compared with 53.9% for Goal2Pixel, 52.8% for JanusVLN, and 55.5% for FutureNav-8B* (Table 1). It is within 1.7 percentage points of the strongest task-data-only SR in this comparison. Thus, its strong RxR-CE performance is accompanied by comparable success on R2R-CE.

3. Observational fairness and GETM attribution.

Thank you for highlighting the observation mismatch in cross-paper comparisons. Our GETM attribution uses Full GETM vs. Raw Anchor (Tables 3 and 4): both use the horizontal and 35° views, with the same depth/pose interface, controller, training data, and evaluation episodes. GETM improves SR/SPL by 1.74/2.04 pp on R2R-CE and 1.61/1.49 pp on RxR-CE under these controls. These gains therefore support the contribution of the memory cues under the same observation setup. The horizontal-view ablation measures view utility separately.

4. Quantitative real-world evaluation.

Thank you for pointing out the need to quantify real-world performance. We now report 31 successes in 48 evaluated trials (64.6% SR) in Table 2, covering 16 routes across six environments—four office buildings and two outdoor environments—with three trials per route. Success requires a model-issued STOP within 3 m of the goal and target visibility in the final camera image. One hardware-interrupted trial is counted as a failure; runs interrupted by network outages are excluded from the 48-trial evaluation.

5. Design choices and the longest routes.

Thank you for raising parameter sensitivity and performance on the longest routes. Table 4 tests gated anchor retention against a recent-frame window. We are also running a 300-episode parameter-sensitivity evaluation with fixed weights and will report the results alongside ViKeyMem-like in a follow-up.

For navigation horizon, Table 5 and Appendix D report the paired length-stratified results and confidence intervals. The gains are not monotonic with route length. On RxR-CE, the strongest improvement occurs in Q3 (+4.26 pp SR, +3.38 pp SPL); Q4 has positive estimates (+1.34/+1.21 pp) whose confidence intervals include zero. Across the longer half, Q3–Q4, the gains are +2.80/+2.29 pp. These results support a benefit on longer routes, without establishing a statistically conclusive improvement in the longest quartile alone.

Area_Chair_gpkxOpenReview ↗

Response Reminder

Dear Reviewer,

As we approach the end of the discussion period, we invite you to review the authors’ response to your comments. Please update us on whether your concerns have been sufficiently addressed and whether you wish to pose further questions to the authors.

We greatly appreciate your contribution.

The authors’ rebuttal addressed some of my concerns, and after considering the other reviewers’ comments, I maintain my original score.

Thank you for acknowledging that our rebuttal addressed some of your concerns.

However, your response does not identify which substantive technical concerns remain unresolved or explain why the original assessment is still warranted. Given that our rebuttal directly addressed several of the issues raised, we would appreciate clarification of the specific remaining deficiencies and their significance to the paper's stated contributions.

We respectfully ask that the final assessment be based on concrete, independently substantiated technical concerns rather than unspecified concerns carried over from other reviews.

Comments on the paper

4 comments
AuthorsOpenReview ↗

Summary of Manuscript Revisions

We have uploaded a revised manuscript with the following additions:

  • A complete 2×2 ablation of trajectory overlays and cross-anchor transition text (Section 4.3, Table 3).
  • A Recent Raw comparison evaluating gated historical anchors against a recent-frame window (Section 4.3, Table 4).
  • Quantitative real-world evaluation over 48 trials across 16 routes (Section 4.2, Table 2), with additional successful and failed examples in Appendix E (Figures 6 and 7).
  • Controlled pose-drift and depth-corruption evaluations (Appendix G, Figure 4).

We also clarified the scope of the shared controller (Section 4.2), the real-world evaluation protocol (Section 4.2), and onboard pose estimation (Appendix E), and corrected minor wording and typographical issues throughout.

AuthorsOpenReview ↗

Additional Results: GETM Parameter Sensitivity

We have completed additional parameter-sensitivity experiments on 300 RxR-CE Val-Unseen episodes, as summarized below. All variants use the same episodes and model weights, without retraining. The default uses a 75° turning threshold, a history cap of eight images, eight points per anchor, and 0.50 m sampling spacing. Each variant changes one parameter while keeping the others fixed.

SettingSR (%)SPL (%)
Default58.0048.70
Turning threshold: 60°60.6750.75
Turning threshold: 90°58.3348.96
History cap: 4 images58.0046.35
History cap: 6 images60.6750.23
Point budget: 4 per anchor57.3347.67
Point budget: 12 per anchor58.6748.77
Sampling spacing: 1.00 m58.0043.59

Both alternative turning thresholds match or improve upon the default. Changing the point budget to four or twelve keeps SR within 0.67 percentage points and SPL within 1.03 points of the default. These results show that performance does not depend on the exact default values of these two parameters within the ranges shown.

Reducing the history cap from eight to six images improves SR/SPL by 2.67/1.54 points. With four images, SR remains unchanged, but SPL falls to 46.35%. Increasing history capacity therefore does not consistently improve navigation on this subset.

Increasing the sampling spacing from 0.50 m to 1.00 m preserves SR at 58.00% but reduces SPL from 48.70% to 43.59%. This comparison supports retaining the default 0.50 m spacing for path efficiency.

These additional sensitivity results and the table above have not yet been included in the revised manuscript.

AuthorsOpenReview ↗

Supplementary Real-world Rollout Videos and Inference Latency

The attached archive contains first-person videos of 10 successful rollouts, each presented in a 2×2 layout showing the GETM memory, horizontal RGB observation, trajectory canvas, and model prediction. We also provide third-person recordings corresponding to three of these same rollouts, allowing the robot’s physical execution to be viewed alongside its observations and predictions. All videos are played at 4× speed, and identifying details have been pixelated to preserve anonymity.

In response to Reviewer Wuuj’s question about inference time, the average inference latency during real-world execution is 896.2 ms.

AuthorsOpenReview ↗

Additional Results: Comparison with ViKeyMem-like

We have completed training and evaluation of the ViKeyMem-like (v2-based) variant, adapting the memory design described in Goal2Pixel v2 to Canvas-VLN. The variant is independently trained under the same protocol as GETM, with only the memory mechanism changed. Training data, dual-view observations, history-image cap, controller, and complete Val-Unseen evaluation sets are unchanged. The results on R2R-CE and RxR-CE Val-Unseen are summarized below.

The adapted memory selects keyframes through visibility and forward-view checks over the executed trajectory prefix, overlays visible trajectory points as blue dots, and retains at most eight historical images. Cross-anchor transition text is omitted.

Memory strategyR2R SRR2R SPLR2R nDTWRxR SRRxR SPLRxR nDTW
ViKeyMem-like (v2-based)52.3246.8261.1757.7447.6065.32
GETM53.8348.0362.1660.5649.7767.32

GETM improves SR/SPL/nDTW over this adaptation by 1.51/1.21/0.99 percentage points on R2R-CE and 2.82/2.17/2.00 points on RxR-CE.

GETM achieves higher success, path efficiency, and trajectory fidelity on both benchmarks under the same training and evaluation protocol. These results support the value of GETM's overall memory design beyond the shared use of trajectory-annotated historical observations.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.