acceptodds
Under review as a conference paper at ICLR 2027

NaviWAM: Coupled World–Action Prediction for Embodied Navigation

Abstract

Embodied navigation requires agents to translate task specifications and evolving visual observations into executable motion. Action labels supervise lowdimensional trajectories but do not directly constrain the richer visual changes along a route. We introduce NaviWAM, a coupled world–action prediction framework that complements trajectory supervision with patch-level next-frame prediction. A vision–language model integrates the task with long-horizon visual history to condition a shared dual-stream predictor. The predictor supports two separately invoked tasks: trajectory prediction (Policy) and next-frame feature prediction (NF). Shared parameters and cross-stream attention allow NF supervision to update components used for navigation. Labeled navigation sequences supervise both tasks, while additional video–instruction pairs provide NF targets without trajectory annotations. The same objective supports offline target-scene adaptation: target videos contribute NF supervision, while source trajectories retain Policy supervision through replay. At deployment, only the Policy task is sampled, without generating future visual observations. With separately trained vision-andlanguage and point-goal models, NaviWAM achieves competitive results across continuous navigation benchmarks. The combined Policy + NF training scheme increases RxR-CE success rate from 58.9% to 63.6%. On VLNVerse, target-scene NF adaptation improves fine- and coarse-grained success rates by 6.91 and 7.06 percentage points, respectively, without target-scene trajectory supervision.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Embodied Navigation, World Action Model

Reviews

Among:
Shape:
1234this paper, Top 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.50 over 2 reviews, above 99% of the others.
Reviewer_orfqOpenReview ↗
Rating
4(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents NaviWAM, a World–Action Prediction framework for embodied navigation. The method combines navigation trajectory prediction (Policy) and next-frame visual feature prediction (NF) through a shared dual-stream predictor, while using a Vision-Language Model (VLM) to encode language instructions and historical visual observations. Experiments across multiple Vision-and-Language Navigation (VLN) and PointGoal navigation benchmarks demonstrate competitive performance. The results further show that incorporating additional video data and performing target-scene adaptation can improve navigation success rates.

Critical strengths
  1. The paper addresses an important problem: leveraging videos without waypoint annotations to improve navigation performance and enable adaptation to new environments. This direction has clear practical value.
  2. The proposed framework combines navigation trajectory prediction with next-frame visual feature prediction, allowing the model to benefit from additional video data without requiring future visual feature prediction during inference.
  3. The experimental evaluation is comprehensive, with competitive results across multiple VLN and PointGoal navigation benchmarks. The authors also provide ablation studies examining factors such as video data volume, prediction objectives, and visual encoders.
Critical weaknesses
  1. Some important architectural choices are not sufficiently justified through ablation studies. Although the results demonstrate the benefits of NF supervision, it remains unclear whether the shared dual-stream predictor and patch-level feature prediction are necessary to achieve these improvements.
  2. The evaluation of target-scene adaptation could be more comprehensive. While the results show that incorporating target-scene videos improves navigation performance, comparisons with alternative adaptation methods are missing. In addition, the target-scene videos are generated in simulation, leaving the effectiveness of the approach with real-world videos unclear.
Questions for authors
  1. Could the authors further clarify the necessity of the shared dual-stream predictor? How would the proposed approach compare with a simpler architecture that combines a navigation policy with an auxiliary visual prediction task?
  2. Why are the improvements in coarse-grained navigation relatively limited? Table 3 shows that additional waypoint supervision leads to substantial improvements. Could the authors explain why NF supervision provides smaller gains in this setting?
2 comments
AuthorsOpenReview ↗

Shared Prediction, Video-Based Transfer, and Coarse Navigation

We thank the reviewer for the constructive feedback on predictor design, adaptation, and coarse-grained navigation. We added controlled architecture/feature and video-source ablations, plus real-scene and physical-robot evidence. The results support shared Policy–NF prediction, clarify which visual features matter, and distinguish NF from direct waypoint supervision.

  1. Shared prediction and feature granularity (Weakness 1; Q1) We compare our Shared dual-stream DiT with Separate Policy/NF DiTs. Both retain the same trainable VLM conditioning and dual-stream structure; only prediction-block sharing differs. Both use the same Stage-I/II schedule, Policy data, and NF video mixtures (including H+R+S+I with Isaac at 40% of the full Stage-II NF mixture). Patch/Pool indicates current/target feature granularity. SR/SPL (Shared vs. Separate):
  • Patch/Patch: RxR-CE 63.6/52.5 vs. 59.0/48.3; VLNVerse-Fine 67.0/60.4 vs. 63.4/56.3.
  • Patch/Pool: RxR-CE 63.8/52.6 vs. 61.2/50.4; Fine 68.1/59.8 vs. 64.2/57.1.
  • Pool/Pool: RxR-CE 61.8/51.2 vs. 58.6/47.5; Fine 64.6/55.7 vs. 62.8/54.6. Shared improves both metrics in all six comparisons. In default Patch/Patch, gains are +4.6/+4.2 (RxR-CE) and +3.6/+4.1 (Fine). As VLM conditioning is shared in both variants, these gains isolate prediction-block sharing: NF directly trains the predictor used for waypoints, rather than acting only through a shared upstream VLM. Feature ablations show that with current Patch features fixed, changing future targets from Patch to Pool gives 63.6/52.5→63.8/52.6 (RxR-CE) and 67.0/60.4→68.1/59.8 (Fine). With future Pool targets fixed, current Patch instead of Pool improves RxR-CE by +2.0/+1.4 and Fine by +3.5/+4.1. Thus preserving spatial detail in current observations matters more consistently than dense future targets. See §3.3, §4.4, Appendix §6.1.1, and Table 8.
  1. Video sources and real-scene transfer (Weakness 2) With Habitat Policy supervision fixed, we vary NF sources: H=Habitat, R=RoomTour3D, S=Sekai, I=Isaac. For H / H+R+S / H+I / H+R+S+I, SR/SPL is:
  • VLNVerse-Fine: 60.1/48.5; 61.8/48.9; 66.2/58.4; 67.0/60.4.
  • RxR-CE: 60.4/50.1; 62.6/51.3; 62.0/50.8; 63.6/52.5. Internet videos help without Isaac; Isaac yields larger Fine gains, and the combined mixture performs best. Because mixtures differ in sample volume, these are not equal-volume source comparisons. NF uses both video types without additional waypoint labels. We also use five real-world 3DGS scenes for NF training and five disjoint scenes for closed-loop evaluation (600 episodes). With other conditions matched, extra 3DGS NF increases SR/SPL from 58.33/42.68 to 64.33/45.67 (+6.00/+2.99) and reduces navigation error from 2.81 to 2.31 m. Preliminary physical-robot success improves from 3/10 to 6/10; physical test sites are excluded from NF training but overlap the held-out reconstructed evaluation scenes. These diagnostic results support transfer beyond simulated training, without claiming broad real-world generalization. Together with our matched two-stage Shared/Separate experiment, they support shared prediction and video-based adaptation. See §4.3–4.5, Tables 7–9, Appendix §6.3.1, and Fig. 8.
  1. Coarse-grained navigation (Q2) Under matched Habitat Policy replay, added NF videos improve VLNVerse-Coarse SR/SPL from 35.09/30.32 to 42.16/36.36 (+7.07/+6.04); the SR gain is comparable to Fine (+6.91). Thus NF helps Coarse despite lower absolute success. NF videos and source Policy trajectories mainly show demonstrated routes and teach task-conditioned visual dynamics, while coarse instructions leave route selection underspecified. Target-domain waypoint labels in the ‘’w/ action‘’ configuration directly teach this mapping and achieve 63.4/59.0 on Coarse. NF and action supervision therefore address complementary challenges. See §4.2–4.3 and Table 3.

Overall, these controls establish the benefits of shared Policy–NF prediction, current-frame spatial features, and video-based adaptation while explaining the complementary value of target-domain waypoint labels.

Thank you for the detailed response and the additional experiments. I appreciate the authors' efforts to address my concerns.

The new shared-versus-separate predictor ablation provides convincing evidence for the proposed architecture. The additional video-source analysis and real-world experiments also strengthen the evaluation, while the explanation regarding coarse-grained navigation is reasonable.

Overall, I believe the authors have adequately addressed my main concerns, and the new evidence has substantially strengthened the paper. I have therefore decided to increase my rating from 3 to 4.

Reviewer_ZfMPOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes NaviWAM, a framework for embodied navigation in continuous environments, covering both VLN and PointGoal. It has three main components:

  • Conditioning. Qwen3-VL-4B encodes the task text together with history frames selected by TDHS, which combines forgetting-curve recency sampling with farthest-point diversity selection. The last-layer hidden states serve as cross-attention context for a 256.6M-parameter dual-stream MMDiT (WAM) trained from scratch.
  • Two separately invoked tasks. The Policy call takes a noisy trajectory and a learnable “future-feature placeholder” and predicts the local trajectory. The NF call takes a noisy grid of next-frame DINOv2-S patch features and learnable action queries, and predicts the next-frame features. Both are trained with flow matching, share the backbone, and interact through joint attention.
  • Two-stage training. Stage I supervises Policy and NF jointly on about 2.3M R2R-CE/RxR-CE samples, plus about 1.38M action-free videos (RoomTour3D, Sekai, and Isaac synthetic data) used for NF only. Stage II adapts the model with target-scene videos (the Isaac scenes used by VLNVerse) through NF supervision, while replaying source-domain Policy data.

At deployment, only Policy is sampled. Main results: 63.6 SR on RxR-CE and 59.8 SR on R2R-CE; RGB-only PointGoal outperforms the depth-based NavDP; and target-scene NF adaptation improves fine/coarse SR on VLNVerse by 6.91/7.06 points.

Critical strengths
  1. The problem is practically relevant. The paper studies how to use videos without action labels to improve a navigation policy, and how to adapt a trained agent to a target scene using only walkthrough videos. Both match realistic deployment needs.

  2. The design is clean and the mechanism is clearly described. Policy and NF are invoked separately, and no future frame is generated at inference. The paper explains clearly which parameters are updated by which loss, and gives a precise definition of what "coupled" means in this context.

  3. Results are competitive and ablations are reasonably complete.

    • On RxR-CE, the monocular RGB model reaches 63.6 SR.
    • On PointGoal, a single RGB frame is enough to outperform methods that use depth, and the drop under dynamic obstacles is smaller.
    • The data-scaling study matches the number of Policy updates across configurations, and the appendix transparently reports FutureNav's full-data numbers for context.
Critical weaknesses
  1. The contribution of the NF objective itself should be isolated more clearly. The Policy baseline in Table 5 reports exactly the same numbers (58.9/47.9) as the 1:1 configuration in Table 9, and Table 9 states that every configuration is trained with both Policy and NF. As a result, it is hard to tell whether the improvement from 58.9 to 63.6 reported in the abstract comes from the NF objective or from the additional 1.38M videos. A Policy-only baseline trained with a matched number of optimizer updates would make the claim that "NF complements trajectory supervision" much easier to support.

  2. The relation between the Isaac data and the VLNVerse target scenes should be clarified. Section 4.1 shows that 0.78M of the additional Stage I samples come from Isaac, which is also the largest single source. The VLNVerse adaptation target is built on the same Isaac scenes. If the two overlap, this affects both the unseen comparison in Table 3 and the meaning of the 0% baseline in Stage II. In addition, the 0% result in Table 10 (60.12/48.54) matches the Stage I numbers.

Questions for authors
  1. Are the Policy row of Table 5 and the 1:1 row of Table 9 produced by the same model?
  2. Does the Isaac data used in Stage I include scenes from the VLNVerse val-unseen split?
  3. Is the 0% setting in Table 10 obtained by continued Stage II training, or is it the Stage I checkpoint used directly?
  4. RoomTour3D provides geometry-aware annotations, including recovered camera trajectories. Have the authors considered using these to derive pseudo-action labels for Policy supervision, and if so, how does that compare to using RoomTour3D for NF only?
3 comments
AuthorsOpenReview ↗

NF Contributions, Scene-Disjoint Adaptation, and Pseudo-Action Supervision

We thank the reviewer for identifying ambiguities in our NF comparison, scene accounting, and adaptation protocol, and for suggesting RoomTour3D pseudo-action supervision. We conducted additional controlled experiments and revised the paper accordingly.

Summary -- With matched training data and optimization budgets, NF independently improves RxR-CE SR/SPL by ''+1.5/+2.2'' points; additional videos yield further gains. We also verify scene-disjoint Isaac training, control Stage-II adaptation budgets, and evaluate pseudo-action supervision.

  1. Isolating NF (Weakness 1; Q1) We compare Policy-only and Policy+NF on the same Habitat trajectories, matching Policy-supervised update counts and total optimizer updates (~30K), following our largest paired-data training schedule. NF targets come only from subsequent observations within those trajectories, with no external videos. RxR-CE SR/SPL is:
  • Policy-only: 58.9/47.9;
  • Policy+NF (Habitat only): 60.4/50.1;
  • Policy+NF (full video mixture):63.6/52.5; Thus, NF itself contributes ''+1.5/+2.2'', and external videos add ''+3.2/+2.4''. Original Table 9 inadvertently repeated the Policy-only result in its 1:1 Policy+NF row; the correct value for this distinct model is ''60.4/50.1''. We apologize and have corrected Table 12 (formerly Table 9), Fig. 3(a), §1, §4.3, and Appendix §6.3.3.
  1. Scene splits and matched adaptation (Weakness 2; Q2–Q3) Full-mixture training includes ~0.30M RoomTour3D, ~0.30M Sekai, and ~0.78M Isaac NF samples. All Isaac videos come from VLNVerse training scenes, with no overlap with val-unseen evaluation scenes. For adaptation, both the ''0%'' and ''40%'' settings start from the same Habitat-only Stage-I checkpoint and undergo Stage-II training with matched Habitat replay, Policy-supervised update counts, and total optimizer updates. The 0% control uses Habitat only; the 40% setting adds RoomTour3D, Sekai, and Isaac videos for NF while retaining identical Habitat Policy supervision. Isaac makes up 40% of the full Stage-II NF mixture, including source replay. VLNVerse SR/SPL improves:
  • Fine: 60.12/48.54 → 67.03/60.42 (+6.91/+11.88).
  • Coarse: 35.09/30.32 → 42.16/36.36 (+7.07/+6.04). These matched comparisons support video-based NF adaptation without additional Policy labels; §4.4 ablates Internet and Isaac video contributions. We clarified the data, scene splits, and replay controls in §3.4, §4.1, §4.3, Appendix §§6.3.4–6.3.5 and §6.4.1, and Table 14.
  1. RoomTour3D pseudo-actions (Q4) We derive approximate pseudo-waypoints from RoomTour3D camera trajectories. A five-frame window is kept only if all four transitions satisfy tolerances for Habitat-compatible motions (0.25 m forward or 30° in-place rotation). Each transition is interpolated into two half-steps, producing eight future [x, y, yaw] waypoints from the released projected coordinates. With the NF mixture fixed at H(habitat; R2RCE, RxRCE)+R(Roomtour)+S(Sekai)+I(Issac), adding these pseudo-actions to Habitat Policy supervision changes SR/SPL as follows:
  • RxR-CE: 63.6/52.5 → 63.9/52.6 (+0.3/+0.1).
  • VLNVerse-Fine: 67.0/60.4 → 67.8/60.6 (+0.8/+0.2). Only 9,264/944,235 measurable windows (0.98%) pass the action-consistency filter, mostly forward-motion sequences. This limited coverage helps explain the modest pseudo-action gains: reliable pseudo-labels complement Policy learning, whereas NF can exploit much broader video data without recovering actions. We added the protocol and results to Appendix §6.3.6 and Table 13.

Overall, the new evidence isolates NF's contribution, validates scene-disjoint and matched adaptation, and clarifies the complementary roles of visual prediction and pseudo-action supervision.

I thank the authors for the thorough response and for openly correcting the Table 9 error. The isolated NF contribution (+1.5/+2.2 on RxR-CE) and the scene-disjoint Isaac data address my two main concerns. The pseudo-action experiment is also appreciated.

With the corrections, the main claim should now read as “NF contributes +1.5/+2.2 points, and additional videos provide a further +3.2/+2.4.” I would encourage the authors to make this decomposition explicit in the abstract and §1, rather than reporting only the combined 58.9 → 63.6 figure.

I am maintaining my rating at 3 (Weak acceptance).

Thank you for your positive reassessment and constructive suggestions.

We fully agree that the contributions of NF supervision and additional videos should be distinguished. Accordingly, we have revised the Abstract and §1 (Introduction) to explicitly report the decomposition on RxR-CE: adding NF under matched data and optimization budgets improves SR/SPL from 58.9/47.9 to 60.4/50.1 (+1.5/+2.2 points), while additional videos provide a further +3.2/+2.4 points, reaching 63.6/52.5. This distinction is also clarified in §4.3 and Appendix §6.3.3.

We sincerely appreciate your feedback, which has helped us present the contributions and supporting evidence more precisely.

Comments on the paper

4 comments
AuthorsOpenReview ↗

Controlled Evidence for NF, Shared Prediction, and Video-Based Adaptation

We thank the reviewer for the rigorous feedback. Our new controls isolate NF gains, evaluate shared prediction, clarify video-data splits and adaptation, and extend evaluation to PointGoal and real scenes.

  1. NF vs. extra videos (Weakness 1; Q1) Policy-only and Policy+NF use identical Habitat trajectories, Policy update counts and ~30K total optimizer steps; NF targets use subsequent observations from those trajectories, not extra videos. On RxR-CE (SR/SPL), Policy-only achieves 58.9/47.9; Policy+NF (H only) 60.4/50.1; Policy+NF (H+R+S+I) 63.6/52.5. Thus NF adds +1.5/+2.2 at fixed data/budget, and external videos add +3.2/+2.4. Original Table 9 mistakenly duplicated the Policy-only number in its 1:1 Policy+NF row. The correct result for that distinct model is 60.4/50.1. We apologize and corrected Table 12 (formerly Table 9), Fig. 3(a), §1, §4.3 and Appendix §6.3.3.

  2. Shared predictor and feature granularity (Weakness 1; Q2) We compare Shared dual-stream DiT with separate Policy/NF DiTs. Both retain the same trainable VLM conditioning, dual-stream design, Stage-I/II Policy data and NF sources (40% Isaac in Stage-II NF); only prediction-block sharing differs. Patch/Pool denotes current/future features. Under Patch/Patch, Shared vs Separate: RxR-CE 63.6/52.5 vs 59.0/48.3 (+4.6/+4.2); VLNVerse-Fine 67.0/60.4 vs 63.4/56.3 (+3.6/+4.1). Shared also wins with Patch/Pool (RxR-CE +2.6/+2.2; Fine +3.9/+2.7) and Pool/Pool (RxR-CE +3.2/+3.7; Fine +1.8/+1.1). Thus NF directly trains the waypoint predictor, beyond shared VLM conditioning. With current Patch fixed, replacing future Patch with Pool yields RxR-CE 63.6/52.5→63.8/52.6 and Fine 67.0/60.4→68.1/59.8. With future Pool fixed, current Patch rather than Pool improves RxR-CE +2.0/+1.4 and Fine +3.5/+4.1. Spatially detailed current observations thus matter more than patch-level future targets. Relative to LDA-1B, UniNav and Fast-WAM, NaviWAM combines task/history-conditioned navigation, shared prediction and video adaptation with source Policy replay. See §2.2, §3.3, §4.4, Table 8, Appendix §6.1.1.

  3. Video sources / scene splits (Weakness 2; Q3) With Habitat Policy supervision fixed, H=Habitat, R=RoomTour3D, S=Sekai, I=Isaac. RxR-CE / Fine SR/SPL:

  • H: 60.4/50.1; 60.1/48.5
  • H+R+S: 62.6/51.3; 61.8/48.9
  • H+I: 62.0/50.8; 66.2/58.4
  • H+R+S+I: 63.6/52.5; 67.0/60.4 Both Internet (R+S) and Isaac videos help without waypoint labels; their mixture is best, though source volumes are not matched. The full setup adds ~0.30M R, ~0.30M S, ~0.78M I NF samples. Isaac videos come exclusively from VLNVerse training scenes disjoint from val-unseen; synthetic videos supervise NF, not Policy. See §3.4, §4.1, §4.3–4.4, Appendix §§6.3.2–6.3.5, Tables 7/11.
  1. Adaptation and real scenes (Weakness 2; Q4) Both 0% and 40% Stage-II runs start from the same Habitat-only Stage-I checkpoint, matching Habitat replay, Policy-supervised updates and total optimization. The 0% control uses Habitat Policy+NF only; the 40% run adds R+S+I NF videos with identical source Policy supervision. Isaac is 40% of the entire Stage-II NF mixture including replay. VLNVerse Fine SR/SPL: 60.12/48.54→67.03/60.42 (+6.91/+11.88); Coarse: 35.09/30.32→42.16/36.36 (+7.07/+6.04). The Shared/Separate ablation uses the complete two-stage recipe, not a Stage-II-only predictor swap from the same Stage-I checkpoint. For real-world transfer, NF training uses five 3DGS scenes; closed-loop evaluation uses five disjoint scenes (600 episodes). Additional 3DGS NF yields SR/SPL 58.33/42.68→64.33/45.67 (+6.00/+2.99) and NE 2.81→2.31 m. Preliminary physical tests yield 6/10 vs 3/10 successes; physical sites are excluded from NF training but overlap held-out reconstructed evaluation scenes. These results are preliminary, not evidence of broad real-world generalization. See §4.3–4.5, Tables 9/14, Appendix §§6.3.1, 6.3.4–6.3.5, 6.4.1.

  2. PointGoal and uncertainty (Q5) With matched Policy samples and updates, NF improves all four PointGoal SR/SPL settings:

  • Sys1-PG Home: 74.4/68.4→77.4/70.2
  • Sys1-PG Commercial: 70.8/68.7→74.7/71.7
  • VLNVerse-PG: 83.2/80.1→87.0/83.7
  • VLNVerse-PG NPC: 75.9/70.7→81.1/78.5 Gains span +3.0–5.2 SR and +1.8–7.8 SPL. Dynamic obstacles reduce SR/SPL vs static VLNVerse-PG by 7.3/9.4 without NF versus 5.9/5.2 with NF, suggesting greater robustness. On Sys1-PG Home, five action-sampling seeds on each fixed checkpoint yield SR 74.4 ± 0.87 (without NF) vs 77.4 ± 0.94 (with NF), mean±SD; this measures inference randomness, not independent training seeds. See Table 2, §4.2.

Overall, these controls support shared visual prediction and NF as a complement to waypoint supervision.

Thank you for the detailed responses and for correcting the duplicated result in the original Table 9. The matched Policy-only versus Policy+NF comparison on the same Habitat corpus and with matched Policy updates addresses my main attribution concern. The scene-split clarification and matched Stage-II protocol also resolve my questions about Isaac-scene overlap and the 0% versus 40% adaptation settings. The shared-versus-separate predictor ablation is useful evidence for the value of sharing prediction blocks.

I retain two caveats. First, the reported five-seed PointGoal results measure inference sampling variability, not variation across independently trained models. The principal NF and adaptation gains would be more convincing with training-seed uncertainty. Second, the Patch-versus-Pool comparison suggests that patch-level future targets are not necessary for the reported gains, while spatial detail in the current input appears more consistently useful. I would therefore keep the claim focused on the tested shared Policy–NF training scheme.

Overall, the revised evidence addresses the specific concerns in my review and does not give me a clear basis to lower my rating. I am maintaining my original rating of 2 (Weak rejection), while noting that the seed-robustness and target-granularity questions limit the strength and novelty of the current claims.

Thank you for your constructive reassessment and valuable suggestions. We appreciate your recognition of the controlled NF comparison, matched adaptation protocol, and shared-predictor ablation. To further characterize training robustness and clarify the architectural contribution, we conduct an additional training-seed analysis using PointGoal as a representative evaluation setting.

  1. Training-seed uncertainty analysis Beyond the previously reported five action-sampling evaluations, we further examine variability across independently trained models on Sys1-PG Home and Commercial. Specifically, we conduct three independent paired training runs. Within each pair, Policy-only and Policy+NF share the same randomly initialized WAM, while different pairs use independent WAM initializations. All runs use identical pretrained Qwen3-VL-4B-Instruct and DINOv2-S weights, matched training data, Policy-supervised updates, and total optimization budgets. Each checkpoint is evaluated on identical episodes with five common action-sampling seeds. We first average the five evaluations per checkpoint and then compute the paired NF gains across the three independent training runs. Thus, the statistical unit is the training run (n = 3), rather than the individual evaluation seed. The resulting NF gains are summarized below (percentage points):

| Comparison | Seed 1 | Seed 2 | Seed 3 | Mean ± SD | 95% CI | | Home SR | +3.0 | +2.8 | +5.2 | 3.67 ± 1.33 | [0.36, 6.97] | | Home SPL | +1.8 | +0.5 | +5.6 | 2.63 ± 2.65 | [-3.95, 9.22] | | Commercial SR | +3.9 | +6.9 | +1.5 | 4.10 ± 2.71 | [-2.62, 10.82] | | Commercial SPL | +3.0 | +5.6 | +1.1 | 3.23 ± 2.26 | [-2.38, 8.85] |

The confidence intervals are computed using paired Student's t intervals with two degrees of freedom. All three independent training pairs yield positive NF gains in both SR and SPL across the evaluated PointGoal settings. The mean improvements range from +3.67 to +4.10 points in SR and +2.63 to +3.23 points in SPL. Although the confidence intervals remain relatively wide with three training pairs, the consistent direction of improvement provides additional evidence that the observed PointGoal gains are not specific to a single WAM initialization. This analysis complements our matched-data NF comparison and controlled Stage-II adaptation experiments by further characterizing the sensitivity of the shared Policy–NF scheme to independent training initialization in PointGoal navigation.

  1. Feature granularity and architectural contribution We agree with your interpretation of the Patch-versus-Pool ablation. With Patch-level current inputs fixed, replacing future Patch targets with Pool targets yields comparable results (RxR-CE: 63.6/52.5 vs. 63.8/52.6; VLNVerse-Fine: 67.0/60.4 vs. 68.1/59.8). In contrast, with future Pool targets fixed, retaining Patch-level current inputs improves SR/SPL by 2.0/1.4 points on RxR-CE and 3.5/4.1 points on VLNVerse-Fine. These results indicate that preserving spatial detail in current observations is more consistently beneficial than retaining patch-level future prediction targets under the evaluated configurations. Accordingly, we have refined our claims to emphasize the shared, task–history-conditioned Policy–NF training scheme, rather than the necessity of patch-level future targets. Patch-level future prediction remains our default implementation, while the ablations demonstrate that the benefits of shared prediction extend to pooled future targets.

Thank you again for your constructive feedback. The additional training-seed analysis and architectural clarifications have helped us strengthen the empirical characterization and more precisely position the contribution of NaviWAM.

AuthorsOpenReview ↗

Summary of Contributions, Revisions, and Additional Experimental Evidence

Dear Area Chair and Reviewers,

We sincerely thank the Area Chair and all reviewers for their careful evaluations and constructive feedback. The discussions have helped us strengthen the experimental evidence, clarify our contributions, and refine the scope of our claims. We have addressed the main concerns through additional controlled experiments, methodological clarifications, and manuscript revisions, with the changes highlighted in blue.

Main Contributions:

NaviWAM investigates how visual predictive supervision can complement trajectory learning for embodied navigation. We propose a task- and history-conditioned framework that couples trajectory prediction (Policy) and next-frame feature prediction (NF) through a shared dual-stream predictor. This design enables navigation policies to learn from additional video–instruction data without trajectory annotations, while avoiding future-frame generation at inference. We further develop a two-stage training scheme that reuses NF supervision for offline target-domain video adaptation while retaining source Policy supervision. Experiments across VLN and PointGoal benchmarks demonstrate the effectiveness of this framework.

Key Revisions and Additional Evidence:

In response to the reviewers' feedback, we have made the following improvements:

  1. Isolating the contribution of NF supervision. We conducted a controlled Policy-only versus Policy+NF comparison using identical Habitat data, matched Policy updates, and matched total optimization budgets. On RxR-CE, NF independently improves SR/SPL from 58.9/47.9 to 60.4/50.1 (+1.5/+2.2 points), while additional videos provide a further +3.2/+2.4 points, reaching 63.6/52.5. We also corrected the previously duplicated result and explicitly separated these contributions in the Abstract, Introduction, and experimental analysis.

  2. Validating the shared predictor and refining architectural claims. New Shared-versus-Separate predictor ablations demonstrate the benefit of shared prediction blocks beyond common VLM conditioning. Under the default configuration, sharing improves RxR-CE SR/SPL from 59.0/48.3 to 63.6/52.5. Additional Patch/Pool ablations show that preserving spatial detail in current observations is more consistently beneficial than using patch-level future targets. Accordingly, we have refined our claims to emphasize the shared Policy–NF learning framework rather than the necessity of a particular future-target granularity.

  3. Strengthening video-data and adaptation evaluations. We added video-source ablations to distinguish the contributions of Internet and Isaac videos, and verified that Isaac training scenes are disjoint from VLNVerse val-unseen scenes. We also clarified that Stage-II adaptation settings share identical initialization, source replay, Policy updates, and optimization budgets. The additional NF video mixture improves VLNVerse fine-grained SR/SPL by +6.91/+11.88 points and coarse-grained SR/SPL by +7.07/+6.04 points, without target-domain trajectory supervision.

  4. Extending empirical validation and reproducibility. We added real-scene 3DGS closed-loop evaluations across 600 episodes in held-out scenes, where additional NF supervision improves SR from 58.33% to 64.33%, together with preliminary physical-robot results. We also investigated RoomTour3D pseudo-action supervision, expanded the data collection and training protocol descriptions, and clarified the complementary roles of NF and waypoint supervision. In our follow-up response, we further reported three paired independent PointGoal training runs to characterize training-seed variability, while acknowledging the uncertainty associated with the limited number of runs.

Final Remarks:

We appreciate the reviewers' recognition that the additional experiments and clarifications have addressed several central concerns, particularly regarding NF attribution, shared prediction, scene separation, and adaptation controls. We also acknowledge the remaining limitations concerning training robustness and coarse-grained navigation, and have refined our claims accordingly.

We sincerely thank the Area Chair and reviewers for their thoughtful feedback and consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.