acceptodds
Desk rejected at ICLR 2027

OpenNWM: A Generalist Navigation World Model with Latent Action Pretraining

Desk rejected at ICLR 2027, so left out of the paper lists and the map.

Abstract

Navigation world models predict future observations conditioned on navigation actions to support planning, but their generalization across real-world environments remains constrained by the limited coverage of action-labeled data. Action-free navigation videos provide diverse visual experience, yet lack the recorded actions required for action-conditioned training. We introduce OpenNWM, a generalist navigation world model with latent action pretraining. It learns universal latent actions from large-scale action-free videos through visual prediction pretraining, and further aligns them with physical motions via action-conditioned post-training. To support video pretraining, we curate NavAnywhere, a large-scale action-free navigation video dataset comprising 17.5 million visual observations from 15 heterogeneous sources. It captures navigation by ground robots, humans, and drones, with diverse viewpoints and motion patterns across five broad scene types: residential areas, public and commercial spaces, urban settings, parks, and natural landscapes. Experiments demonstrate improved visual prediction on in-domain and out-of-domain benchmarks, together with improved navigation performance. These results show that latent-action pretraining enables diverse, action-free videos to improve dynamics learning and generalization for open-world navigation.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Navigation World Model

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_eXj8OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes OpenNWM, which uses action-free videos to train navigation world models. A latent action model trained with pixel reconstruction plus action supervision provides latent actions to pretrain a CDiT-B/2 world model on NavAnywhere ; the latent-action adapter is then replaced by a physical-action adapter for post-training and CEM planning. The paper claims state-of-the-art visual prediction and navigation on in-domain and out-of-domain benchmarks.

Critical strengths
  1. The problem is well motivated. Action-labeled navigation data is limited, and using action-free video is a sensible direction.
  2. The ablations on pretraining conditioning and LAM supervision (Tables 4-5) are informative.
Critical weaknesses
  1. The navigation gains are very small (RECON ATE 1.13 to 1.12, SCAND 1.01 to 0.98 in Table 2).
  2. Weak experimental comparison. It is not stated which baselines were re-run and which numbers were taken from prior papers (e.g., the NWM row is listed at 279M parameters, while its RECON numbers match those reported for the original larger model), and the ID-average advantage over RAE-NWM mostly comes from RAE-NWM's collapse on TartanDrive (LPIPS 0.662, Table 9). It would be better if the authors could unify all baselines under the same protocol or clarify the comparison.
  3. Missing controls for the main claim. There is no no-pretraining baseline, and no comparison with pseudo-action pretraining (an inverse dynamics model or VO/camera poses, which many NavAnywhere sources already have). Without these, it is unclear why latent actions are needed.
  4. Several claims are overstated. The claim that UNLAs "disentangle agent motion from environmental dynamics" is never tested, and the latent adapter is discarded at post-training, so there is no "shared latent-action interface" at test time. NavAnywhere is also about 75% Ego4D (13.09M of 17.5M frames), which weakens the diversity claim.
Questions for authors

As in weaknesses.

3 comments

Thank you for the constructive feedback. We address each weakness and question below.

W1: Small navigation gains.

R1: OpenNWM is designed to improve generalization to unseen environments by learning from videos without action labels. During pretraining, latent actions provide a common representation of motion across videos, allowing the world model to learn how motion affects future observations across diverse scenes. We therefore expect the benefit to be greater in unseen environments.

On RECON and SCAND, both models are trained and evaluated within the same domains. This places less demand on cross-domain generalization and may partly explain the modest ATE reductions from 1.13 to 1.12 on RECON and from 1.01 to 0.98 on SCAND. On the unseen Office Go2 dataset, OpenNWM reduces ATE from 3.03 to 2.89, a 4.6% reduction relative to NWM (Table 2 of the paper).

To further assess generalization, we evaluated both models offline on recordings collected with a Unitree Go2 quadruped in unseen real-world environments. We used the same image-goal navigation protocol for both models, without adapting either model to the recordings. Across all test recordings, OpenNWM reduced ATE by 8.1% and RPE by 8.8% relative to NWM (Table 1).

Table 1. Offline image-goal navigation on Unitree Go2 recordings from unseen real-world environments.

MethodATE (m) ↓RPE (m) ↓
NWM0.11770.0478
OpenNWM0.10820.0436

Side-by-side videos are available on our Real-world supplementary page.

W2: Baseline provenance and protocol.

R2: All baseline results in Table 1 of the paper are local evaluations of released checkpoints under the same visual-prediction benchmark protocol. In Table 2 of the paper, the results for GNM, NoMaD, CompACT, Garrido et al., and the RECON/SCAND entries of RAE-NWM are paper-reported; NWM/OpenNWM and all Office Go2 results are locally evaluated. We have marked every paper-reported number with a superscript asterisk (*) in the revised PDF. Both paper-reported and locally evaluated results follow the same benchmark tasks and metric definitions.

Regarding the reviewer’s concern about the NWM model size, we rechecked the checkpoint and evaluation outputs. The NWM baseline in Table 2 uses the released CDiT-B checkpoint with 279M parameters. We additionally evaluated the CDiT-XL checkpoint on with 1.096B parameters on RECON:

ModelParametersATE ↓RPE ↓
NWM-B (Table 2)279M1.13120.3499
NWM-XL (100K)1.096B1.17510.3465

The CDiT-B results round to 1.13/0.35. Although these match the original paper’s XL results at two-decimal precision, our checkpoint verification confirms that they were obtained using CDiT-B, with no model-size mismatch.

Excluding TartanDrive, OpenNWM achieves comparable in-domain visual-prediction performance to RAE-NWM (Appendix Table 9 in the paper). It also achieves better visual-prediction results on the unseen Go Stanford, TUM RGB-D, Planetary Rover, and Office Go2 domains (Table 1 in the paper), and better navigation performance on RECON, SCAND, and Office Go2 (Table 2 in the paper). These results highlight OpenNWM's consistent advantages in navigation and its ability to generalize to unseen environments. We have revised the manuscript to clarify the provenance of the baseline results and the evaluation protocols used.

W3. No-pretraining and pseudo-action controls.

R3: We have revised Table 4 to explicitly label the no-pretraining baseline and clarify how the geometry-based pseudo-actions are obtained. We present the two comparisons separately below.

1. Does latent-action pretraining improve downstream performance? The baseline skips world-model pretraining and trains the world model directly on physical actions. Both variants use the same architecture and downstream training data. Latent-action pretraining improves all six metrics, reducing direct LPIPS on Office Go2 from 0.628 to 0.587 and navigation ATE by 6.8% on RECON and 24.1% on SCAND (Response Table 2).

Table 2. Effect of latent-action pretraining on visual prediction and navigation.

Pretraining settingOffice Go2 direct LPIPS ↓Go Stanford AR LPIPS ↓RECON ATE ↓RECON RPE ↓SCAND ATE ↓SCAND RPE ↓
No pretraining0.6280.6671.1980.3671.2520.316
Latent-action pretraining0.5870.6581.1170.3560.9500.267

2. How do learned latent actions compare with geometry-based pseudo-actions? We use VGGT-Omega to estimate camera trajectories from RGB videos and derive pseudo-actions from relative pose changes. The two variants use the same video subset, time conditioning, world-model training schedule, and downstream evaluation protocol. Learned latent actions perform better on five of six metrics, including both visual-prediction metrics and navigation ATE on both datasets. Geometry-based pseudo-actions yield slightly lower RECON RPE (0.353 vs. 0.356) (Response Table 3).

Response Table 3. Effect of the action signal used during world-model pretraining.

Pretraining action signalOffice Go2 direct LPIPS ↓Go Stanford AR LPIPS ↓RECON ATE ↓RECON RPE ↓SCAND ATE ↓SCAND RPE ↓
Geometry-based pseudo-actions (VGGT-Omega)0.5940.6681.1520.3531.0390.283
Learned latent actions0.5870.6581.1170.3560.9500.267

Overall, these results demonstrate the effectiveness of latent action pretraining in leveraging diverse action-free visual experiences to improve world model learning and downstream navigation performance. Furthermore, our learned latent actions offer an advantage over geometry-based pseudo-actions by capturing more effective motion representations from visual experiences, providing stronger motion conditioning for world-model pretraining.

W4: Claims and dataset diversity.

R4: We agree that our experiments do not directly test whether agent motion is disentangled from environmental dynamics. We have revised the claim to state that physical-motion supervision encourages motion-related structure in the latent representation. We have also clarified that the shared latent-action interface is used only during world-model pretraining. During post-training, we replace the latent-action adapter with a physical-action adapter. At inference, the planner samples physical waypoint actions directly, without using the LAM.

We acknowledge that NavAnywhere is imbalanced in its raw data composition, with Ego4D accounting for 74.8% of the sampled frames (13.09M out of 17.50M, Appendix Table 6). Importantly, source-balanced sampling is already adopted in our training pipeline to prevent larger datasets from dominating the training process. With this strategy, Ego4D contributes only 27.1% of the samples drawn from NavAnywhere during pretraining. Moreover, Ego4D itself covers diverse activities and a wide range of indoor and outdoor environments. We have clarified the source-balanced sampling strategy in the revised PDF.

Dear Reviewer eXj8,

We sincerely appreciate your time and effort in reviewing our manuscript and offering valuable suggestions. As we're near the end of discussion period, we would like to confirm whether our responses have effectively addressed your concerns. We have provided detailed responses to your concerns, and we hope they have adequately addressed your issues.

Best regards.

Reviewer_sTC9OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors present a generalist world model, OpenNVM, that learns universal latent actions from large-scale action-free videos. These latent actions are further aligned with physical motions through action-conditioned post-training. This whole process enables generalizable navigation capabilities across diverse environments, embodiments and motion patterns. To support training, the paper introduces the NavAnywhere dataset, compiled from diverse sources, alongside Universal Navigation Latent Actions (UNLAs), a mechanism that disentangles agent motion from environmental dynamics and unifies heterogeneous visual experiences within a shared latent space. The introduced OpenNWM achieves state-of-the-art performance in visual prediction, navigation, and out-of-distribution generalization.

Critical strengths
  • The core concept of using large-scale, action-free navigation videos for latent-action pretraining is intriguing and strong. By proposing continuous UNLAs, the method effectively captures visual transitions while preserving controllable motion semantics. This approach addresses the scarcity issue of action-labeled dataset for action-conditioned training by disentangling ego-motion from complex environmental dynamics. Consequently, this enables the learned world model to achieve strong out-of-distribution generalization across heterogeneous platforms, diverse environments, and motion dynamics.
  • The experiment section is comprehensive and well-excuted. The paper includes not only visual prediction and downstream navigation performance benchmark results but also extensive out-of-distribution generalization experiments which is critical to demonstrate the scalability of the proposed methods. Furthermore, details of dataset description, thorough implementation details, quantitative and qualitative results, and insightful analysis in the appendix make the whole paper complete.
Critical weaknesses

The main weakness of this paper is the presentation, which makes key architecture and training procedure hard to follow:

  • The connection between text and figures are weak. For example, Figure 2 and the main method section are poorly integrated. Linking each subsection or paragraph to corresponding modules in the figure explicitly would significantly improve the readability.
  • It’s ambiguous to the reader of the multi-stage training pipeline. Although Figure 3 shows the three-stage training process, the precise training objectives or optimized parameters for each phase remain unclear. For instance, in Figure 3(a), it is ambiguous whether the latent action encoder is updated or frozen during this phase.
  • Some notations are not defined or vaguely described in the paper (see the Questions section below).
Questions for authors
  1. Does the pipeline exhibit a bias toward scenarios where physical action supervision is available compared to purely action-free data?
  2. The notation for visual observations appears inconsistent between text (using xx) and Figure 2 (using oo). Are they both representations for visual observations? If so, we should unify them.
  3. In the “Conditional Diffusion Dynamics” section, in the text, should the history indexing of s0:τs_{0:\tau} be revised to sτ−h:τs_{\tau-h:\tau} to correctly reflect the history horizon hh and to align with the equation below? Moreover, the equation misses reference number.
  4. What are mathematical definition of CnavC_{nav} and SgoalS_{goal} ?
  5. It’s hard to interpret Figure 5. Could the authors clarify why joint supervision shows more coherent cluster organization?
3 comments

Thank you for the detailed suggestions. We have clarified the figures, training stages, and notation in the PDF. We address each weakness and question below.

W1: Connecting Fig. 2 to the method.

R1: We have revised Secs. 3.2 and 3.3 to link the method description directly to the corresponding modules in Fig. 2. Sec. 3.2 identifies the latent-action encoder and the auxiliary heads used for RGB reconstruction and physical-action prediction. Sec. 3.3 explains how the frozen image tokenizer encodes visual observations, how the CDiT attends to historical visual tokens through cross-attention, and how the latent-action adapter conditions the CDiT. The revised figure also distinguishes the latent action passed to the world model from the auxiliary heads used only during LAM training.

W2: Training stages and updated parameters.

R2: The latent-action encoder is frozen during world-model pretraining in Fig. 3(a). We train the LAM separately beforehand and use its encoder to infer latent actions from video frames. We have revised Sec. 4.1 and Fig. 3 to specify the objective and the parameters updated at each stage. Fig. 3(a) now explicitly marks the encoder as frozen.

  • Fig. 3(a) — World-model pretraining. We train the CDiT and latent-action adapter with a diffusion objective conditioned on the prediction horizon and, for eligible frame pairs, the latent actions inferred by the frozen encoder.
  • Fig. 3(b) — Physical-action adapter warm-up. We replace the latent-action adapter with a newly initialized physical-action adapter. With the CDiT frozen, the physical-action-conditioned diffusion objective updates only the new adapter.
  • Fig. 3(c) — Joint post-training. We unfreeze the CDiT and use the same objective to update the CDiT and physical-action adapter jointly.

The image VAE remains frozen throughout all three stages. Appendix C.1 (pp. 19–20, Table 8) provides the detailed optimization settings.

W3. Undefined or inconsistent notation.

R3: We have standardized the notation for visual observations, corrected the history indices, added the missing equation number, and defined the quantities used in the planner. Our responses to Questions 2–4 below detail these revisions.

Q1. Does the training pipeline favor scenarios with physical-action supervision?

R1: OpenNWM is designed to improve generalization to unseen environments by learning from large-scale video data without action labels. Action-free videos expose the model to a wider range of scenes, viewpoints, and motion patterns. Action-labeled trajectories help ground the learned latent actions in physical translation and rotation. During latent action model (LAM) training, we reserve 25% of each batch for action-labeled samples (Appendix C.1). Both labeled and unlabeled samples contribute to the future-frame reconstruction loss, while labeled samples also contribute to the physical-motion prediction loss (Sec. 3.2, Eq. 4). During world-model pretraining, the frozen LAM infers latent actions from RGB frame pairs, enabling the world model to learn action-conditioned dynamics from videos without recorded actions.

Contribution of action-free videos. We held the amount of action-labeled data fixed and varied the fraction of the unlabeled video pool used to train the LAM. As this fraction increased from 25% to 100%, the DreamSim score on RECON decreased at every step, with a 6.08% relative reduction overall (Appendix E, Fig. 8).

Complementary roles of visual and physical-action supervision. Joint supervision yields the lowest prediction error on both held-out domains in Response Table 1. Adding visual reconstruction to action-only supervision reduces direct-prediction LPIPS from 0.598 to 0.587 on Office Go2 and autoregressive LPIPS from 0.670 to 0.658 on Go Stanford. Conversely, adding physical-motion prediction to visual-only supervision improves both held-out prediction metrics and three of the four navigation metrics.

Table 1. Effect of LAM supervision on downstream visual prediction and navigation (Table 5 of the paper). Lower is better for all metrics.

LAM supervisionOffice Go2 direct LPIPSGo Stanford autoregressive LPIPSRECON ATERECON RPESCAND ATESCAND RPE
Action only0.5980.6701.1380.3650.9810.274
Visual only0.6050.6741.1760.3520.9820.274
Visual + action0.5870.6581.1170.3560.9500.267

We also evaluated OpenNWM and NWM offline on recordings collected with a Unitree Go2 quadruped in unseen real-world environments. Without adapting either model to these environments, OpenNWM achieved 8.1% lower ATE and 8.8% lower RPE than NWM. These findings show that OpenNWM benefits from action-free video and transfers to environments that provided no action labels during training. A systematic preference for action-labeled scenarios remains unquantified, as the experiments do not compare labeled and unlabeled scenarios with matched scene distributions. Real-world evaluation

Q2. Observation notation.

R2: Both oo and xx denote RGB observations. We have replaced oo with xx in Fig. 2 to match the notation in the text.

Q3. History indexing and equation numbering.

R3: We have corrected the history notation to sτ−h+1:τs_{\tau-h+1:\tau}, where hh is the history length and h=4h=4 in our experiments. We have also numbered the denoising equation in Sec. 3.3.

Q4. Definitions of SgoalS_{\mathrm{goal}} and CnavC_{\mathrm{nav}}.

R4: SgoalS_{\mathrm{goal}} scores how closely the predicted terminal observation matches the goal, and CnavC_{\mathrm{nav}} is an optional penalty for collisions or infeasible actions. In our experiments, we set

\qquad C_{\mathrm{nav}} = 0,$$ where $\hat{x}_H$ is the predicted terminal image, $\tilde{x}_g$ is the VAE reconstruction of the goal image, and $\mathrm{Alex}$ denotes the AlexNet backbone used to compute LPIPS. Substituting these definitions into Eq. (9) gives $E = \mathrm{LPIPS}_{\mathrm{Alex}}(\hat{x}_H, \tilde{x}_g)$ for each rollout. The planner therefore minimizes the expected terminal LPIPS distance to the goal. We have added these definitions to Sec. 3.4 of the revised manuscript. **Q5. Interpreting Fig. 5.** **R5:** Each point represents a pair of observations. We apply K-means ($K=8$) separately in each model’s 32-dimensional latent-action space. A point’s color indicates its latent cluster, and its position gives the measured physical motion $(\Delta x, \Delta y, \Delta\mathrm{yaw})$ between the observations. Within each panel, a compact group of same-colored points therefore indicates that samples assigned to the same latent cluster correspond to similar physical motions. Compared with pixel-only supervision, joint action–pixel supervision produces more coherent groups in physical-motion space. The action-prediction objective encourages the latent representation to retain information about translation and rotation alongside the visual changes needed for reconstruction. Action-only supervision yields the most compact clusters, while joint supervision retains a similar motion-related structure (Appendix F.2). This is qualitative evidence of alignment with physical motion. We have clarified the plotting procedure and this interpretation in the revised Fig. 5 caption.
Reviewer_7hnjOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes OpenNWM. The proposed model has a latent action encoder in pretraining stage outputing latent actions as condition to the CDiT of NWM. In post training, the latent action encoder is replaced by an actual action encoder through two-stage post-training. By using latent action, OpenNWM can be pretrained on data without action annotation. The authors organized a NavAnywhere dataset comprising 17.5M frames. The final model exhibits performance improvements in predicting future visual observations and action trajectories.

Critical strengths
  1. It is a notable effort to ensemble 15 open dataset with a wide coverage. Large scale pretraining and making use of video data without action label are important.

  2. Experiments show improvements over many baselines. Ablation backs the latent action conditioning and pixel+action supervision during pretraining.

Critical weaknesses
  1. empirical strength. latent action pretraining is promising but the actual performance improvements compared to baseline such as NWM in table 2 seem to be limited, especially considering the pretraining budget that the baseline does not have. Are we really seeing strong enough positive signs from the pretraining compared to the baselines?

  2. Key baseline missing. I think for navigation pretraining when action label is absent, a key baseline to compare with is using visual odometry to get action estimates as what was done in CityWalker. It would be much more convincing if there is an experiment comparing the latent action and the estimated action head-to-head in pretraining, even in a smaller scale, and show that the latent action outperform the estimated action.

  3. Clarity. The latent action encoder is only used in pretraining and replaced by an actual action encoder from scratch during post training. The latent action is also not relevant at all during inference time sampling. This is shown in Figure 3 but only discussed in appendix. But I think it is an important detail to mention in the main part of the paper.

Questions for authors

If we skip pretraining and train OpenNWM with actual action directly on posttraining data, what would be the performance?

4 comments

Thank you for the helpful suggestions. We address each weakness and question below.

W1&Q1: Performance gains and pretraining budget.

R1. We address the reviewer’s concern with two comparisons.

1) Latent-action pretraining improves both visual prediction and navigation. We compare the same OpenNWM architecture trained on the same downstream data under two settings: with latent-action pretraining and without any pretraining (the no-pretraining baseline). This no-pretraining baseline was already included in the first row of Table 4 in the original paper, but its label was not sufficiently explicit. We have now explicitly mentioned it as no-pretraining in the revised PDF. As shown in Response Table 1, pretraining improves all six metrics. It reduces direct prediction LPIPS from 0.628 to 0.587 and lowers navigation ATE by 6.8% on RECON and 24.1% on SCAND.

Table 1. Effect of latent-action pretraining on OpenNWM (paper Table 4).

PretrainingOffice Go2 direct LPIPS ↓Go Stanford AR LPIPS ↓RECON ATE ↓RECON RPE ↓SCAND ATE ↓SCAND RPE ↓
Without pretraining0.6280.6671.1980.3671.2520.316
With latent-action pretraining0.5870.6581.1170.3560.9500.267

2) At a comparable scale of unlabeled video data, OpenNWM outperforms a substantially larger model on navigation. NWM XL + Ego4D uses over 900 hours of additional Ego4D video, comparable in source-video duration to the approximately 1,200 hours in the full NavAnywhere corpus. Under the same navigation evaluation protocol, OpenNWM uses roughly one-quarter as many parameters (280M vs. 1,099M) and achieves lower ATE on all three datasets. The relative reductions are 14.7% on RECON, 3.5% on SCAND, and 17.9% on the out-of-domain Office Go2 dataset (Response Table 2).

Table 2. Navigation comparison with NWM XL + Ego4D.

MethodParams. (M)RECON ATE ↓SCAND ATE ↓Office Go2 ATE ↓
NWM XL + Ego4D1,0991.3151.0183.518
OpenNWM2801.1220.9822.887

W2: Geometry pseudo-action pretraining.

R2: We agree that the approach used in CityWalker is sound. We added a geometry-based control using VGGT-Omega. Its pseudo-labels are relative camera poses estimated from RGB, without ground-truth pose supervision. We have added an explanation of estimated action trajectories to the PDF.

We replace latent actions with these geometry-based pseudo-labels for world-model pretraining, keeping the world-model training recipe and downstream evaluation protocol unchanged. Response Table 3 compares no-pretraining, OpenNWM's latent action pretraining, and pretraining with actions estimated by VGGT-Omega, using the corresponding results from the revised PDF.

Table 3. Pretraining ablation. Selected results from the revised paper. Est. Traj. denotes camera trajectories estimated from visual observations. Direct denotes direct prediction of the target frame; AR (autoregressive) denotes sequential prediction. Best results are bold. Time + Latent Action is OpenNWM. Time + Est. Traj. uses VGGT-Omega estimates.

Pretraining ConditioningOffice Go2Go StanfordRECONRECONSCANDSCAND
Direct-LPIPS ↓AR-LPIPS ↓ATE ↓RPE ↓ATE ↓RPE ↓
None (no pretraining)0.6280.6671.1980.3671.2520.316
Time + Latent Action (latent action pretraining)0.5870.6581.1170.3560.9500.267
Time + Est. Traj. (estimated action pretraining)0.5940.6681.1520.3531.0390.283

OpenNWM improves all six metrics over no pretraining and five of six over estimated action pretraining. Estimated action pretraining is slightly better only on RECON RPE (0.353 vs. 0.356). These results demonstrate the effectiveness of OpenNWM and the overall advantage of learned latent actions over estimated actions in this comparison.

W3: Unclear role of latent actions across stages

R3: In the revised PDF, we have clarified the training procedure in Sec. 4.1 (Implementation Details and Fig. 3) and expanded the inference details in Appendix C.1, complementing the navigation planning formulation in Sec. 3.4. We train the Latent Action Model (LAM) separately and freeze it during world model pretraining. During post-training, we replace the latent adapter with a newly initialized physical-action adapter. We first train this adapter with the CDiT backbone frozen, then fine-tune both jointly. At inference, the planner samples physical waypoint actions directly, without using the LAM or the latent adapter.

Thank you for addressing my questions and concerns. I especially appreciate the new results with pseudo action label from VGGT-Omega. I have no further questions.

Thank you for your valuable suggestions. Your suggestion was very valuable in helping us better demonstrate the effectiveness of latent action pretraining. Thank you again for your insightful feedback!

Comments on the paper

3 comments
AuthorsOpenReview ↗

To all reviewers

We thank all reviewers for their careful reading and constructive feedback. We appreciate the recognition of the potential of learning navigation world models from large-scale action-free videos (Reviewers 7hnj, sTC9, and eXj8). Reviewers 7hnj and sTC9 highlighted the value of NavAnywhere, which brings together 15 diverse data sources, as well as the potential of latent-action conditioning for learning generalizable motion representations. We also appreciate the positive assessments of our experimental results, particularly the comprehensive visual prediction and navigation evaluations, extensive out-of-distribution generalization studies, and informative ablations on latent-action conditioning and LAM supervision (Reviewers 7hnj, sTC9, and eXj8).

In the following responses, we address concerns regarding the effectiveness of latent-action pretraining, comparisons with geometry-based pseudo-actions, baseline consistency, and the clarity of our training pipeline and technical presentation. We provide additional comparisons against no-pretraining and VGGT-Omega-based pseudo-action pretraining, further demonstrate the benefits of source-balanced sampling, and report additional real-world navigation results on Unitree Go2 recordings. We also clarify baseline provenance, training objectives and parameter updates at each stage, the role of latent actions during inference, and the relevant notations and experimental analyses. The revised manuscript incorporates these additional experiments and clarifications. We sincerely appreciate the reviewers' valuable feedback, which has helped us strengthen both the empirical evidence and presentation of OpenNWM.

Dear Reviewers,

Thank you for your time and thoughtful feedback. We would appreciate it if you could take a look at our responses and the updates to the manuscript when you have a chance. We would be very happy to discuss any remaining concerns and provide further clarification. Thank you again for your consideration.

Thank you for addressing my concerns!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.