acceptodds
Withdrawn from ICLR 2027

Native Action-Prior Learning from Videos for World Action Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

World action models integrate future visual dynamics with robot action prediction, but their scalability remains limited by the need for action-annotated robot trajectories. Observation-only videos contain rich evidence about interaction dynamics, but existing approaches typically use them either to pretrain visual representations that must later be adapted for control, or to infer latent actions that are subsequently grounded to robot commands. We present NAVA-WAM, which introduces native action-prior learning by directly pre-training the action policy from observation-only videos, avoiding indirect representation-to-control transfer or a separate latent-action model. Our training consists of two stages. First, we pretrain on observation-only videos, where future-video flow-matching supervision over visual transitions is propagated through transition-structured joint attention to optimize the Action-DiT and learn action-relevant priors. Second, we use action-labeled demonstrations to post-train the Action-DiT for robot control through joint video-action flow matching, while asymmetric attention decouples the visual branch from iterative action denoising and enables efficient action-only inference. Extensive experiments show that NAVA-WAM consistently outperforms prior approaches under both in-distribution and out-of-distribution settings, while demonstrating strong action-label efficiency and effective real-robot generalization. These results establish native action-prior learning as an effective approach to directly pretrain action policies from observation-only videos, providing a scalable path beyond action-labeled robot data.

Primary area:
applications to robotics, autonomy, planning
Keywords:
World Action Model

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_bzifOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents NAVA-WAM, which introduces native action-prior learning by directly pretraining the action policy from observation-only videos. NAVA-WAM first pretrains on observation-only videos, where future-video flow-matching supervision over visual transitions is propagated through transition-structured joint attention to optimize the Action-DiT and learn action-relevant priors. Then, NAVA-WAM uses action-labeled demonstrations to post-train the Action-DiT for robot control through joint video–action flow matching. Experiments cover LIBERO, LIBERO-Plus, RoboTwin 2.0, and real-robot results.

Critical strengths
  1. The ablation studies demonstrate the effectiveness of the pretraining process. The experimental results show that pretraining improves performance and sample efficiency, confirming the effectiveness of the proposed pretraining method.
  2. The experimental evaluation covers a relatively broad range, including two simulation environments, LIBERO and RoboTwin 2.0, evaluation on LIBERO-Plus with perturbations, and three real-robot tasks. This broad task evaluation demonstrates the wide applicability of the proposed method.
Critical weaknesses
  1. In contrast to methods that learn visual representations or latent actions, this paper proposes a third approach: learning native action priors directly from observation-only videos. Although the paper cites UWM, it neither distinguishes its approach from UWM nor provides a comparison with it. UWM already integrates action and video diffusion processes within a unified Transformer architecture and supports pretraining on action-free videos. It therefore remains unclear what advantages NAVA-WAM’s approach of freezing the Video-DiT and optimizing only the Action-DiT through joint attention offers over UWM. The paper also lacks relevant ablation studies or variant experiments demonstrating the necessity of these design choices.

  2. When the Video-DiT is frozen and the Action-DiT is optimized through joint attention using only future-video flow-matching supervision, what information does the resulting native action prior actually encode, and why does it benefit downstream policy learning?

Questions for authors
  1. How does the third approach proposed by NAVA-WAM differ from UWM, and what evidence demonstrates the additional benefits of this distinction? Could the authors compare against UWM in settings where a fair comparison is feasible, and provide ablation studies or variant experiments demonstrating the necessity of design choices such as freezing the Video-DiT and optimizing the Action-DiT solely through joint attention?

  2. Could the authors clarify what knowledge is transferred to the Action-DiT by pretraining on observation-only videos through joint attention alone? What exactly does “native action prior” refer to? Furthermore, please explain why this prior facilitates downstream policy learning.

  3. Please specify what video data are used for the action-free target-domain cold start.

If the authors satisfactorily address these concerns, I would be willing to raise my score.

Reviewer_nBkZOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes NAVA-WAM, a method to use actionless videos to directly pretrain the action expert of a world action model. The main idea is to freeze a pretrained Video-DiT and train the Action-DiT through future video flow matching supervision, using joint attention between video and action tokens. The model is then finetuned on robot demonstrations with joint video-action flow matching. At inference, the video features are cached and only actions are iteratively denoised.

The authors evaluate on LIBERO, LIBERO-Plus, RoboTwin 2.0, and real robot manipulation. NAVA-WAM achieves strong results on RoboTwin Random (73.6%), although its own baselines without native action-prior pretraining already perform relatively well (69.7% without the proposed action-prior pretraining and 71.2% with latent-action pretraining).

Critical strengths
  1. Pretraining the Action-DiT directly from videos, without an explicit latent-action model, is an interesting direction. The method is relatively simple and fits naturally into the WAM architecture.
  2. The results are encouraging, particularly on RoboTwin Random and in the low-demonstration setting. The authors also include useful ablations on pretraining and make an effort to study the learned representations through action probing and motion-transfer experiments.
  3. I appreciate the real-robot evaluation. After post-training on DROID, the policy achieves better performance than DreamZero and π0.5\pi_{0.5} on three Franka tasks without task-specific finetuning.
Critical weaknesses
  1. My main concern is what the proposed pretraining is actually encouraging the Action-DiT to learn. The Video-DiT is already a pretrained video generator and has direct access to the previous and noisy future observations. Since only the Action-DiT is trained, it could simply learn to help the frozen video model denoise, rather than learn something specifically useful for action prediction. The action probing in Table 2b and motion-transfer results are encouraging, but don't quite distinguish these explanations. I would have liked an ablation that isolates the transition-structured attention, for example by preventing the Action-DiT from attending to the noisy future while still allowing it to contribute to video prediction.
  2. I found the pretraining comparisons in Table 2a somewhat difficult to assess. The authors state that all variants use the same downstream protocol, but give little detail on how the representation based and latent action based pretraining methods are implemented, or whether their pretraining data and compute are comparable. The improvements are meaningful, particularly with fewer demonstrations, but at 50 demonstrations the gain is 3.9 points over no proposed pretraining and 2.4 points over latent action pretraining. This is quite different from the 17.9-point gap to MV-WAM in Table 1. Also, Table 2a only reports Random-domain results. The corresponding Clean results would help understand whether the pretraining particularly improves OOD robustness or policy performance more generally.
  3. I also think the related work discussion could be improved. Prior video-action approaches such as UWM [1] and PAD [2], and latent action approaches such as villa-X [3] and ViPRA [4], are relevant to the motivation here. More recent works such as InternW0 [5] and ProWAM [6] also explore closely related architectural choices. I don't necessarily expect comparisons with concurrent works, but a better discussion of these connections would help clarify what is distinctive about the proposed approach, especially given the claims about limitations of explicit latent action learning.
Questions for authors
  1. How important is the Action-DiT's access to the noisy future observation during pretraining? Could the authors ablate this connection while keeping the Action-DiT trainable through the video prediction loss?
  2. Could the authors provide more details on how the representation based and latent action based baselines in Table 2a were implemented, particularly whether their pretraining data and compute budgets are comparable? It would also be useful to see the corresponding Clean-domain results to understand whether the improvements are specific to OOD generalization.

References

[1] Zhu et al. Unified World Models: Coupling Video and Action Diffusion for Pretraining on Large Robotic Datasets. RSS, 2025.

[2] Guo et al. Prediction with Action: Visual Policy Learning via Joint Denoising Process. NeurIPS, 2024.

[3] Chen et al. villa-X: Enhancing Latent Action Modeling in Vision-Language-Action Models. arXiv:2507.23682, 2025.

[4] Routray et al. ViPRA: Video Prediction for Robot Actions. ICLR, 2026.

[5] Cai et al. InternW0: A Foundational Physical World Model for Efficient Real-World Interactions. arXiv:2609.27656, 2026.

[6] Zhang et al. World Action Modeling with Progressive Visual Planning. arXiv:2610.02508, 2026.

Reviewer_Wpw2OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The papers introduce a novel method for using action-free pretraining videos to improve downstream WAM performance. They key idea is to jointly attend between the past and future video and action embeddings during pretraining, which optimizes the action head for the video prediction objective and implicitly supervises action prediction without action labels. The authors show that this achieves strong results across LIBERO, LIBERO-plus, and RoboTwin 2.0 as well as a real pick and place benchmark on the DROID platform. The authors show that the observation-only training consistently leads to a downstream performance benefit.

Critical strengths
  • The method seems novel and interesting. This is a creative way to supervise the action head during action-free pretraining without adding extra inductive biases like a latent bottleneck.
  • The authors address a very relevant and timely problem (action-free pretraining for robot foundation models)
  • The method section is clear and easy to follow
  • The results clearly show that their method consistently beats a baseline without action-free pretraining and that performance improves through scaling action-free pretraining
Critical weaknesses

Experimental Soundness

My biggest concern is that the experimental results do not fully justify the claims. The core claim is that, compared with other available action-free pretraining methods, theirs leads to better downstream policy performance. In order to substantiate this claim the core experiment authors need is a few key baselines trained under paired training conditions (compute budget, data mix, video model backbone, etc) and results showing that under these controlled conditions, the NAVA-WAM architecture leads to stronger downstream performance than these baselines. In my view, the key baselines would be

  • a basic WAM architecture such as mimic Video-style with paired compute budget, backbone, data, etc. There is already plenty of evidence that these benefit from large scale action-free pretraining so this is the most important comparison
  • Perhaps some other methods for latent actions, which the authors do seem to compare to (see note about the ablation study in the next section)

Without these carefully controlled ablation studies, it is impossible for me to tell whether the authors' method achieves the results it does because of the NAVA-WAM architectural contribution or because the authors have designed a better-engineered pretraining / post training pipeline. This is the key issue that motivates my score.

Other issues

  • I cannot tell what the ablation study is testing. The authors say they compare to "latent-based" and "representation-based" WAMs but do not specify what either of these architectures entail
  • There are several works which achieve slightly stronger performance than the authors on their simulated benchmarks. For example, the authors compare to ABot-M0 which they outperform, but do not compare to ABot-M0.5 which has been available since July 1 and outperforms them for LIBERO. Another study reports higher results than they do for π0.5\pi_{0.5} (https://arxiv.org/html/2603.22078v2). I do not personally think that competing to saturate relatively easy benchmarks like LIBERO is particularly useful and think it's fine for the authors to not achieve SOTA, but it would be good to include a more up-to-date comparison to SOTA on the simulated benchmarks
Questions for authors
  • Can you clarify precisely what your ablation studies test and if possible make an argument against my key complaint about the lack of paired ablations?
  • Can you confirm that the ablation experiments you ran carefully control for changes in pre- and post-train data mix, training budget, video backbone, etc?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.