acceptodds
Withdrawn from ICLR 2027

Next-WAM: Any-to-any World Action Model for Robotic Manipulation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

10 views

Abstract

Vision-Language-Action (VLA) models and World-Action Models (WAMs) have advanced robot manipulation by leveraging multimodal understanding and predictive modeling, respectively. Nevertheless, both paradigms struggle to jointly reason over task intent, forecast visual outcomes, and output precise actions. In this work, we introduce Next-WAM, an any-to-any World-Action Model that unifies these capabilities within a single multimodal framework. Built upon a Mixture-of-Transformers (MoT) architecture, our model consists of understanding, generation, and action experts with disjoint parameters and structured joint attention. It fuses modality-specific encodings while enabling cross-modal information exchange. Powered by this design, Next-WAM consumes visual history, produces subtask text as control guidance, predicts future observations, and outputs continuous action chunks. Notably, by masking future-image latents from the action attention stream, Next-WAM leverages visual prediction to supervise control representations, without needing to generate future images at action inference time. Trained without extra embodied pretraining, Next-WAM attains 98.4% success on LIBERO and 70.5% on LIBERO-Plus, outperforming Fast-WAM by 0.8% and 19.0%, respectively. Real-robot experiments further validate Next-WAM’s capacity to reason from task instructions for manipulation. Code will be made public.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Embodied AI

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_oc4COpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents Next-WAM, an any-to-any world-action model for robotic manipulation that unifies language reasoning, future visual prediction, and continuous action generation. The model uses three Transformer experts for understanding, generation, and action, with structured joint attention for cross-modal information exchange. A key design choice is to use future-image prediction as training supervision while masking future-image latents from the action stream, so future images do not need to be generated during policy inference. The model also introduces Reason/Act control tokens to support online subtask reasoning. Experiments are conducted on LIBERO, LIBERO-Plus, and two real-robot reasoning tasks.

Critical strengths

The overall design is clean and practical. In particular, using future visual prediction as training supervision without requiring future generation during action inference is a sensible choice. The comparison against the IDM-style variant also provides some evidence that explicit future generation is not necessary for action prediction.

The real-robot tasks are reasonably well designed for testing reasoning. Conditional object grasping and Coke restocking test logical instruction following and task-progress tracking rather than only low-level manipulation success.

Critical weaknesses
  1. The main architectural claim is not sufficiently isolated experimentally. The paper motivates the three-expert MoT design as a way to reduce interference between language reasoning, visual prediction, and action learning, but does not compare it against a similarly sized shared-backbone model or other parameter-sharing variants. As a result, it is difficult to attribute the observed gains specifically to expert separation.

  2. The LIBERO-Plus comparison is not sufficiently comprehensive with respect to stronger recent baselines. The reported 70.5% result is strong relative to the selected methods in Table 3, but the comparison does not reflect the strongest currently reported results on LIBERO-Plus. This makes the robustness claim less convincing and makes it harder to judge the method against the current state of the art.

  3. The contributions of future prediction and explicit text reasoning are not clearly disentangled. The paper reduces the image loss weight, but does not show a complete removal of future-image supervision, nor an ablation without explicit text reasoning. These experiments are important because the paper’s main contribution is the combination of reasoning, world prediction, and action generation.

Questions for authors
  1. Can the authors provide a comparison between the proposed three-expert architecture and a shared Transformer with a comparable parameter count? This would help verify whether expert separation itself is responsible for the improvement.

  2. Can the authors clarify the selection of LIBERO-Plus baselines and compare against stronger recent methods, or explain why those results are not directly comparable?

  3. What is the performance when the future-image loss is completely removed rather than only down-weighted? Similarly, what happens when explicit text reasoning is removed? These ablations would help separate the contribution of each component.

  4. The paper shows that conditioning action prediction on generated futures slightly hurts LIBERO performance. Does the same trend hold on the real-robot tasks, especially the longer-horizon Coke-restocking task?

Reviewer_AJTkOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

Next-WAM combines understanding, visual generation, and action experts through masked joint attention. It uses future-image supervision during training and persistent generated subtasks for reasoning-dependent real-robot tasks, with evaluation on LIBERO, LIBERO-Plus, and two real-robot task families.

Critical strengths
  1. The model achieves competitive LIBERO performance and improved robustness over the reported Fast-WAM baseline.

  2. Persistent subtask text provides an explicit mechanism for tracking task progress, and the inference-acceleration study documents useful implementation details.

Critical weaknesses
  1. Lance already supports unified multimodal understanding and generation, yet the ablations do not separate inherited capabilities from the benefits of expert separation, future-image supervision, and language prediction. The authors should add controlled ablations of these components, particularly comparing models with and without generated subtasks under matched supervision and training settings, to establish the source of the reported gains.

  2. The real-robot comparison confounds architecture with additional supervision and memory. Next-WAM uses annotated subtasks and persistent text state, but equivalent supervision and task-state mechanisms are not established for the baselines. The results therefore do not isolate the contribution of world modeling or the three-expert architecture to the reasoning gains.

  3. The distinction from closely related unified models remains unclear. Motus already combines understanding, video-generation, and action experts through joint attention, while BagelVLA integrates textual planning, visual forecasting, and action generation. The paper does not sufficiently establish what its attention design and persistent subtask mechanism contribute beyond these existing formulations.

  4. The empirical case for the proposed integration is incomplete. Next-WAM reports 70.5% on LIBERO-Plus, compared with ImageWAM’s reported 71.2–83.1% across backbones. Although these are not controlled comparisons, the results motivate a clearer account of what the additional language-generation capability contributes and under which evaluation settings it is actually used.

Questions for authors
  1. What are the annotation source, quantity, and training use of the real-robot subtask labels, including for the baselines?

  2. What evidence isolates future-image supervision and expert separation under the same pretrained backbone?

  3. Does the reported latency include control-token selection and autoregressive subtask generation when reasoning is triggered?

Other suggestions (not decisive for my recommendation): Correct or explain the missing negative sign in the text loss in Eq. 10; reconcile the visual-history motivation with Table 4, where historical inputs reduce average success; distinguish the Appendix D profiling configuration from the policy-evaluation configurations.

Reviewer_BVcLOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents Next-WAM, which adds an action expert to the understanding and generation modules of pretrained Lance-3B. Structured masked joint attention connects language understanding, future-image prediction, and continuous action generation. Future-image prediction provides auxiliary supervision during training, while action tokens are prevented from attending to future-image tokens, allowing action inference without generating future images. For real-robot tasks with text annotations, the model uses Reason/Act control tokens to dynamically generate and retain subtask text for closed-loop execution. Experiments cover LIBERO, LIBERO-Plus, and two real-robot task families, Conditional Object Grasping and Coke Restocking.

Critical strengths
  1. Strong manipulation performance and robustness without additional embodied pretraining. In Table 2, Next-WAM achieves a 98.4% success rate on LIBERO, compared with 97.6% for Fast-WAM. On LIBERO-Plus, Table 3 reports an average success rate of 70.5%, compared with 51.5% for Fast-WAM and 69.6% for OpenVLA-OFT. Next-WAM improves over Fast-WAM across all seven perturbation categories.
  2. Subtask text generation is integrated with continuous control and shows practical benefits in real-robot deployment. The real-robot tasks require conditional target selection and stopping once a target count is reached, directly testing semantic guidance for control. Next-WAM achieves success rates of 70% on Conditional Object Grasping and approximately 63% on Coke Restocking, substantially outperforming the two evaluated baselines. Appendix D also reports concrete inference optimizations and a latency of 168.29 ms for 10-step action inference on an RTX 4090.
Critical weaknesses
  1. The training-objective ablations do not sufficiently establish the contributions of subtask text generation and future-image prediction to control performance. Next-WAM adds an action expert to Lance’s understanding and generation experts. Although Appendix B examines changes in the image loss weight, it does not include a comparison that fully removes future-image prediction supervision. The real-robot experiments also lack a comparison without subtask text generation and conditioning. Under the same backbone and training settings, evaluating future-image supervision on LIBERO or LIBERO-Plus and the text-based subtask mechanism on real-robot tasks with text annotations would help establish their respective contributions.
  2. The description of text supervision and the related baseline comparisons are insufficient in the real-robot experiments. The paper states that text supervision is applied only to samples with output-text annotations, but does not explain how these annotations are produced, how subtasks are defined, or how they are aligned with trajectories. The real-robot experiments compare against π0.5 and Fast-WAM, but do not clarify whether or how these baselines use the corresponding subtask information. They also lack matched comparisons that account for subtask generation or persistent task state.
Questions for authors
  1. What are the main sources of Next-WAM’s performance gains? Could the authors explain more clearly how subtask text generation, future-image prediction, and visual history encoding each help control, and how their roles relate to or differ from memory and subtask reasoning?
  2. Figure 5 shows generated subtask text alongside real-robot execution, while Figure 6 shows future-image predictions. Could the authors provide aligned examples of subtask text, predicted future observations, and actual action execution for the same input? These examples would help assess whether the outputs agree on target objects, subtasks, and state changes, including cases where inconsistencies occur.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.