Coordinating Imagination and Action: A Coupled Noise Prior for World Action Models
Abstract
World Action Models (WAMs) combine visual imagination with action generation for robot control, yet imagined futures may not always align with the outcomes produced by generated actions. In our empirical analysis, failed rollouts exhibit larger discrepancies between imagined futures and actual execution than successful ones, highlighting the importance of imagination–execution consistency. Prior work has shown that stochastic initialization enables diffusion- and flow-based generative policies to express diverse plausible behaviors under the same conditioning, suggesting that source noise implicitly specifies choices among behavioral modes. In WAMs, such stochastic choices exist in both video imagination and action generation, but their source noises are commonly initialized independently. This leaves the two choices uncoordinated at initialization and requires subsequent cross-modal interaction to establish correspondence, potentially resulting in individually plausible yet incompatible imagination and action. To address this issue, we introduce Action-Led Imagination via Gaussian Noise Coupling (ALIGN), an action-anchored coupled Gaussian prior that coordinates imagination and action from the source of stochastic generation. ALIGN learns where to couple through a task-relevant stochastic subspace and how strongly to couple through a learnable coupling strength, providing a shared stochastic reference while preserving the original Gaussian marginals of both modalities. Across two representative WAM paradigms, ALIGN yields absolute success-rate gains of 3.4% and 1.4% for Fast-WAM-IDM and OpenWAM-Joint on LIBERO-Plus, respectively, and 2.7% and 1.1% on RoboCasa-GR1. Further analyses show improved imagination–execution consistency while maintaining behavioral diversity, supporting stochastic coordination between imagination and action as an effective design principle for WAMs.
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.