acceptodds
Withdrawn from ICLR 2027

Coordinating Imagination and Action: A Coupled Noise Prior for World Action Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

World Action Models (WAMs) combine visual imagination with action generation for robot control, yet imagined futures may not always align with the outcomes produced by generated actions. In our empirical analysis, failed rollouts exhibit larger discrepancies between imagined futures and actual execution than successful ones, highlighting the importance of imagination–execution consistency. Prior work has shown that stochastic initialization enables diffusion- and flow-based generative policies to express diverse plausible behaviors under the same conditioning, suggesting that source noise implicitly specifies choices among behavioral modes. In WAMs, such stochastic choices exist in both video imagination and action generation, but their source noises are commonly initialized independently. This leaves the two choices uncoordinated at initialization and requires subsequent cross-modal interaction to establish correspondence, potentially resulting in individually plausible yet incompatible imagination and action. To address this issue, we introduce Action-Led Imagination via Gaussian Noise Coupling (ALIGN), an action-anchored coupled Gaussian prior that coordinates imagination and action from the source of stochastic generation. ALIGN learns where to couple through a task-relevant stochastic subspace and how strongly to couple through a learnable coupling strength, providing a shared stochastic reference while preserving the original Gaussian marginals of both modalities. Across two representative WAM paradigms, ALIGN yields absolute success-rate gains of 3.4% and 1.4% for Fast-WAM-IDM and OpenWAM-Joint on LIBERO-Plus, respectively, and 2.7% and 1.1% on RoboCasa-GR1. Further analyses show improved imagination–execution consistency while maintaining behavioral diversity, supporting stochastic coordination between imagination and action as an effective design principle for WAMs.

Primary area:
applications to robotics, autonomy, planning
Keywords:
World Models, Embodied AI, Robotics

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_5z8fOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes ALIGN, a method that correlates the initial noise used for video and action generation in World Action Models. It learns a coupling subspace and strength while preserving the Gaussian distribution of each modality’s noise. Experiments on two WAM backbones show improved task success and agreement between predicted videos and actual execution.

Critical strengths

s1. The change is simple and keeps the original model architecture and generation process.

s2. The method improves both tested WAMs, with evaluations in simulation and on real robots.

Critical weaknesses

w1. The experiments show higher success rates and better video–execution agreement, but do not clearly establish that the gains come from coordinating the behavioral choices of the two branches. More direct evidence is needed to support this central claim.

w2. The appendix states that action-referenced and video-referenced sampling produce the same joint distribution for the same coupling parameters. It is therefore unclear why their performance difference supports the claim that using actions as the anchor is better.

w3. The final simulation gains are small, especially for the joint model. Results from multiple independent training runs are needed to show that these gains are reliable.

Questions for authors

q1. If we compare only successful rollouts, or only failed rollouts, does ALIGN still improve video–execution agreement?

q2. What differs between the action-referenced and video-referenced experiments, given their distributional equivalence at the same coupling parameters?

q3. Would a fixed random coupling achieve similar gains? How much does learning the coupling itself help?

q4. Do the gains hold across training seeds and after longer training?

Reviewer_gRUYOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes ALIGN, a learning-based noise-coupling technique for World Action Models i.e., models designed to generate future visual sequences and robot actions. In this approach, action noise is linked to a learned orthonormal subspace of the visual noise. A learnable intensity parameter, ρ, modulates the strength of this coupling, while a mutual-information penalty prevents the coupling from becoming excessively strong. Each modality maintains a standard Gaussian marginal distribution. ALIGN improves the success rates of Fast-WAM-IDM and OpenWAM-Joint across LIBERO-Plus, RoboCasa-GR1, and three real-world task environments.

Critical strengths
  1. A simple and robust mechanism.

ALIGN introduces cross-modal dependencies while preserving standard Gaussian marginal distributions. It accommodates various video and action dimensions without altering the generation schedule of the backbone.

  1. Consistent performance improvements.

Both backbones show enhanced performance on LIBERO-Plus (+3.4/+1.4 points) and RoboCasa-GR1 (+2.7/+1.1 points). These gains are maintained across all 7 perturbation categories and 3 real-world tasks in LIBERO-Plus.

Critical weaknesses
  1. The ablation experiment results do not demonstrate that learned couplings or action-driven couplings are significant.

Table 5 is interpreted as evidence that action-based noise serves as a better anchor than video-based noise. However, to the best of my understanding, when PP and ρ\rho are aligned, the action-based reference (Equation 9), the video-based reference (Equation 14), and the shared-source configuration all yield the same joint Gaussian distribution, sharing identical marginal distributions and a cross-covariance of ρP\rho P. This is also noted in Appendix C.2.

Consequently, the differences observed in Table 5 must stem from other factors, such as initialization, optimization, or random seed variance - none of which are addressed in the paper. Furthermore, two baselines are missing: one using a fixed, random orthonormal basis with the same learned ρ\rho, and another using a fixed ρ\rho tuned to a value between 0 and 1. The only fixed-strength baseline employed uses ρ=1\rho = 1, which represents a degenerate case where the joint distribution becomes singular. Outperforming this specific baseline provides little insight into whether learning ρ\rho is actually beneficial.

  1. It is difficult to assess the consistency of the evidence or the magnitude of the gains.

Figures 1 through 3 illustrate the consistency between imagination and execution without specifying the feature space, distance metrics, temporal alignment, normalization, or sample sizes.

While the correlation between baseline discrepancies and the magnitude of gains is indicative, it does not guarantee causality. The primary results lack information regarding random seeds or error bars. This is particularly significant given the reported gains of 1.1–1.4 points for OpenWAM-Joint; in some practical scenarios, this translates to only 3–4 additional successful trials out of 50 attempts. Finally, the diversity analysis revealed a similar average distance between successful trajectories; however, this does not imply that distinct behavioral patterns are preserved.

Questions for authors
  1. The three configurations in Table 5 are distributionally identical; what, then, accounts for the observed differences? Are these differences reproducible regardless of the random seed used?

  2. How does ALIGN compare to using fixed random basis functions with a learned ρ\rho, or to using a fixed ρ\rho tuned within the range (0,1)(0, 1)?

  3. How is consistency measured, and are the gains maintained when starting from identical initial states?

  4. What were the key settings (embedding dimension, ρ\rho initialization, regularization weights, and the learned ρ\rho values), and what was the additional computational cost?

Reviewer_sMxjOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses the inconsistency between imagined futures and generated actions in World Action Models (WAMs) by proposing ALIGN. The method introduces learnable Gaussian noise coupling to establish shared stochastic information between video and action generation, while preserving standard Gaussian marginal distributions for each modality. The authors validate the approach on two WAM architectures across both simulation and real-robot tasks. The results show that ALIGN improves task success rates and imagination–execution consistency to some extent.

Critical strengths
  1. Simple and easy-to-integrate method. ALIGN does not require modifying the original generation architecture. It achieves cross-modal coordination through source noise coupling while mathematically preserving the original Gaussian marginal distributions.
  2. Relatively comprehensive experiments. The method achieves consistent performance improvements across two WAM paradigms and multiple simulation and real-robot tasks, suggesting a degree of generality.
Critical weaknesses
  1. Limited novelty. The inconsistency between imagined futures and generated actions has been studied in several prior works. The main contribution of this paper lies in applying Gaussian noise coupling to WAM initialization. However, this statistical construction is not fundamentally new, and the paper lacks thorough comparisons with existing consistency modeling approaches. As a result, the overall technical contribution appears incremental.
  2. Insufficient validation of the core mechanism. Statistical correlation between noise variables does not necessarily imply behavioral or semantic consistency in the generated outputs. The current experiments mainly report improvements in final performance and consistency metrics, which are insufficient to demonstrate that the gains arise from better coordination of behavioral patterns rather than from additional regularization or optimization effects. Moreover, some improvements in success rate are small, and the paper does not provide adequate statistical significance analysis.
Questions for authors
  1. How can you demonstrate that noise coupling truly promotes alignment at the behavioral-semantic level between the two branches, rather than merely introducing statistical correlation? Could this be verified through controlled experiments that fix or swap the source noise?
  2. Compared with directly imposing video–action consistency constraints or adopting existing cross-modal alignment methods, what unique advantages does source noise coupling offer? Could you provide fair experimental comparisons?
  3. Could you report variance or confidence intervals across multiple random seeds, and compare the proposed method with a random orthogonal projection matrix? This would help assess the stability of the observed performance gains and the necessity of learning the coupling subspace.
Reviewer_uGX7OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes ALIGN, a noise coupling method for World Action Models (WAMs). The authors argue that independently sampled video and action noises may lead to inconsistencies between imagined futures and generated actions. ALIGN introduces a learnable Gaussian coupling between the two noise sources while preserving their marginal distributions. The method is evaluated on two WAM architectures, Fast-WAM-IDM and OpenWAM-Joint. Experiments on LIBERO-Plus, RoboCasa-GR1, and real-world manipulation tasks show improvements in task success rates and imagination–execution consistency.

Critical strengths
  1. Simple and well-formulated method. ALIGN introduces a learnable noise coupling mechanism while preserving the Gaussian marginal distributions of both modalities. The method is mathematically well-defined and can be integrated into existing WAM architectures.

  2. Reasonable and comprehensive experimental evaluation. The paper evaluates ALIGN across two WAM architectures, multiple simulation benchmarks, and real-world manipulation tasks. It also includes relevant ablation studies and analyses to support the proposed method.

Critical weaknesses
  1. Limited practical significance and unclear scalability. WAMs already coordinate video and action generation through cross-modal interactions such as cross-attention and video conditioning. ALIGN introduces an additional connection at the noise level, but its success-rate improvements on LIBERO-Plus (+3.4% / +1.4%) and RoboCasa-GR1 (+2.7% / +1.1%) are relatively modest. As WAMs move toward large-scale pre-training, it remains unclear whether this additional coupling is necessary or would continue to provide meaningful benefits when stronger cross-modal interactions can be learned from larger datasets.
Questions for authors
  1. How do the authors expect ALIGN to perform as WAMs scale up in model size and training data? Would noise coupling remain beneficial as cross-modal interactions become better learned? I am not requesting additional experiments, but suggest discussing this question in the Introduction and Conclusion to better clarify the potential long-term relevance of the proposed method.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.