acceptodds
Under review as a conference paper at ICLR 2027

Beyond Future Prediction: Denoising as Generative Adaptation for Robot Control

Abstract

Pretrained generative Diffusion Transformers (DiTs) capture rich pixel-level visual and language-conditioned structure through large-scale image and video generation training. A growing line of robot policies builds on this generative prior through generative visual and action co-training, commonly instantiated as future visual prediction, yet recent methods increasingly move this prediction out of the inference and keep it only for co-training. This shift leaves open a more basic question, what a pretrained generative DiT actually contributes to action learning, and how this prior should be adapted for control. We introduce NowWAM, a future-target-free co-training formulation that denoises the current observation and predicts robot actions from the same visual stream, directly coupling the native generative objective to the action-facing representation across the denoising trajectory. Under matched controlled settings, past and future visual targets perform comparably, whereas restricting training to the clean endpoint substantially reduces robustness. This suggests that a separate future target is not essential for generative adaptation, but the continuum of denoising states remains an effective interface for control. On LIBERO-Plus, NowWAM reaches 87.7% with FLUX2-Klein, improving over the future-target co-training baseline by 6.1 points while halving training visual tokens (784392) and reducing step time from 2.85 s to 1.63 s, a 1.8 speedup. With the pure text-to-image Z-Image backbone, NowWAM still reaches 87.8%, confirming that strong control adaptation does not depend on video generation or image-editing backbones.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Generative robot control, Diffusion Transformers

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_tYbcOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates an interesting question in generative robot policy learning: whether future visual prediction is necessary for effectively adapting pretrained generative Diffusion Transformers (DiTs) to robot control. The authors propose NowWAM, a future-target-free co-training framework that applies denoising supervision directly to the current visual stream used for action prediction. Rather than maintaining a separate auxiliary visual target, NowWAM couples generative denoising and action learning through the same visual representation. During inference, the policy operates on clean observations without additional visual generation. The paper reports promising results on LIBERO-Plus and RoboCasa, along with improvements in training efficiency. The authors also conduct controlled experiments investigating pretrained initialization, temporal target selection, and the role of the denoising trajectory. Overall, I find this work interesting and promising, particularly in its attempt to reconsider the role of future prediction in generative robot policy learning.

Critical strengths

1. An interesting and meaningful research question.

I appreciate the paper's attempt to revisit the necessity of future prediction in generative robot policies. As recent approaches increasingly move future prediction out of the inference process and retain it primarily as an auxiliary training objective, understanding what actually contributes to effective generative adaptation is an important question.

The controlled comparison between past and future visual targets is particularly interesting. The observation that past targets achieve performance comparable to future targets suggests that forward temporal prediction is not necessarily required for effective generative co-training in the studied setting.

I believe this finding provides a useful perspective on how pretrained generative representations can be adapted to downstream robot control.

2. A simple and effective method with practical efficiency benefits.

The proposed NowWAM framework is conceptually simple and well motivated. Applying denoising supervision directly to the action-facing current visual stream creates a more direct connection between representation adaptation and action learning, without introducing unnecessary architectural complexity.

In particular, the same noisy current visual representation serves both denoising supervision and action prediction during training, while inference uses the clean observation. This design removes the separate auxiliary visual target and reduces the computational overhead associated with processing additional visual tokens.

I appreciate that the method achieves improvements in both robustness and training efficiency while maintaining a relatively simple formulation.

3. Encouraging experimental results and informative ablations.

The paper provides promising results across multiple benchmarks. In particular, NowWAM improves the success rate from 81.6% to 87.7% over the future-target baseline on LIBERO-Plus, demonstrating encouraging robustness under distribution shifts. The results on RoboCasa further support the effectiveness of the proposed approach in a different manipulation setting.

The evaluation using the pure text-to-image Z-Image backbone is also valuable, suggesting that the proposed formulation is not restricted to image-editing or video-generation backbones.

I particularly appreciate the controlled experiments in Table 3. Table 3(a) demonstrates the substantial benefits of generative pretraining while also showing that denoising-based visual supervision provides improvements under random initialization. Table 3(b) investigates the role of temporal target selection, and Table 3(c) provides evidence that both noisy-current training and visual denoising supervision contribute to the effectiveness of NowWAM.

These analyses provide useful insights beyond benchmark improvements and help support the paper's main empirical findings.

Overall, I have a positive impression of this paper.

I appreciate the authors' effort to revisit an important assumption in generative robot policy learning. The paper raises a meaningful research question and provides an elegant and practical approach supported by encouraging experimental results.

In particular, I find the observation that future-target supervision is not necessarily required for effective generative co-training interesting. This provides a useful direction for reconsidering how pretrained generative models should be adapted to robot control.

The empirical findings are valuable. However, further disentangling the contributions of pretrained generative representations and downstream denoising-based self-supervised learning would help provide a more precise understanding of the underlying mechanism and strengthen the conceptual contribution of the paper. In particular, understanding the additional role of coupling denoising and action prediction within the same visual stream could offer useful insights for future research.

Additionally, a clearer discussion of the scope of the conclusions regarding future prediction, especially in relation to approaches that explicitly use future representations for action prediction, would help better contextualize the findings.

I view these points primarily as opportunities to strengthen the analysis and clarify the paper's contributions, rather than fundamental limitations that undermine its empirical effectiveness.

Overall, I believe this is a meaningful and promising exploration, and I lean toward acceptance.

Critical weaknesses

1. An Alternative Perspective: Denoising-Based Self-Supervised Representation Learning

While the paper frames NowWAM primarily as a form of generative adaptation, I wonder whether denoising-based self-supervised representation learning might provide a more direct and general interpretation of its effectiveness.

Prior work, such as Deconstructing Denoising Diffusion Models for Self-Supervised Learning (Chen et al., ICLR 2025), has shown that denoising objectives can learn effective visual representations without necessarily relying on the full generative modeling framework. From this perspective, the benefits of NowWAM may arise not only from adapting a pretrained generative model, but also from the more general representation-learning and regularization effects of denoising supervision.

Interestingly, the paper's own ablation results seem consistent with this interpretation. Table 3(a) shows that visual denoising supervision provides additional benefits under both random and pretrained initialization, while Table 3(b) suggests that forward temporal prediction is not essential for effective generative co-training. Furthermore, Table 3(c) demonstrates that denoising supervision provides benefits beyond noisy-current training alone. Taken together, these observations suggest that denoising-based self-supervised learning may play an important role in improving the representations used for downstream control.

I believe this perspective could complement the authors' current interpretation and potentially broaden the significance of the findings. It would be valuable for the authors to discuss more explicitly how NowWAM relates to denoising-based self-supervised representation learning, and whether this perspective might help explain the observed robustness improvements. Such a discussion could provide a clearer understanding of why the proposed approach works and further strengthen the conceptual contribution of the paper.

2. The scope of the conclusions regarding future prediction could be discussed more explicitly.

The finding that past and future visual targets yield comparable performance is interesting and supports the authors' argument that forward temporal prediction is not necessarily required as an auxiliary generative co-training objective.

However, I think it is important to distinguish between future prediction as an auxiliary training objective and future representations as direct inputs to action prediction.

In the setting studied by NowWAM, the auxiliary future visual target primarily serves as generative supervision, while the action expert does not directly consume predicted future representations at inference time.

In contrast, recent studies such as Simple-WAM suggest that explicitly incorporating future representations into the action prediction process can provide important benefits for certain out-of-distribution generalization settings, particularly task-level generalization.

I do not view these findings as contradictory. Rather, they suggest that future modeling may serve different roles in robot policy learning: as an auxiliary objective for representation adaptation and as a source of predictive information for action selection.

I would encourage the authors to clarify this distinction and discuss the scope of their conclusions more explicitly. In particular, the finding that future-target supervision is not necessary for effective generative co-training should not be interpreted as implying that future representations are generally unnecessary for robot control.

I believe such a discussion would help better position NowWAM relative to other world-action modeling approaches and make its main contribution more precise.

Minor Comment

The experimental results are encouraging, but the paper does not report variability across multiple training seeds. Providing additional stability analysis, particularly for the controlled ablations where some differences are relatively small, would further strengthen confidence in the empirical findings.

Questions for authors
  1. How do the authors interpret the relationship between generative adaptation and denoising-based self-supervised representation learning? To what extent could the improvements of NowWAM be explained by general representation-learning or regularization effects of denoising supervision?

  2. Table 3(a) already demonstrates the benefits of denoising-based visual supervision under both random and pretrained initialization, while Table 3(c) establishes the importance of combining noisy-current training with visual denoising supervision. Could the authors further clarify what additional benefits arise specifically from coupling denoising and action prediction within the same visual stream, rather than using denoising as a separate auxiliary visual objective?

  3. How broadly do the authors expect their conclusions regarding the necessity of future prediction to generalize, particularly to settings where predicted future representations are explicitly used for action selection, task generalization, or long-horizon planning?

Reviewer_AUkQOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Authors propose an variation of ImageWAM style models for robot control, by denoising the current observation to obtain features used for action prediction. Inference time uses the clean observation in a single forward pass. This allows the underlying image DiT model to be simply a T2I model instead of an I2I model. This single image input that drops the conditional image input improves training efficiency. Experiments across three simulated benchmarks are used to validate improvements from proposed method.

Critical strengths
  1. Clever, simple modification to achieve training efficiency.
  2. Strong evaluations and extensive ablations on simulated benchmarks.
  3. Interesting attention visualization.
Critical weaknesses
  1. No real world experiments - this questions generality of proposed method to real world data. Perhaps this denoising only objective works due to less data diversity in the simulated benchmarks. Existence of higher variability and high frequency noise in real world data could lead to different results.
  2. No reporting of inference latency.
Questions for authors
  1. Does the model improve inference speed as well? How would the accuracy-speed pareto frontier look like?
1 comment

Thank you for the new real world experiments and latency measurements.

Many important details on these experiments (what training data was used, how data was collected, exact visualizations or details of all 3 tasks evaluated, what was the task-specific success criteria) are still missing.

Also the paper needs visualization of more rollouts across all 4 tasks for real world, especially since these real world experiments were only just introduced during rebuttal.

Retaining the weak reject rating for above reasons.

Reviewer_J1DzOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces NowWAM, a method for adapting pretrained generative models to robot control through joint training on current-observation denoising and action prediction. Unlike prior approaches, NowWAM removes the future-image prediction objective and applies denoising directly to the current visual representation, which is used for action prediction. Experiments on robot manipulation benchmarks show that NowWAM reduces training costs and improves robustness while maintaining competitive in-distribution performance.

Critical strengths
  1. Novel perspective. The paper suggests a new perspective on the common prior framework that future prediction is useful for generalization and action learning. The controlled comparison of future and past prediction targets provides intuitive evidence supporting this perspective.

  2. Simple and effective method. NowWAM provides a simpler and more computationally efficient approach to train a world-action model, improving robustness while maintaining competitive in-distribution performance.

  3. Clear presentation. The paper is well written and easy to follow, with a clear explanation of the motivation and proposed method.

Critical weaknesses
  1. Limited explanation about current observation denoising. While the experiments suggest that future prediction is not essential for effective action learning, the paper provides limited explanation of why current-observation denoising is beneficial and why future targets offer no clear advantage. Further analysis would help to understand the paper.
Questions for authors
  1. Future image prediction from prior approaches could improve semantic generalization to novel language instructions. Beyond the language perturbations evaluated in LIBERO-Plus, do the authors expect NowWAM to retain its advantage when performing entirely new tasks?

  2. What motivates shifting the noise-level distribution toward lower noise levels, and why do the authors think this performs better than uniform sampling?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.