acceptodds
Under review as a conference paper at ICLR 2027

Learning What Actions Change: Action Effect Matching for Dynamics-Faithful World Models

Abstract

World models aim to predict environment dynamics under actions, enabling the prediction of action consequences without executing them in the real world. Yet despite producing increasingly realistic videos, current world models often fail to capture how small action changes affect future observations. This limits their use in tasks such as robotic policy evaluation and model-based planning, where nearly identical actions can lead to qualitatively different outcomes. To address this, we propose AFFECT, an auxiliary training objective for learning the distribution of visual changes induced by action perturbations. Specifically, we inject temporally correlated noise into the actions of a policy trained on task-agnostic human teleoperation. We then train the world model so that its generated videos respond to this noise as the recorded videos do, without requiring repeated executions from the same initial state. Across OGBench simulation and real-world DROID and ALOHA-2 setups, AFFECT makes world-model rollouts more faithful to real outcomes. On DROID, the Pearson correlation between policy success rates in the world model and on the real robot rises from 0.47 to 0.79, alongside up to 23% lower FVD. These gains carry over to planning, where selecting actions with AFFECT-trained world models improves task success by up to %.

Primary area:
applications to robotics, autonomy, planning
Keywords:
world models, action-conditioned video prediction, robot manipulation, video generation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_eC4DOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

World models are usually good at generating plausible videos but often lack precise action following. The paper introduces AFFECT, an auxiliary training objective to force the model to pay attention to actions. The authors show improvement in world model generation on both simulated and real world rollouts. Further, they show the improved world model helps in better planning when selecting actions during policy rollouts.

Critical strengths
  1. The idea of training the model with the perturbed play data seems interesting so the world model would need to pay attention to the action rather than simply memorizing the behavior of the policy it is trained from.
  2. Authors show world modeling performance improved on both real and simulated environments across multiple embodiments.
  3. Gains in success rate from planning using world model is a good way to verify improved action fidelity.
Critical weaknesses
  1. The selected video features seem somewhat arbitrary. Can the authors explain why those features were selected and why not choose standard feature extractors like DINOv2?
  2. There are no qualitative results in the paper.
  3. Each embodiment was evaluated over 12 episodes per 4 checkpoints across 2 tasks which seems to be quite low.
Questions for authors
  1. Cosmos3 Pearson correlation (0.091) in Table 1 seems pretty low. Can the authors recheck that data point?
  2. The perturbed training data already forces the model to pay attention to the actions since the injected noise is random and can’t be predicted from the scene. Why isn’t that enough? Also, a good ablation could be to show difference between training model on pure play policy rollouts compared to perturbed rollouts.
  3. During AFFECT steps the model uses N=2 step denoising. The paper argued one step wasn’t enough but can they show some empirical/qualitative evidence that two step would work. Did they try the common alternative of denoising more steps but with gradients only enabled in the last step?
  4. Can the authors share more details about the training data - episodes, hours, teleop data used to train the play policy?
2 comments
AuthorsOpenReview ↗

Reponse to Reviewer eC4D (1/2)

Dear reviewer eC4D,

Thank you for your valuable comments and suggestions in reviewing our work. We address each of your questions and concerns individually as follows.


[W1] Choice of video features, and why not DINOv2?

We chose these features to capture the direction, magnitude and motion of the visual change caused by actions. The channel and projection features measure signed changes in the latent, the squared projection and energy features measure their magnitude, and the flow features measure temporal motion. All features are computed on the world model's own latent, so the gradient can pass through the two-step sampler without decoding the video to pixels or running a large vision encoder inside the training loop. We also keep the features fixed on purpose, since learnable features admit a trivial solution that makes the loss zero for any generator. As shown in Figure 4, these features respond 1.6-2.2× more strongly than the shuffled baseline in regions where the arm, cube and shadow move, while staying at chance level on static regions such as the table and backdrop. This shows that the features do capture where the action changes the scene.

Nevertheless, we evaluate a variant using DINOv2 features, where the generated video latents are decoded into images that are subsequently fed into the DINOv2 feature extractor, where the patch features are then projected with fixed random directions to generate features compatible with our method. This resulted in higher training cost with less of a performance gain compared to the base DreamDojo model.

MethodShort PSNR ↑Short LPIPS ↓Short FVD ↓Long PSNR ↑Long LPIPS ↓Long FVD ↓
DreamDojo26.030.068524.870.08108
w/ AFFECT26.780.057325.580.0793
w/ DINOv226.330.057924.320.08101

[W2] Qualitative results

Following your suggestion, we have added qualitative results in Appendix C of the revised manuscript, comparing real, baseline and AFFECT rollouts from the same initial state, following the same action trajectories in long-horizon open-loop execution.


[W3] Number of evaluation episodes

We agree that the number of evaluation episodes is limited, mainly due to the cost of evaluating real robots. However, AFFECT improves the correlation in all six backbone-embodiment combinations, which under a one-sided sign test, where 6 out of 6 combinations improved gives a statistically significant result. Also, given that result, the evidence for improved action fidelity does not rest on these correlations alone. In simulation, where repeated evaluation is affordable, AFFECT also improves action sensitivity over 270 controlled trials (response to Reviewer bju7, W2) and planning success over 400 rollouts per method with non-overlapping confidence intervals for Cosmos3 (response to Reviewer bju7, Q4).


[Q1] Cosmos3 Pearson correlation in Table 1

We rechecked the result and confirmed the value. Under the same training regime as other models, base Cosmos3 often lost track of the target object during manipulation, especially during long-horizon rollouts. As a result, base Cosmos3 rollouts rarely reached task success, and the imagined success rates of all checkpoints were close to zero. Because these near-constant predictions carry little information about the differences between checkpoints, the correlation with real success rates is dominated by noise, which explains the low value.


[Q2] Is perturbed data alone enough?

We believe the flow loss alone is not enough because most of the action can be predicted from the scene, since the play policy chooses its nominal actions by looking at it. Only the injected perturbation is random given the context, and the flow loss does not give it any special weight, as the loss is dominated by reconstructing the scene appearance. AFFECT directly supervises how the generated video should change with this perturbation. Our main table results, which compare AFFECT against world models trained on only perturbed data, show that our method is useful beyond just the perturbed data.

AuthorsOpenReview ↗

Reponse to Reviewer eC4D (2/2)

[Q3] Two-step denoising and alternatives

We use two steps because a single Euler step from pure noise returns the conditional mean E[Ot∣ot,At]\textbf{E}[O_t | o_t, A_t] at the optimum. This output has no sampling variance, so for squared features it omits the Var(Z∣ot,At)\text{Var}(Z | o_t, A_t) term and matches only mean effects (Appendix A.8). With N = 2, the second network call is evaluated at an intermediate latent that depends on the initial noise, so the output varies across noise draws and the magnitude features receive a nonzero variance term. N = 2 is still a coarse discretization that under-represents the variance of the full sampler. We chose it as the smallest N for which this term is present, since each additional step adds a forward and backward pass.

Nevertheless, to answer your question, we increase the N value during AFFECT training the UR5e setting, which showcases that increasing N value indeed gives a better discretization, and better video quality at the cost of higher training cost for each AFFECT training step

MethodShort PSNR ↑Short LPIPS ↓Short FVD ↓Long PSNR ↑Long LPIPS ↓Long FVD ↓
DreamDojo26.030.068524.870.08108
w/ AFFECT (N=2)26.780.057325.580.0793
w/ AFFECT (N=4)27.010.056626.320.0686
w/ AFFECT (N=8)27.130.046426.390.0683

[Q4] Training data details

We provide a brief overview of training data, and have added full details to Appendix A.2 of the revised manuscript. For the real-world FR3 and ALOHA-2 settings, we collect around 100 mins of teleoperation data to train the play policy, which is then rolled out under perturbations to generate around 5 hours of data each.

Reviewer_tuyrOpenReview ↗
Rating
3
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates how small action perturbations affect the predictions of robotic world models and proposes AFFECT, an auxiliary training objective that matches the statistical relationships between action perturbations and their induced visual changes. During data collection, temporally correlated noise is injected into actions generated by a play policy, and the resulting executions are recorded. The world model is then trained to reproduce the observed relationships between action variations and visual outcomes.

Critical strengths

I find the motivation compelling. In physical environments, even small action perturbations can lead to substantially different outcomes. The proposed evaluation protocol is also valuable, as it provides a task-relevant measure of how faithfully a world model captures action-conditioned dynamics.

Critical weaknesses

Although the paper demonstrates improvements in downstream planning success rates using DreamDojo, these experiments are not extended to Cosmos3, despite AFFECT being evaluated on both backbones for world-model fidelity. This limits the evidence for the generalizability of the downstream benefits. Additionally, the planning results are reported without confidence intervals or statistical significance analysis, making it difficult to assess the robustness of the observed improvements.

Questions for authors

Could the authors clarify whether the improvements in downstream task success also hold when using Cosmos3 and Cosmos3 + AFFECT for planning? Additionally, could the authors provide statistical uncertainty estimates for the success-rate improvements reported in Figure 3?

1 comment
AuthorsOpenReview ↗

Response to Reviewer tuyr

Dear reviewer tuyr,

Thank you for your valuable comments and positive feedback in reviewing our work. We address your questions and concerns as follows.

[W1, Q1, Q2] Planning with Cosmos3 and Uncertainty Estimates

Following your suggestion, we evaluate planning with the Cosmos3 backbone as well on the UR5e tasks. We use the same proposal-selection protocol as in Section 4.2 while also providing Wilson 95% confidence intervals below. To ensure statistical robustness, we expand the evaluation regime to cover both cube-single and cube-double tasks within OGBench, and run 10 seeds with 4 policy model checkpoints across 10 tasks, for a total of 400 rollouts for each baseline

Methodcube-singlecube-doubleAverage
Uniform Selection0.490 [0.422, 0.559]0.515 [0.446, 0.583]0.503 [0.454, 0.551]
Cosmos30.655 [0.587, 0.717]0.535 [0.466, 0.603]0.595 [0.546, 0.642]
Cosmos3 + AFFECT0.870 [0.816, 0.910]0.770 [0.707, 0.823]0.820 [0.779, 0.855]
DreamDojo0.615 [0.546, 0.680]0.565 [0.496, 0.632]0.590 [0.541, 0.637]
DreamDojo + AFFECT0.680 [0.612, 0.741]0.660 [0.592, 0.722]0.670 [0.622, 0.714]
Reviewer_bju7OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes AFFECT, an auxiliary training objective for improving action fidelity in robotic world models. The authors collect robot interaction data with perturbed actions and introduce a distribution-matching loss that encourages generated videos to capture the relationship between action perturbations and their visual consequences. The method is evaluated on simulated UR5e manipulation and real-world DROID and ALOHA-2 setups using DreamDojo and Cosmos3 backbones. Results show improvements in video prediction metrics, agreement between simulated and real policy success rates, and downstream planning performance.

Critical strengths
  1. The paper addresses an important limitation of current video-based world models: visually plausible predictions do not necessarily capture the actual consequences of robot actions.
  2. The evaluation is relatively comprehensive, covering multiple world-model backbones, simulated and real-world robot data, and downstream planning. The improvements in policy evaluation correlation and planning success are encouraging.
Critical weaknesses
  1. The proposed objective is essentially a form of feature or moment matching between action perturbations and visual outcomes. Although the formulation is reasonable, it is unclear how fundamentally different it is from existing distribution-matching or action-consistency objectives. The paper would benefit from stronger comparisons with simpler alternatives, such as direct action-effect regression or contrastive action-conditioned training.
  2. The main evaluation relies on video quality metrics and Pearson correlation between predicted and real policy success rates. While these metrics are useful, they do not directly measure whether the model predicts the correct physical consequences of individual action perturbations. For example, a model could improve aggregate success-rate correlation while still incorrectly predicting contact dynamics, object displacement, or grasp outcomes. More direct quantitative evaluations of action sensitivity would strengthen the claims.
  3. The proposed method relies on injecting temporally correlated noise into actions generated by a play policy. It is unclear whether the learned action effects generalize beyond this perturbation distribution, particularly to unseen tasks, contact-rich interactions, or substantially different action sequences. The experiments do not sufficiently establish whether the improvements extend beyond the action variations encountered during training.
Questions for authors
  1. How does AFFECT differ fundamentally from existing feature-matching, distribution-matching, or action-consistency objectives? Can the authors compare against simpler auxiliary losses?
  2. Can the authors directly evaluate whether predicted object motion, contact events, and grasp outcomes match real-world consequences under controlled action perturbations?
  3. How sensitive is AFFECT to the magnitude and temporal structure of the injected action noise? Does the method generalize to action variations outside the training perturbation distribution?
  4. How statistically significant are the improvements in downstream planning? Would the reported gains remain consistent across additional tasks and random seeds?
2 comments
AuthorsOpenReview ↗

Response to Reviewer bju7 (1/2)

Dear reviewer bju7,

Thank you for your valuable comments and suggestions in reviewing our work. We address each of your questions and concerns individually as follows.


[W1, Q1] Difference from feature-matching and action-consistency objectives

We clarify that we do formulate AFFECT as a form of feature matching in Section 3.3. However, we believe there is a meaningful difference from existing feature-matching or action-consistency objectives. Standard feature matching compares the average features of recorded and generated videos, while AFFECT compares how these features change with the injected perturbation. For exmaple, a model that moves an object in the opposite direction of the real one can still match all marginal features without regarding the perturbations, but this error is directly caught by the product with the perturbation in our method.

Nevertheless, following your suggestion, we compare AFFECT against simpler auxiliary losses that can be trained on the same data: an inverse-dynamics loss that reconstructs the action from the generated video, and a per-sample regression loss on the same frozen features without the perturbation products. All comparisons were done with the DreamDojo backbone on the UR5e simulation environment.

MethodShort PSNR ↑Short LPIPS ↓Short FVD ↓Long PSNR ↑Long LPIPS ↓Long FVD ↓
DreamDojo26.030.068524.870.08108
w/ AFFECT26.780.057325.580.0793
w/ IDM Loss26.120.068724.900.08106
w/ Regression25.960.079824.680.09131

We would also like to clarify that contrastive action-conditioned training is a baseline for a different setting. Such methods require tuples of different actions executed from the same initial state, so that the futures under different actions can be contrasted with each other. These tuples are not obtainable at scale on real robots, since the scene must be reset to the exact same state for every action. In contrast, AFFECT only requires a single rollout per window and matches the action effects across rollouts.


[W2, Q2] Direct evaluation of action sensitivity

We agree that the correlation of success rates is an aggregate measure that could overlook individual mismatches. We used it as our main metric since it directly reflects how world models are used for policy evaluation, following prior work [1, 2]. To address your concern, we additionally evaluate action sensitivity directly in the UR5e simulation within OGBench, where the environment can be reset to the same state.

Specifically, we sample 30 initial states and execute each of them under 9 perturbations from the training distribution, recording the ground-truth object poses and contacts. We then report the outcome agreement (grasp / miss) and object displacement error. Outcome agreement is for pre-grasp starting states, where given an action-condition, we calculate the accuracy of the predicted model’s outcome (either grasping success or miss) against the ground-truth video of simulated futures. For object displacement error, since we have privileged object coordinate information within the simulator, we train a DINOv2 base model with a learned output head to predict 3D coordinates of the target object, then compare the movement of the object during action rollout, and report the displacement error in cm between the predicted video and actual rollout. The results show that beyond the aggregate policy simulation Pearson results, our AFFECT model gives better fidelity in action following for video generation.

MethodOutcome Agreement (accuracy) ↑Object Displacement Error (cm) ↓
Cosmos30.6199.6
Cosmos3 + AFFECT0.8267.3

[W3, Q3] Generalization beyond the perturbation distribution

We would like to clarify that our policy evaluation results already test the world model on actions outside the perturbation distribution. Our main experimental results, shown in Tables 1-2, are conducted by training world models on perturbed task-agnostic play data, and evaluated on specific semantic tasks. Hence, the action sequences seen at evaluation time already differ from the perturbed play actions used for training.

AuthorsOpenReview ↗

Response to Reviewer bju7 (2/2)

[Q4] Statistical significance of planning

To answer your question, we incorporate additional tasks and random seeds to the UR5e planning experiments on a new Cosmos3 backbone, while also providing Wilson 95% confidence intervals below. We evaluate on additional cube-double tasks within OGBench, and run 10 seeds with 4 policy model checkpoints across 10 tasks, for a total of 400 rollouts for each baseline. We show that the results remain statistically robust across task and seed variation.

Methodcube-singlecube-doubleAverage
Uniform Selection0.490 [0.422, 0.559]0.515 [0.446, 0.583]0.503 [0.454, 0.551]
Cosmos30.655 [0.587, 0.717]0.535 [0.466, 0.603]0.595 [0.546, 0.642]
Cosmos3 + AFFECT0.870 [0.816, 0.910]0.770 [0.707, 0.823]0.820 [0.779, 0.855]
DreamDojo0.615 [0.546, 0.680]0.565 [0.496, 0.632]0.590 [0.541, 0.637]
DreamDojo + AFFECT0.680 [0.612, 0.741]0.660 [0.592, 0.722]0.670 [0.622, 0.714]

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.