- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
World models are usually good at generating plausible videos but often lack precise action following. The paper introduces AFFECT, an auxiliary training objective to force the model to pay attention to actions. The authors show improvement in world model generation on both simulated and real world rollouts. Further, they show the improved world model helps in better planning when selecting actions during policy rollouts.
Critical strengths
- The idea of training the model with the perturbed play data seems interesting so the world model would need to pay attention to the action rather than simply memorizing the behavior of the policy it is trained from.
- Authors show world modeling performance improved on both real and simulated environments across multiple embodiments.
- Gains in success rate from planning using world model is a good way to verify improved action fidelity.
Critical weaknesses
- The selected video features seem somewhat arbitrary. Can the authors explain why those features were selected and why not choose standard feature extractors like DINOv2?
- There are no qualitative results in the paper.
- Each embodiment was evaluated over 12 episodes per 4 checkpoints across 2 tasks which seems to be quite low.
Questions for authors
- Cosmos3 Pearson correlation (0.091) in Table 1 seems pretty low. Can the authors recheck that data point?
- The perturbed training data already forces the model to pay attention to the actions since the injected noise is random and can’t be predicted from the scene. Why isn’t that enough? Also, a good ablation could be to show difference between training model on pure play policy rollouts compared to perturbed rollouts.
- During AFFECT steps the model uses N=2 step denoising. The paper argued one step wasn’t enough but can they show some empirical/qualitative evidence that two step would work. Did they try the common alternative of denoising more steps but with gradients only enabled in the last step?
- Can the authors share more details about the training data - episodes, hours, teleop data used to train the play policy?
2 comments
Reponse to Reviewer eC4D (2/2)
[Q3] Two-step denoising and alternatives
We use two steps because a single Euler step from pure noise returns the conditional mean at the optimum. This output has no sampling variance, so for squared features it omits the term and matches only mean effects (Appendix A.8). With N = 2, the second network call is evaluated at an intermediate latent that depends on the initial noise, so the output varies across noise draws and the magnitude features receive a nonzero variance term. N = 2 is still a coarse discretization that under-represents the variance of the full sampler. We chose it as the smallest N for which this term is present, since each additional step adds a forward and backward pass.
Nevertheless, to answer your question, we increase the N value during AFFECT training the UR5e setting, which showcases that increasing N value indeed gives a better discretization, and better video quality at the cost of higher training cost for each AFFECT training step
| Method | Short PSNR ↑ | Short LPIPS ↓ | Short FVD ↓ | Long PSNR ↑ | Long LPIPS ↓ | Long FVD ↓ |
|---|---|---|---|---|---|---|
| DreamDojo | 26.03 | 0.06 | 85 | 24.87 | 0.08 | 108 |
| w/ AFFECT (N=2) | 26.78 | 0.05 | 73 | 25.58 | 0.07 | 93 |
| w/ AFFECT (N=4) | 27.01 | 0.05 | 66 | 26.32 | 0.06 | 86 |
| w/ AFFECT (N=8) | 27.13 | 0.04 | 64 | 26.39 | 0.06 | 83 |
[Q4] Training data details
We provide a brief overview of training data, and have added full details to Appendix A.2 of the revised manuscript. For the real-world FR3 and ALOHA-2 settings, we collect around 100 mins of teleoperation data to train the play policy, which is then rolled out under perturbations to generate around 5 hours of data each.
Reponse to Reviewer eC4D (1/2)
Dear reviewer eC4D,
Thank you for your valuable comments and suggestions in reviewing our work. We address each of your questions and concerns individually as follows.
[W1] Choice of video features, and why not DINOv2?
We chose these features to capture the direction, magnitude and motion of the visual change caused by actions. The channel and projection features measure signed changes in the latent, the squared projection and energy features measure their magnitude, and the flow features measure temporal motion. All features are computed on the world model's own latent, so the gradient can pass through the two-step sampler without decoding the video to pixels or running a large vision encoder inside the training loop. We also keep the features fixed on purpose, since learnable features admit a trivial solution that makes the loss zero for any generator. As shown in Figure 4, these features respond 1.6-2.2× more strongly than the shuffled baseline in regions where the arm, cube and shadow move, while staying at chance level on static regions such as the table and backdrop. This shows that the features do capture where the action changes the scene.
Nevertheless, we evaluate a variant using DINOv2 features, where the generated video latents are decoded into images that are subsequently fed into the DINOv2 feature extractor, where the patch features are then projected with fixed random directions to generate features compatible with our method. This resulted in higher training cost with less of a performance gain compared to the base DreamDojo model.
[W2] Qualitative results
Following your suggestion, we have added qualitative results in Appendix C of the revised manuscript, comparing real, baseline and AFFECT rollouts from the same initial state, following the same action trajectories in long-horizon open-loop execution.
[W3] Number of evaluation episodes
We agree that the number of evaluation episodes is limited, mainly due to the cost of evaluating real robots. However, AFFECT improves the correlation in all six backbone-embodiment combinations, which under a one-sided sign test, where 6 out of 6 combinations improved gives a statistically significant result. Also, given that result, the evidence for improved action fidelity does not rest on these correlations alone. In simulation, where repeated evaluation is affordable, AFFECT also improves action sensitivity over 270 controlled trials (response to Reviewer bju7, W2) and planning success over 400 rollouts per method with non-overlapping confidence intervals for Cosmos3 (response to Reviewer bju7, Q4).
[Q1] Cosmos3 Pearson correlation in Table 1
We rechecked the result and confirmed the value. Under the same training regime as other models, base Cosmos3 often lost track of the target object during manipulation, especially during long-horizon rollouts. As a result, base Cosmos3 rollouts rarely reached task success, and the imagined success rates of all checkpoints were close to zero. Because these near-constant predictions carry little information about the differences between checkpoints, the correlation with real success rates is dominated by noise, which explains the low value.
[Q2] Is perturbed data alone enough?
We believe the flow loss alone is not enough because most of the action can be predicted from the scene, since the play policy chooses its nominal actions by looking at it. Only the injected perturbation is random given the context, and the flow loss does not give it any special weight, as the loss is dominated by reconstructing the scene appearance. AFFECT directly supervises how the generated video should change with this perturbation. Our main table results, which compare AFFECT against world models trained on only perturbed data, show that our method is useful beyond just the perturbed data.