- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper examines the mismatch between training world models for next-state prediction and using them for gradient-based action optimization. It diagnoses a gap between predicted planning loss and actual execution, then studies two responses: Online World Modeling (OWM), which adds simulator trajectories induced by the planner, and Adversarial World Modeling (AWM), which fine-tunes the model using perturbations of latent states and actions. Experiments with DINO-WM cover six simulation tasks and egocentric pose planning on Nymeria. AWM improves gradient-based planning and is reported to approach CEM performance at substantially lower planning time.
Critical strengths
-
The paper identifies a meaningful distinction between predictive accuracy and suitability for optimization. Section 2 and Figure 1 make the issue concrete by comparing the loss optimized inside the model with the outcome of executing the resulting actions. The AWM objective in Section 3.2 addresses this distinction through local adversarial robustness of the transition predictor. This is a well-motivated methodological response to the diagnosed problem, and it can be applied without collecting new simulator transitions. It makes the model's behavior during downstream action optimization an explicit consideration in training.
-
The improvement is supported across several planning settings. Figure 3 reports substantial gains over the original model with GBP, including PushT from 76% to 92%, PointMaze from 68% to 98%, and Reacher from 42% to 76%. The comparison with OWM tests a different response to the same training–planning mismatch rather than only comparing against an unchanged model. Figure 5 extends the evaluation to longer-horizon pose planning with a 48-dimensional action space. These results support AWM's practical value beyond a single navigation example.
Critical weaknesses
W1. The evidence supports AWM's effectiveness more strongly than the proposed explanation of why it works. Section 4.4 attributes the improvement to a smoother action-optimization landscape and draws a broad conclusion about the insufficiency of compute and data scaling from OWM's mixed results. Figures 1–2 are informative, but the study does not report matched clean fine-tuning or non-adversarial perturbation controls that separate the effect of adversarial perturbations from additional training and ordinary local regularization. The central mechanism remains plausible, but its attribution and the dismissal of data coverage are stronger than the evidence establishes.
W2. The main efficiency comparison may also reflect different planning objectives. Section 4.1 states that GBP uses Weighted Goal Loss, including intermediate predicted states, whereas CEM's Algorithm 4 scores the terminal state. Appendix C.4 specifies task-dependent weights, including greater weight on earlier states for PushT, the task used for the time–performance comparison in Section 4.3 and Figure 4. This matters because intermediate-state shaping can change optimization difficulty. The reported results demonstrate the performance of the complete pipelines, but do not yet isolate the claimed efficiency advantage of AWM with GBP over CEM under the same objective.
Questions for authors
Q1 (W1). With the same starting checkpoint, data, and fine-tuning budget, how does AWM compare with clean fine-tuning and random perturbations of the same scale? Does its advantage remain specific to adversarial perturbations, and do these comparisons support the landscape explanation in Section 4.4?
Q2 (W2). Was CEM given the same Weighted Goal Loss as GBP in Figure 4? If the objectives differed, does the reported time–performance advantage remain when both planners optimize the same loss, including the same intermediate-state weights?
2 comments
Response to authors
Thanks for adding the clean fine-tuning and random-noise controls. AWM's advantage on Wall and PointMaze gives much stronger evidence that the gains are not explained by extra training or ordinary noise alone. This addresses W1.
The shared-WGL experiment is useful and shows that AWM also benefits CEM. However, gains of 9.3 and 4.7 percentage points do not tell us CEM's final success rates or how long it takes to reach them. Similar runtime across environments at fixed settings does not establish how much computation each planner needs to reach comparable success.
Could you report the absolute success rates and measured single-horizon planning times of GBP and CEM using the same AWM model and WGL, including the same intermediate-state weights? An updated PushT comparison, as in Figure 4, would be the most direct answer.
I would like to resolve this remaining efficiency comparison before reconsidering my recommendation.
Rebuttal
We thank the reviewer for recognizing that the paper identifies a meaningful distinction between predictive accuracy and suitability for optimization. We also appreciate that the reviewer found AWM a well-motivated response that needs no new simulator transitions, with improvements across several planning settings.
"...With the same starting checkpoint, data, and fine-tuning budget, how does AWM compare with clean fine-tuning and random perturbations of the same scale? Does its advantage remain specific to adversarial perturbations..." We agree, and we have added these controls. Each arm uses the same checkpoint, data and number of updates, with 3 seeds and 50 shared cases. Gaussian and Uniform noise are drawn at AWM's scale.
"...and do these comparisons support the landscape explanation in Section 4.4?" Partially. These controls are consistent with an AWM benefit, but do not establish that smoothing mediates the improvement. To test this more directly, we checked whether the model's directional derivatives of the planning loss along random action directions agree in sign with the simulator's:
"...its attribution and the dismissal of data coverage are stronger than the evidence establishes." We agree, particularly since OWM itself substantially improves PointMaze. We have removed the statement that "compute and data scaling alone will not make GBP viable" and limited the conclusion to the tested configuration. However, it still stands that AWM significantly outperforms a reasonable attempt at applying DAgger and increasing data coverage.
"...Was CEM given the same Weighted Goal Loss as GBP in Figure 4? If the objectives differed, does the reported time–performance advantage remain when both planners optimize the same loss, including the same intermediate-state weights?" We ran AWM+CEM with the same model and WGL objective used for GBP. AWM improves both planners, so the improvement is not specific to gradient-based planning, although the gain is larger for GBP:
Both GBP and CEM optimize action sequences using only the learned world model, without querying the environment simulator for predicted trajectories. Consequently, with the same model architecture, input dimensions, hardware and planning hyperparameters, planning times should be comparable across PushT, Wall and PointMaze.
Thank you again for your valuable feedback. We believe your feedback has substantially improved our paper. We would greatly appreciate it if you would consider increasing your score accordingly. Do you have any other questions we can address?