- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper investigates when test-time refinement can correct errors in action-conditioned world-model rollouts. Across DeepMind Control locomotion environments, the authors evaluate proprioceptive states and DINOv2 visual representations. They report predominantly linear, rather than exponential, error growth and improve the transition-verifier AUROC through paired-drift negative training. However, gradient-based correction, directional regression, and verifier-weighted particle selection do not consistently improve prediction fidelity. A fine-tuned Δ-probe estimates how much prediction error is learnable from rollout inputs, highlighting differences in model bias and manifold geometry. Finally, the authors derive a correction-budget bound relating achievable error reduction to the fraction of unexplained prediction variance.
Critical strengths
Some of the strengths of this paper are:
-
The paper examines an important problem in world-model-based prediction: whether test-time refinement can mitigate accumulated rollout errors. The paper provides some useful empirical evidence about the relationship between transition plausibility and prediction fidelity.
-
Although the paper lacks a lot of details, I particularly find the -probe experiments quite interesting. It allows distinguishing between the predictability of rollout errors and the geometric alignment of correction directions with the ground truth.
Critical weaknesses
The paper seems to perform a benchmarking study with a proper, coherent analysis and description of key research questions. The key weaknesses of the current draft are as follows:
-
The paper investigates an interesting problem, but its experimental contribution is not presented through a sufficiently coherent and systematic framework. Much of the work reads as an empirical benchmarking or diagnostic study of different correction mechanisms. However, the authors do not clearly establish the research questions, comparison criteria, and experimental controls that would make such a study convincing.
-
A fundamental methodological question concerns the motivation for using an energy-based verifier in the first place. The authors have access to ground-truth rollout trajectories and can generate the corresponding drifted predictions from their frozen world model. Therefore, they also have the supervision necessary to train a network to predict the true correction residuals directly or to fine-tune the world model on its own drifted contexts. Indeed, Section 4 introduces a supervised directional regression head, and Section 5 subsequently introduces the Δ-probe, which fine-tunes a copy of the original world model to predict ground-truth next states from drifted contexts. In summary, it is unclear what motivates introducing the energy verifier instead of directly learning correction residuals or fine-tuning the world model.
-
Important details of the world-model setup and input representations are missing (Section 2). The paper provides insufficient information about the construction and preprocessing of the proprioceptive state representations. This is particularly concerning because the paper's central findings depend on differences between proprioceptive and pixel-based representations. The authors should clarify the following: a. Which physical variables constitute the reported 17–24-dimensional proprioceptive states? What is the exact dimensionality in each environment? b. Are all simulator observation components used directly, or are some selected, transformed, or excluded? c. Are the proprioceptive vectors passed directly to the Transformer after standardization, or is an additional representation-learning step involved? d. How are the normalization statistics estimated, and which dataset split is used? e. What are the complete world-model architecture, training objective, and relevant optimization settings?
Although Appendix C provides some reproduction details, these questions remain insufficiently addressed.
-
The verifier training objective is not sufficiently specified (Section 3.2). In particular, the authors should provide the exact loss function used to train , including the treatment of positive and negative candidates, the negative-sampling procedure, and the mechanism used to anchor energy zero for genuine on-manifold transitions at (k=0). The treatment of ground-truth candidates under drifted conditioning also requires clarification.
-
Section 4 is arguably the most important experimental section of the paper because it examines whether the trained verifier can actually improve rollout predictions. However, it provides comparatively little information about the implementation of the three correction strategies. a. For gradient-based refinement, the authors should explain how the residual scale is estimated, how the trust region and gradient normalization are implemented, and how the number of refinement steps and early-stopping criteria are determined. b. For regression-based correction, the manuscript does not sufficiently describe the directional head's architecture, its exact training loss, or whether any parameters are shared with the energy verifier. c. For particle-based selection, the authors should specify the particle count, proposal distribution, perturbation magnitude, resampling procedure, and selection or tuning of .
-
The authors report a substantial deterioration in prediction performance when applying particle-based imagination to 1,024-dimensional DINOv2 representations. They attribute this deterioration to perturbations pushing predictions away from the data manifold. However, DINOv2 representations are structured, and arbitrary perturbations in the feature space do not necessarily yield representations of realistic images or physically meaningful states. Consequently, the failure may originate from the candidate-generation mechanism rather than from verifier-weighted selection itself.
-
The primary experimental protocol initializes the world model using 16 ground-truth states. While this is a reasonable setting for studying open-loop error accumulation, it assumes an accurate initial observation history. The authors do not examine the robustness of their findings when the initial proprioceptive observations are noisy or inaccurate. A straightforward additional experiment would be to introduce controlled random noise into the initial observations and evaluate how the different correction mechanisms behave. This would help determine whether the reported findings remain consistent under less-ideal observation conditions.
Some additional comments:
-
Furthermore, the authors repeatedly characterize their pixel-based world models as data-starved. Although the reported validation-to-training error ratios and the data-availability experiment in Section 5.3 provide some supporting evidence, the explanation is not fully established. The limited number of training-data configurations makes it difficult to isolate insufficient data as the principal cause. Related claims should also be supported by appropriate citations to existing literature on world-model overfitting, generalization, and sample efficiency.
-
Several established techniques are introduced without adequate methodological references or implementation explanations. For example, Section 4 employs Sequential Importance Resampling (SIR) and refers to PETS-style particle control, but the relevant methodological background and implementation choices are insufficiently documented. This is true for many other such critical claims and approaches used in this paper. In general, the paper's overall writing and presentation need significant rework.
Questions for authors
The current draft lacks sufficient information on critical parameters, training objectives, loss functions, experimental setups, and relevant hyperparameters. I would request that the authors carefully review the critical weaknesses to understand the key questions.