acceptodds
Under review as a conference paper at ICLR 2027

Visual Fidelity and Physical Reliability in World-Model Rollouts

Abstract

World models are often judged by predicted observations, even when autonomous rollouts are used to reason about physical state. We ask how much a model's native visual prediction error tells us about physical accuracy at the rollout level. Across DreamerV3 and PlaNet, lower visual loss selects rollouts with lower autonomous physical readout error than matched random selection over a broad coverage range, but this ranking does not establish a numerical physical-error scale. We therefore ask what visual loss misses. Around some rollout roots, the queried physical quantity changes strongly under perturbations that produce comparatively little change in the rendered trajectory. A finite-scale query-to-render sensitivity score predicts realized perturbation responses and autonomous physical error beyond visual loss in two independent DreamerV3 cohorts. In a prospectively specified third cohort of 500 episode families, using the same score after visual filtering reduces mean physical error from 0.198 to 0.166 at matched 10% coverage, a 16.0% relative reduction (95% whole-family bootstrap CI for the mean difference, [-0.0488, -0.0162]). Finally, we show that positive-loss screening results do not determine whether physical error must vanish as visual loss approaches zero. Visual fidelity is useful for ranking world-model rollouts, but is not sufficient evidence that a particular physical query is reliable.

Primary area:
reinforcement learning
Keywords:
world models, model-based reinforcement learning, rollout evaluation, physical reliability, visual fidelity, predictive representations

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_WQ5QOpenReview ↗
Rating
1
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates whether loss metrics based on raw visual rollouts can rank the accuracy of fitted physical readouts. A visual filtering mechanism reduces the average autonomous readout error in DreamerV3 and PlaNet models. Across two experimental datasets, a simulator-based "query-to-render" sensitivity score—a metric of limited scale—provides additional predictive information beyond that offered by visual loss rankings. In a third dataset, a fixed two-stage filtering rule reduces error by approximately 16% while maintaining 10% sample coverage. Theoretical analysis demonstrates that if the loss value exceeds a certain positive lower bound, the average error curve derived from filtering cannot guarantee "zero loss in the worst case."

Critical strengths
  1. The visual-20%-then-low-sensitivity rule is fixed before accessing Cohort 3 readout errors; both comparators retain 100 roots. Whole-family resampling reapplies the selection rules and yields a negative paired interval. This is stronger evidence than optimizing a threshold on the same reported outcomes.
  2. The paper does not claim that finite-loss data prove a nonzero zero-loss residual, or that the observation-aided/autonomous contrast is an equal-information causal comparison. The explicit two-function construction correctly separates selected means from a worst-case certificate.
Critical weaknesses
  1. Error is a median across standardized coordinates from finite-capacity separately fitted heads, not a direct physical-state prediction or a risk-sensitive loss. Readout-capacity/calibration controls in Appendix C are primarily for the earlier TD-MPC2 contrast; they do not establish robustness of the main DreamerV3 selection advantage to readout choices. The score is constructed from true simulator dynamics and rendering and could identify general observability or trajectory difficulty. Its association with dA−dE is unsupported, and after additional difficulty controls the two intervals include zero. This does not invalidate the prospective result, but it weakens the explanation of what the score diagnoses.

  2. The separation of visual quality from physical usefulness is already well documented in the cited literature; the generalized-eigenvalue ratio is a conditioning/observability diagnostic. The specific new contribution is a tested finite-scale score and an offline selection result, restricted to one DreamerV3 Cheetah checkpoint. Sensitivity requires privileged state perturbations and many true-simulator rollouts, while visual screening itself needs future observations. Comparisons with simpler privileged signals—query sensitivity alone, visual sensitivity alone, or nonlinear predictors from visual loss—are missing. The theorem is valid but generic, rather than a model-specific certificate or an algorithmic advance. These limitations leave significance below the acceptance threshold under a strict standard.

Questions for authors
  1. Does the advantage of Cohort 3 persist across affine and alternative-capacity readout methods, as well as metrics based on coordinate-wise mean/RMSE, worst-coordinate error, or weighted physical error?
  2. How does this score compare—while maintaining identical retained coverage and evaluation costs—against "query-only" sensitivity, "visual-only" sensitivity, state velocity, and non-linear visual loss predictors fitted on development-phase data?
  3. Can this fixed score improve screening performance for other world model checkpoints or physical tasks without requiring a redefinition of the perturbation/query interface?
  4. Can the specific implications of the "zero-loss theorem" for the model in question be clarified, and can sufficient information regarding rollouts and checkpoint sources be provided to enable independent replication of the key readout and screening results?
Reviewer_grreOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
1
Soundness
2(+1)
Read review
Paper summary

This submission investigates the relationship between visual prediction/pixel-level accuracy and physical reliability in world-model rollouts. The authors evaluate whether native visual loss can serve as an indicator of the accuracy of physical redout using two different model families. Their main finding is that visual loss is useful for ranking rollouts but does not provide a reliable numerical measurement of physical error. To adress this limitation the authors introduce a query-to-render sensitivity measure based on pertubations. They demonstrate that this measure provides additional information about physical error and report a 26% reduction in mean physical error when utilizing it also within rollout selection. Additionally, the authors present a theoretical result showing that finite positive-loss observations cannot establish guarantees about physical error in the zero visual-loss limit.

Critical strengths
  • The paper addresses a relevant question concerning the reliability of world models that goes beyond classical visual metrics
  • The paper is partially backed by two independent model families
Critical weaknesses

Major Weaknesses:

MW1: Clarity and overall structure

This reviewer finds the paper unnecessarily hard to follow, which makes it difficult to identify and evaluate the central contributions. The authors introduce numerous metrics, definitions, experimental settings, and evaluation objectives, but the connections among them are often insufficiently motivated or clearly explained. The organization (where related work is discussed in section 6, for example, or where the difference between methodology and experiments is hard to identify). While not inherently incorrect, it makes it very hard to follow and to identify where the authors separate from related work, for example.

MW2: Limited experimental scope and generalizability:

While the paper starts with two independent Model families, the entire evaluation of the main sensitivity-based result is performed on a single Checkpoint and a single task. This substantially limits the evidence for the general usability of the proposed sensitivity measure. Also, the approach required access to controlled-environment perturbations, which may not be available for a larger number of tasks. As written in the paper, this reviewer finds the evidence to be limited to support the actual claim.

MW3: Interpretation of experiments:

Most of the experiments and results are just number-dropping. An actual interpretation of what the underlying numbers mean is hard to find in the paper. From this reviewer's perspective, it would be more useful to present a central results table or figure and interpret the results in the text rather than simply listing their numerical metric values. This would also directly contribute in fixing parts of MW1.

Minor Weaknesses:

w1: Significance of the theoretical contribution:

The theorem in Section 5 establishes that observations at finite positive visual loss do not necessarily determine physical-error behavior in the zero-loss limit. However, this conclusion appears relatively intuitive without additional structural assumptions. It is unclear to me what additional insights the formal construction provides beyond this observation, or how it meaningfully advances the empirical findings presented in the rest of the paper.

Questions for authors

Novelty and Contribution:

Section 4.1:

What is the specific methodological novelty of the proposed query-to-render sensitivity score compared to existing intervention-based diagnostics discussed in section 6? In particular, is the contribution the generalized eigenvalue formulation, the multi-scale aggregation, or its application to rollout selection?

Section 5:

What additional insight does Theorem 1 provide beyond the observation that measurements restricted to visual losses above a positive threshold cannot determine behavior at zero loss? How does this theoretical result inform the sensitivity-based selection method proposed in Section 4?

Experimental validation

Section 3 & 4

The visual-loss screening results are evaluated on both DreamerV3 and PlaNet, whereas the sensitivity-based results are restricted to DreamerV3 Cheetah. What prevents the sensitivity analysis from being evaluated on PlaNet, and which aspects of the proposed score are expected to transfer across model architectures?

Section 4.4

The two-stage selection rule retains the 20% with the lowest visual loss and then selects half of them based on sensitivity. What motivates this particulat 20%-20% selection strategy? Is there evidence that the improvement is robust to other initial screening fractions? ##Section 4.1 and 4.5: The sensitivity measure is evaluated at four fixed perturbation scales, and the authors report that rankings are not uniformly stable across scales. How were these particular scales selected, and how sensitive is the reported selection improvement in section 4.4 to the choice of scales?

Methodological limitations and interpretation

Section 2.2 and 4.3:

Physical error is measured using separately fitted readouts. To what extent can the observed relationship between sensitivity and autonomous physical error be attributed to properties of the world model rather than limitations of the learned readouts?

Reviewer_TzzmOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates how well visual prediction error reflects the underlying physical accuracy of world-model rollouts. Using DreamerV3 and PlaNet, the authors show that while filtering for low visual error generally reduces physical readout error, visual fidelity alone is not enough to guarantee physical reliability. To address this, they propose a multi-scale sensitivity score computed by perturbing simulator states and comparing how much the physical target changes relative to the rendered trajectory. On an independent DreamerV3 Cheetah evaluation cohort, combining visual filtering with this sensitivity score yields a clear reduction in physical readout error compared to visual filtering alone at the same coverage level. Finally, the paper provides a theoretical construction demonstrating that average screening performance at finite visual loss cannot bound worst-case physical error as visual loss approaches zero.

Critical strengths

S1. Clear problem setup and convincing validation on a held-out cohort.

Rather than just noting the gap between visual fidelity and physical accuracy, the paper tests a practical question: can physical sensitivity help filter rollouts? The evaluation on Cohort 3 is well-executed, locking in the selection rule before testing on 500 fresh episode families. At a matched 10% coverage, two-stage filtering drops mean error by ~16% (with confidence intervals clear of zero), showing that sensitivity captures useful signal beyond visual error.

S2. Rigorous statistical hygiene and honest discussion of limitations.

The authors carefully handle statistical dependencies by bootstrapping at the source-episode level rather than treating correlated rollout roots as independent. They are also upfront about their setup's boundaries—acknowledging the reliance on simulator access, fitted readouts, and a narrow task suite—and appropriately frame their theoretical result as an information limit rather than an overclaimed empirical law.

Critical weaknesses

W1. Missing ablations against simpler heuristics and unclear compute cost.

While two-stage selection helps, it is unclear if the gain comes from the physical perturbation, the visual perturbation, or specifically their ratio. Controlling for velocity and action magnitude in a correlation analysis (Appendix E.5) is no substitute for directly comparing these simpler heuristics—along with physical-only and visual-only sensitivity—in the actual selection task at matched coverage. Additionally, scoring a single root requires 144 simulated trajectories (Appendix D). The paper should report the wall-clock runtime and test whether fewer perturbations could achieve similar gains more cheaply.

W2. Narrow empirical scope and untested readout sensitivity.

The core two-stage selection gain is only demonstrated on DreamerV3 Cheetah; PlaNet, TD-MPC2, and the other tasks in the appendix only replicate partial trends, leaving generalization across models and environments unclear. Furthermore, physical error is measured via learned readouts. While the appendix explores readout capacity, the authors do not verify whether the main selection gains hold up as readout settings change, nor do they break down which specific physical quantities improve.

Questions for authors

1.At a matched 10% coverage, how does the second-stage selection perform if you rank solely by physical change, solely by visual change, or by simple heuristics like state velocity and action magnitude? Running these ablations would really help isolate what the relative sensitivity ratio is contributing.

2.Are all the Cheetah cohorts generated from the exact same DreamerV3 checkpoint? Also, do you have any preliminary results showing the two-stage selection gain on a second task or model architecture?

3.What hardware was used for the sensitivity perturbations, and what is the wall-clock runtime per root? Have you checked if you can subsample the 144 perturbation trajectories (fewer scales/directions) without losing the selection gain?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.