acceptodds
Under review as a conference paper at ICLR 2027

When can a world model control ?

Abstract

World models enable agents to simulate and plan, but when do decisions evaluated inside a model remain valid during physical interaction? In this work, we present a normative theoretical framework establishing the conditions under which a world model supports downstream physical control. By formalizing the response map from feedback policies to interaction laws, we show that matching optimal outcome values alone leaves command-consequence correspondence undetermined: a model and a physical system can agree on achievable task scores yet respond differently to the same command. We prove that preserving this policy-specific correspondence tightly bounds downstream planning loss by model substitution error. Through an exact projection onto causal behavior, we decompose a target trajectory's requirements into initial preparation, pre-action information, and post-action physical response. For Gaussian systems, we solve physical realization analytically over all randomized controls, exposing fundamental joint limits on precision, correlation, and sensor timing: a sensor reduces the outcome variance limit only if its reading precedes corrective action. Furthermore, our framework explains a counterintuitive practical failure: why expanding skill libraries can require finer state distinctions and amplify model selection errors. We validate our theoretical predictions across exact finite processes, learned visual models, and continuous contact simulations by systematically varying feedback timing, response coverage, and skill availability. These results connect prediction objectives, latent representations, and execution interfaces directly to the downstream control decisions a world model must support.

Primary area:
reinforcement learning
Keywords:
World Models,Partially Observed Markov Decision Processes (POMDP),Physical Realizability,Decision Making & Control

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_eQpdOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper asks when decisions made inside a world model still hold up when they are used in the real environment. The authors create a theory on how to evaluate this. Experiments are conducted on small-scale custom-built environments combined with custom model architectures.

Critical strengths
  • The paper's general idea is worth considering.
  • The reproducibility material and appendix is extensive (checkpoints, code, seeds, etc.)
Critical weaknesses
  • The paper is very hard to follow. Very abstract and complicated written. Without considering experiments/results at this point, in the current form this paper is not appropriate for publication, because of the used terminology and writing style. In terms of terminology it gets very confusing. To name a few, what is "command-consequence correspondence", "legal causal policy", "model substitution discrepancy", "post-action physical response", "physical noise floor", "probing consequential comparisons", "admissible control continuations", "A useful starting bank makes errors that displace its incumbent consequential.", "relevant control continuations"? This is far outside the scope of world models / model-based RL literature/terminology and a major concern.

  • The authors motivate the paper with Genie, Cosmos, V-JEPA 2 and DreamerV4, but no experiment touches a model like this, or any established world-model in the size of like ("Dreamerv2", "LeWorldModel", "TD-MPC2", "PlaNet", "World Models"), nor any real-robot data. Given the evidence, their theory is not tested under realistic settings and the title, framed as a question, is therefore not substantially answered.

  • The paper does not use any known or standard datasets that related methods consider. The data and all environments are generated by the authors which are tabular, small grid or custom 2D environments. These experimental results provide no anchor to ground their work in the broader literature.

References:

  • Hafner, Danijar, et al. "Mastering atari with discrete world models." arXiv preprint arXiv:2010.02193 (2020).
  • Maes, Lucas, et al. "Leworldmodel: Stable end-to-end joint-embedding predictive architecture from pixels." arXiv preprint arXiv:2603.19312 (2026).
  • Hansen, Nick, Hao Su, and Xiaolong Wang. "Td-mpc2: Scalable, robust world models for continuous control." International Conference on Learning Representations. Vol. 2024. 2024.
  • Hafner, Danijar, et al. "Learning latent dynamics for planning from pixels." International conference on machine learning. PMLR, 2019.
  • Ha, David, and Jürgen Schmidhuber. "World models." arXiv preprint arXiv:1803.10122 2.3 (2018)
Questions for authors
  • Which of Theorems 1-3 do you consider novel?
Reviewer_RduYOpenReview ↗
Rating
1
Confidence
3(+2)
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

I unfortunately failed to understand the contribution of the paper. I believe it somehow relates to world models, but its beyond my human understanding.

Critical strengths

Not that I could acknowledge, even if there is.

Critical weaknesses

I've tried to read the paper very carefully, but it's really, really hard to understand the core motivation, and the overall idea on what's going on in the paper. Even if there's a contribution, it's also important to convey this information in an understandable narrative and format.

Questions for authors

None, honestly.

Reviewer_Ho6SOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies when a world model preserves the effects of control. It compares interaction distributions under the same feedback policy, not only the best outcome scores. For rewards in [0,1], an η-optimal model policy has deployment regret at most 2Δ+η, relative to the best physical policy in the same policy class. It also gives a three-part KL projection onto randomized causal policies in finite spaces, a local feasibility test for sufficient states, and exact Gaussian precision limits. Experiments study information timing, response learning, and skill selection. [Theorem 1–3; pp.6–9]

Critical strengths
  • The paper separates problems that need different fixes. Matching the best scores does not mean the same command has the same effect. Also, a correct model cannot make every target possible. The Gaussian result is a useful concrete example: it finds the best KL projection over all controls independent of the later Gaussian noise, including non-Gaussian controls, with unrestricted additive actuation. It is more than just a covariance lower bound. [pp.4–5; p.30, Proposition 10]

  • The controlled comparisons help explain the mechanisms. Study H uses exact dynamics to isolate information timing. Study I keeps the model architecture and training updates fixed while changing data collection. At 6,000 labels, probes improve success over routine collection by 17.06 percentage points, with a paired 95% interval of 10.55–23.31. The paper also reports limits: for position cost on original bodies at this budget, expansion helps when averaged over all 2,925 starting banks, although it hurts selection from the chosen direct-push bank. [pp.38, 41–43; p.51, Table 3]

Critical weaknesses
  • The new theoretical contribution needs to be clearer. The paper already discusses simulation lemmas, KL chain rules, predictive states, and selection errors. Still, it is hard to separate the common viewpoint from results that go beyond these existing tools. A closer comparison of the actual assumptions and conclusions, especially for Proposition 10, would help judge the contribution. I am not claiming the result already exists. [p.3; pp.74–76]

  • The experiments do not yet show reliable checks for deployment. Study F masks the nuisance before encoding and directly changes outputs used by the planner. Its three-step probe gives a lower bound, not an upper guarantee for eight-step planning. The conservative skill-acceptance rule is also not tested with calibrated error bounds. Study I's intervals are conditional on one collected dataset per condition. These limits do not invalidate the theory, but leave its practical use less clear. I am not asking for a new SOTA system. [p.29; p.48; pp.69–71]

Questions for authors
  • Which results go beyond existing theorems, especially Proposition 10 relative to the covariance-steering work in Appendix D.4?
  • What existing evidence supports using the proposed diagnostics beyond the constructed visual interventions and the short-horizon probe?
  • Do you have results for the acceptance rule using calibrated bounds that hold together across candidate comparisons, or independent repetitions of data collection? Such evidence would affect my assessment. Ensemble spread alone does not establish the required guarantee.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.