acceptodds
Under review as a conference paper at ICLR 2027

What Must A World Model Distinguish for Planning?

Abstract

World models simulate the consequences of action candidates, but good planning need not preserve every physical distinction required for accurate prediction. We formalize this gap through a hierarchy of mechanism, response, and decision sufficiency. Given a candidate set, the planning query determines which physical variations matter and how precisely they must be preserved: coarse decisions can discard much of the information needed for prediction, whereas fine decisions may require nearly the same resolution. In practice, planners often adaptively search to construct candidates, and information unnecessary for final selection may still be needed to discover good candidates. What a world model must preserve therefore depends on the query, the candidate set, and the planner. We study these effects in a collision system, nonlinear dynamics, and robotic planning. These varying requirements raise a design question: where should query information enter the planning system? A model that jointly generates actions and outcomes conditioned on the query achieves lower regret than an action-conditioned world model on seen objectives, but this advantage largely disappears when generalizing to unseen objectives. Motivated by this, we propose a modular design in which the query determines where to look and an action-conditioned model predicts what will happen, allowing the same predictions to be reused across objectives.

Primary area:
applications to robotics, autonomy, planning
Keywords:
World Models, Physical Distinguishability, Decision Sufficiency

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_atN5OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes 3 progressively weaker equivalence relations for world models (mechanism/response/decision sufficiency) and aims at using them to analyze what a world model must preserve for planning. These ideas are tested on a collision system, a double pendulum, and two robotic pushing tasks, motivating a modular design where the planning objective guides candidate proposals but not outcome prediction.

Critical strengths
  1. The main question is clean and well motivated. The proposed hierarchy itself is interesting and may be useful.

  2. Several controls about the experiment design are careful:

  • Encoders are frozen. A common scorer is retrained on top. So differences are because of retained information and not head capacity.
  • Rates are measured from actual range-coded streams, not surrogates.
  • The search-selection crossing in Push-T is genuinely nice: Full/Full=.180, Full/Decision=.179, Decision/Full=.691. It shows that information that is sufficient for final selection can fail during search. This is, in my opinion, the most convincing evidence for the claim that search is needing different information than selection.
Critical weaknesses
  1. The theory is thin relative to previous works. Lemma 1 is quite standard 2ϵ2\epsilon argument. Decision & response sufficiency seem to be close to policy- and Q-irrelevance abstractions (Li et al. 2006) and value equivalence, only with being restricted to a finite candidate set. The authors have cited these but are in my opinion not precise enough about what's new here. Prop.1 is more a toy example than a general result. All this limits the theoretical contribution.

  2. The mechanism-response-decision hierarchy is presented as the main contribution, but it is not used to organize the paper.

  • It is used formally only in Sec.4.2 (collision).
  • The pendulum (Sec.4.3): response-supervised rates are never reported, so response vs. decision is not compared.
  • Sec.5 and Sec.6.1 use loose analogues (Cost vs. Decision messages, Value vs. Planner).
  • Sec.6.2 discusses query placement extensively, but has nothing to do with the hierarchy.
  • Mechanism sufficiency is defined but is never studied as such. The closest is the Known control in App. C.4, which is not analyzed in hierarchy terms.

Hence, the experimental part consists of 4 case studies that are loosely connected. Each has its own metric and normalization. Since the hierarchy is the main conceptual contribution here, its absence from most experiments is weakening the central claim.

  1. The headline design claim is small and confounded.
  • Held-out regrets for ACWM, Joint and Modular are 0.164, 0.160 and 0.153 (Table 13), from 3 seeds, with no significance tests and unexplained error bars (Fig.8(b)).
  • Modular has the most parameters of the three (4.80M vs. 3.80M ACWM and 4.36M Joint), and Joint+action, with 7.54M, beats everything, so capacity cannot be ruled out as the explanation for the ranking.
  • The authors themselves say (ll.437-439) that the Joint vs. ACWM comparison confounds query placement with action exposure.

Since the Modular design is the main practical recommendation in the paper, these small and confounded effects leave it insufficiently supported.

Questions for authors
  1. I have doubts about the novelty of the Modular design. A query-guided proposal plus a query-independent world model for evaluation seems to be what SAGE (Cheng et al. 2026, cited at ll.089-091) already does. Policy priors guiding planning over a learned model are also quite standard practice. How is your approach different?

  2. What precisely is new relative to value equivalence (Grimm et al. 2020) and π\pi*/Q-irrelevance abstractions (Li et al. 2006)?

  3. The paper essentially juggles between two contributions: the hierarchy AND answering the question from Sec.6 on "how a world model should preserve it or where the query should enter". I think the paper would gain clarity and impact by focussing on one. Which of the two do you consider the main contribution?

  4. What do the error bars in Fig.8 show? Can you report confidence intervals and/or apply significance tests for Fig.8?

  5. Does Modular still beat Joint when parameter count are matched?

Reviewer_rjgSOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper questions the distinctions between different physical worlds that a world model must preserve in order to be considered to “plan well”. To get at this question, the authors define three notions of sufficiency, namely “mechanism, response and decision”. They then show that cost errors (of epsilon) give at at most 2 epsilon regret, using a collision example in 1D to compare the difference between threshold and tracking queries. They find that threshold queries need far less information. They then provide experiments on different tasks (collison, pendulum, Push-T and PushCube) and end the paper with a recommendation for a Modular design where the query guides proposals.

Critical strengths
  • The paper is generally well written and engages with existing and recent world-model literature.
  • The search-versus-selection experiments are interesting (Figure 4B) and show neat differences in what is good for selection vs search.
  • Most experiments clearly illustrate the premise they are trying to convey. The collison example shows how different queries depend on different physical quantities, which is neat and intuitive for me as the reader.
Critical weaknesses
  • The theoretical novelty is not sufficiently clear. It seems the work is closely related to similar distinctions studied by Li et al. and Grimm et al.. Also, unless my interpretation is incorrect, the (2) regret bound is a standard approximation argument.
  • Modular is a neat design, but the pairing (with the world model) that they study is present in prior planners and work such as SAGE.
  • Design conclusion is not convincing. It seems difficult to isolate experimentally. If ACWM and Joint differ in many ways, then the corresponding drop in Joint’s advantages (over ACWM)(-0.027) to (-0.004) is not easy to attribute to say how the predictor is conditioned. This is especially the case because Modular’s advantage also shrinks, from (-0.041) to (-0.011).
  • The small Joint interface (4.36M) gives very limited data points about how we should expect a large world-action model to behave.
Questions for authors
  • What result of this paper in particular changes our understanding of proper design for world models, in contrast to the previous work cited in your paper and cited above?
  • What would a practitioner change in their world model architecture and/or training informed by this paper uniquely? For example, DINO-WM or LeWM (or any recently world model of choice).
  • In Section 6.2, could the authors evaluate Joint’s predictor with Modular’s proposal, training it on the ACWM broad actions and matching capacity? It would help the main claim if conditioning the predictor on the query then still hurts held-out objectives.
Reviewer_65sZOpenReview ↗
Rating
4
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes a hierarchy of world model abstractions that preserves different properties as they relate to the planning process. They define 3 levels of preservation based on what the model can simulate (mechanism sufficiency), what problems it can evaluate (planning queries) and decision sufficiency (what is the low-regret action to take). The paper includes a systematic presentation of the effects of this hierarchy in different planning domains that showcase different structures, showing that predictive quality of a world model is not fully aligned to the requirements of planning and acting.

Critical strengths

The paper is well-organized and written. Though notation and content is information dense and requires careful reading. Moreover, the authors tackle the question of the required structure that world models need to be useful for planning in a systematic way and propose a useful framework in which to think about world model learning.

Critical weaknesses

In general, my concern is that the such an information and notation dense paper sometimes becomes hard to parse. However, the structure is generally amicable to the reader.

Questions for authors

See above.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.