acceptodds
Under review as a conference paper at ICLR 2027

When Prediction Chooses the Representation: Learning Dynamics and Planning in World Models

Abstract

World models are trained to predict but deployed to make decisions. What happens between these goals when an encoder and latent dynamics model learn together? Predictive training creates a competition: easy-to-predict features gain capacity early, accelerate their own dynamics learning, and suppress harder features that determine which actions succeed. When selection varies across the state space, it bends latent geometry, changing which trajectories appear near or inexpensive to a planner. A model can thus predict accurately and avoid collapse while presenting the wrong world for decision making. Constraints on marginal latent distributions cannot rule out this failure, because the same healthy statistics can encode controllable structure or irrelevant nuisance. Our analysis reveals the missing design object: the geometry of action-conditioned consequences. Useful representations must preserve distinctions that change attainable futures, identify variations that do not, and stop changing once this decision geometry is adequate. This connection between learning dynamics and planning quality turns diagnosis into model-design guidance. Latent capacity should protect decision directions; training should prevent predictable nuisances from starving them; poor action coverage should prompt informative interaction rather than stronger regularization; and constraints should become silent after forming planning-compatible geometry. Evaluation must therefore look beyond prediction loss and latent health to ask whether the learned geometry preserves decisions. A world model should not merely predict the world, but organize it around the decisions it will face.

Primary area:
learning theory
Keywords:
world models, representation learning, latent dynamics, planning, training dynamics, representation geometry, action-conditioned prediction, anti-collapse regularization

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_1hQwOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper analyzes JEPA-style world model architectures noting that high predictive accuracy in latent space does not entail representation of the features salient to decisionmaking. They view the latent space as comprising "decision" and "nuisance" components. They analyze theoretically pathologies of the dominant JEPA training objectives that cause it to latch onto nuisance components at the expense of decision components, propose a fix that amounts to encouraging invariance to nuisance dimensions, and then show that this surpasses JEPA on 2 toy control problems.

Critical strengths

None

Critical weaknesses
  1. The method appears to assume privileged access to histories that are the same w.r.t. decision-relevant features but different w.r.t. nuisance features. This is basically saying that someone has already labeled the solution to the problem.
  2. But you don't need to label the solution to the problem because it's already solved by the prior art. And there is no comparison to inverse dynamics, k-step inverses, quasimetrics, etc -- which are great ways of solving this exact problem! That whole literature is ignored. John Langford is not even cited, much less compared to. If the authors are curious they could look into k-step inverses; sensorimotor world models from Scholkopf's group is a more gentle (but much less cited) version of essentially the same idea in a JEPA setting.
  3. Experiments are very toy. Would be easily solved by much prior work such as AC-State, which also does much harder state representation learning.
  4. Theorems are trivial once you wade through the AI slop. Theorem 4.1 just says "if there's both decision-relevant feature and decision-irrelevant features then there exist representations that basically only encode one or the other, and just looking at the entropy of the resulting distribution won't tell you which". Theorem 4.2 is better illustrated with a picture or a 2D latent space example
  5. Writing is extremely hard to understand, at least to me, and I'm reasonably fluent in the world model literature. It reeks of claude-y faux-sophistication, and behind the veneer of technical depth is little of substance that I can dig out. It's plagued by more ancillary AI-isms, such as repeatedly defining the same notation (e.g., its notation for ReLU is defined multiple times, as though each section was made by a different chat)
Questions for authors

The only thing that could change my assessment is:

  1. Do your current results assume privileged access to information about the quotient set of histories under the span of nuisance variables?
  2. How do your results compare to just adding an inverse dynamics term to your loss?
  3. Can you beat strong baselines (like those under critical weaknesses) on harder benchmarks?
Reviewer_fGY4OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies how joint encoder-predictor training can suppress decision-relevant features while retaining predictable nuisance. It establishes that identical latent marginals and zero prediction loss need not imply identical decision sufficiency, then analyzes feature competition in a normalized diagonal population model. Its central result is an exact recoverability boundary under bounded directional score interventions when the nuisance predictor is already fitted. A complementary boundary controller preserves a specified decision share and guarantees exponential decay of its predictor mismatch.

A separate planning bound assumes faithful representation geometry. The action-conditioned DQF constraint is an empirical illustration, explicitly not an implementation of the exact controller. Validation combines finite-step population optimization with two controlled visual planning tasks. These experiments support prediction-planning dissociation and benefits from decision-directed supervision, but do not establish the analytic recovery boundary in neural models.

Critical strengths
  1. A sharp recoverability result rather than another qualitative starvation explanation. Theorem 4.3 distinguishes states recoverable by maximal intervention from states that every admissible intervention fails to recover, including the boundary case. The conserved-quantity argument and cooperative comparison in §8.3 support this stronger conclusion. This is the main reason to support acceptance. Prior work already analyzes feature starvation and encoder-predictor collapse basins; the specific added value here is the explicit characterization under bounded score intervention, not the existence of learning feedback alone.

  2. A complementary prevention guarantee with relevant empirical illustrations. Theorem 5.1 turns retained modal share into the concrete learning bound �0��0�, with a well-defined, instantaneously minimal boundary correction. The Wallpaper result also demonstrates a substantial practical dissociation: SIGReg has lower reported prediction loss but success 0.0040.004, versus 0.4420.442 for DQF. This connects the theoretical motivation to actual planning rather than relying solely on latent diagnostics.

Critical weaknesses
  1. The reduction from controlled prediction to the analyzed objective leaves an important assumption unstated. Equation (5) contains �0��0�, but the objective preceding Theorem 4.2 contains only innovation cost and state-predictor mismatch. For example, with Ei(x)=wixiE_i(x)=w_ix_i and a conventional predictor Fi(z,u)=pizi+B^i⊤u,F_i(z,u)=p_i z_i+\widehat B_i^\top u, even mutually independent centered state, action, and innovation produce the additional loss (wiBi−B^i)⊤Σu(wiBi−B^i).(w_iB_i-\widehat B_i)^\top\Sigma_u(w_iB_i-\widehat B_i). This generally changes the coupled gradients. A tied exact action branch or an explicitly idealized reduced objective could resolve the issue; merely initializing the action branch as fitted does not preserve that property as the encoder changes. This is not a counterexample to the analysis of the stated JJ, but it limits the interpretation of that analysis as controlled world-model training until the parameterization is specified.

  2. The experimental evidence is not fully reproducible or internally reconciled. Equation (18) and Table 7 do not operationally specify the construction of UcU_c, the nuisance-equivalent pair sampler, or the planning-head parameterization. These details determine how much decision geometry is supplied through supervision rather than learned. Acknowledging richer relational supervision does not establish its provenance.

There is also a consequential numerical discrepancy: the five Distractor-DQF prediction losses in Table 2 average 0.028862, with sample standard deviation 0.001376, rather than Table 1’s 0.0589±0.00140.0589\pm0.0014. The per-seed values reverse DQF’s prediction-loss ordering relative to SIGReg’s 0.04250.0425. This does not erase the Wallpaper finding, but the cross-task interpretation requires reconciliation.

Questions for authors
  1. Model clarification: What action-predictor parameterization and orthogonality assumptions reduce Equations (2)/(5) to the modal objective JJ? Is action-response learning deliberately excluded? This determines the controlled-learning setting covered by the analysis.

  2. Method clarification: How exactly are UcU_c and nuisance-equivalent pairs generated, and what is the planning head? Does training require simulator state, resettable interventions, or only recorded observations and actions? This determines reproducibility and the supervision assumptions behind DQF.

  3. Reporting clarification: Which Distractor-DQF prediction losses are correct? Do Tables 1 and 2 use identical runs and loss normalization? This determines whether the prediction-loss reversal is supported on both tasks.

Reviewer_sgmcOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies why joint-embedding predictive world models can achieve low prediction loss without representation collapse while still failing at downstream planning.

Critical strengths

The dynamical systems analysis of feature starvation in coupled encoder-predictor training is exact and insightful. While slow-feature bias [1] and gradient starvation [2] in self-supervised learning are known qualitatively, identifying the conserved quantity H(r,y)=blog⁡y−y+clog⁡(1+r)\mathcal{H}(r,y) = b \log y - y + c \log(1+r) in Theorem 4.3 yields a sharp analytical phase transition (Φ0=0\Phi_0 = 0) showing when bounded regularization arrives too late to rescue decision features. This mechanism is convincingly verified by finite-step gradient descent in Figure 4.

Critical weaknesses

The empirical loss RDQFR_{\text{DQF}} (Eq. 18) is disconnected from the unsupervised theory in Theorem 5.1 and relies on undisclosed privileged supervision. Computing lfibl_{\text{fib}} and leql_{\text{eq}} requires nuisance-equivalent pairs h∼Hh′h \sim_H h', action-consequence coordinates UcU_c, and co-branch endpoints. How these are obtained from pixels is never explained, though they appear to require simulator resets and ground-truth state pairing across backgrounds. Comparing an oracle-supervised objective against an unsupervised marginal regularizer like SIGReg [3] is not a fair comparison and fails to validate the self-supervised dynamics theory.

Experiments are limited to toy 2D point-mass tasks rather than standard distracting control benchmarks [4], with DQF reaching under 50% planning success despite privileged supervision. The ablation in Figure 5 is also confounded, as removing leql_{\text{eq}} allows trivial head collapse, and the variance-matched control acknowledged as necessary in Section 6 is missing. Finally, there are several editing issues, including a mismatch in DQF's Distractor prediction loss between the summary in Table 1 and the per-seed breakdown in Table 2, an obvious typo in the first row of Table 4, and an unreferenced Ricci flow derivation in Appendix 8.5.

Questions for authors
  1. How exactly are UcU_c and the nuisance-equivalent pairs h∼Hh′h \sim_H h' in Eq. 18 constructed during pixel training? Does DQF require ground-truth agent coordinates and simulator state resets to generate co-branch endpoints, and if so, how can it be applied to standard RL settings or fairly compared against observational regularizers like SIGReg [3] and bisimulation methods [5]?
  2. Since removing leql_{\text{eq}} in Figure 5 allows trivial collapse of PψP_\psi, can you provide the variance-matched no-coordinate control mentioned in Section 6 (e.g., replacing leql_{\text{eq}} with SIGReg or VICReg variance regularization on PψP_\psi) to demonstrate that action-consequence direction—rather than mere head non-collapse—drives the improvement?
  3. Which prediction loss for DQF on Distractor is correct—0.05890.0589 in Table 1 or 0.02890.0289 from the per-seed breakdown in Table 2—and how does this affect the claim that DQF trades higher prediction loss for better planning on Distractor?

[1] Joint Embedding Predictive Architectures Focus on Slow Features

[2] Gradient Starvation: A Learning Proclivity in Neural Networks

[3] LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixels

[4] The Distracting Control Suite – A Challenging Benchmark for Reinforcement Learning from Pixels

[5] Learning Invariant Representations for Reinforcement Learning without Reconstruction

Comments on the paper

1 comment

Dear authors and reviewers,

The author-reviewer discussion period closes on November 18 (AoE). Please engage in response periods so that we have our issues discussed thoroughly.

AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.