acceptodds
Under review as a conference paper at ICLR 2027

Learning World Models with Contingent Self-Prediction

Abstract

World models supervised via latent self-prediction naturally prioritize predictable environment dynamics. However, complex settings are rife with predictable distractors, such as periodic motion or moving backgrounds, that are trivial to predict yet uninfluenced by an agent's actions. Crucially, standard non-collapse regularizers cannot eliminate them, as representations saturated with task-irrelevant noise can satisfy distribution constraints at zero prediction loss. To resolve this, we exploit time homogeneity: identifying features over a future planning horizon already determined by present history is equivalent to identifying features of the present that were already foreseeable from an older context. Conditioning on this older context allows us to decompose state predictions into a predetermined component (fixed by past history) and a contingent component (driven by recent actions and observations). We call the resulting approach contingent self-prediction: an objective that enforces non-collapse constraints exclusively on the contingent subspace. Our method is provably invariant to predictable distractors, ensuring state capacity is prioritized for plan-dependent dynamics.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
world models, representation learning, self-supervised learning

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_YnBUOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The manuscript identifies a failure mode of self-predictive world models: (in short) when the environment has a strong predictable factor, an optimal learned state of the history is entirely that factor and this can't be solved with standard non-collapse fixes. The proposed fix is to apply anti-collapse regularization only to parts of the state that were not foreseeable from history. Experiments that appear to validate the theoretical insights are conducted.

Critical strengths
  • The problem the paper focuses on is very relevant for training world models. The theoretical work done in the paper is non-trivial and yields a proposed fix (admittedly under moderately strong assumptions, but maintaining enough generality to be useful).
Critical weaknesses
  1. The experiment departs from the theory in non-trivial ways. The context is fixed once (episode-start frames, both during training and at planning time) and reused unchanged across the whole 25-step rollouts. The context never slides forward as planning progresses, even though the theory's sliding-window picture assumes it would. This leaves me unsure the experiments are actually validating the theoretical insights (e.g., your Eq 11 in this case, could it not be simply doing "background subtraction" on the regularization term, which would explain your results?).
  2. The experiment "distractors" are highly artificial and of a single kind (textured backgrounds). This makes it hard to assess whether the theoretical formalism actually transfers to situations that are messier or with any other types of distractors.
Questions for authors
  1. The paper is generally well-written, but it required more effort than usual to keep the connection of the stated goals in mind while reading through the technical sections. The writing also leaves implicit the domains of key objects (e.g., the domains of ff, ϕ\phi, ψ\psi in section 2) and how they relate to applied world models. Adding a little bit more exposition would make the manuscript more amenable to readers who are not deep into operator/spectral theory of stochastic processes.
  2. What are the most relevant works from pre-2020 literature? The operator-theoretic framework in Section 2 parallels classical subspace system identification works (e.g., Larimore, 1990; Van Overschee & De Moor, 1994, Akaike 1974, etc.). Is the specific collapse/impossibility result a genuinely new finding within existing frameworks, and why does the manuscript not discuss standard literature?
  3. (Optional, would strengthen but not required) Have you verified the main proofs with a proof checker?
1 comment

We thank the reviewer for all the questions and feedback. We note that we've updated the main table with improved results on all 4 environments (see Summary of Changes).


W1 Context not sliding

The theory defines the contingent state for any set of old frames relative to which the task variables are contingent, not necessarily a window that advances with the rollout. While our experiments set the context to the first frames of the episode, we now ablate Δ\Delta in new Table 3 setting Δ=10,30\Delta=10, 30. The representation is mostly unchanged. What the lag affects is planning (SR 0.88 → 0.74 from Δ=10→30\Delta=10 \to 30). With a fixed lag the predictor only ever sees a context at Δ\Delta steps during training, while at rollout the context is fixed and its lag grows. Episode-start training exposes the predictor to every lag during training, so every rollout lag is in-distribution and the context does not need to slide. The paper is now explicit about why we made this choice.

New Table 3: PushT Δ Ablation

Agent (α=0.0)Blk pos (α=0.0)Blk rot (α=0.0)SR (α=0.0)Agent (α=0.2)Blk pos (α=0.2)Blk rot (α=0.2)SR (α=0.2)
Δ=100.980.970.940.880.980.970.890.88
Δ=300.980.960.940.740.980.970.940.76

W1 Background subtraction

For a static texture, deflation by E[Ψ∣c]\mathbb{E}[\boldsymbol{\Psi} \mid c] should be tantamount to a background subtraction in embedding space. However, CSP also removes predetermined content that is not static, as shown by the new experiments below, which better validate the theory.


W2 One kind of distractors

We added 2 more distractor types:

  • Temporally evolving non-background distractors. A new experiment in Sec 4.1 in an environment containing 3 deterministic clocks, 2 irrelevant and 1 that gates the dynamics of a task signal. The distractors are not static and fully predictable. We plot the linear probe R2R^2 during training in the new Fig 2. LeWM fills its state with the clocks (R2R^2 0.6–0.8) and loses the signal (R2→0R^2 \to 0); CSP keeps the signal (R2>0.9R^2 >0.9), does not waste state capacity on the irrelevant clocks (R2≈0.01R^2 \approx 0.01), and retains the gating clock (R2>0.85R^2 >0.85) needed to predict the signal.

  • Moving backgrounds. A new scrolling background distractor benchmark, where the texture translates at a fixed velocity throughout the episode (Sec 4.4). CSP matches its static-distractor performance on all three environments while the baselines degrade.

New Table 2: Two-Room Scrolling Texture

Agent (α=0.1)SR (α=0.1)Agent (α=0.2)SR (α=0.2)
LeWM0.00.290.00.48
CSP1.00.961.00.96

New Table 2: PushT Scrolling Texture

Agent (α=0.1)Blk rot (α=0.1)SR (α=0.1)Agent (α=0.2)Blk rot (α=0.2)SR (α=0.2)
LeWM0.00.00.050.620.090.1
CSP0.980.90.840.980.910.9

Q1 Exposition

We have revised Sec 2 to ensure each object is given its non-operator, world-model meaning when it is introduced.


Q2 Pre-2020 literature

The operator Tf=E[f(future)∣past]T f = \mathbb{E}[f(\text{future}) \mid \text{past}] and its SVD are indeed Akaike’s canonical correlation analysis (CCA) between past and future (1974), and CVA (Larimore) and N4SID (Van Overschee & De Moor) are its linear-Gaussian algorithms. They pick the state as the leading canonical directions of the past-future correlation, thus also ranking by predictability. The same idea also arises in spectral Predictive State Representations. The revised paper now introduces TT this way and cites these works in Sec 2. While the predictability-based feature ranking principle is classical, our contributions are:

  • Our operator TΔf=E[f∣h,a]−E[f∣c]T^\Delta f =\mathbb{E}[f \mid h, a] - \mathbb{E}[f \mid c] partials out an old context cc and ranks by what became predictable since cc.

  • The collapse result is about learned nonlinear features with an anti-collapse constraint. In the linear-Gaussian setting with a known state dimension, deterministic components are a separate, finite-dimensional block that the classical algorithms handle explicitly. With a learned nonlinear feature map, the predetermined content is a flat block of unbounded dimension (any function of a deterministic factor is perfectly predictable), and a finite budget of whitened features is filled by it at zero cost. Theorem 2 shows the optimum is a function of the predetermined factor alone.

  • The impossibility result targets how modern self-supervised methods prevent collapse by constraining the marginal law of the state (whitening, VICReg, SIGReg). Theorem 3 shows that for any target law QQ there is a state that is exactly QQ-distributed, fully predictable and task-blind. Thus, no constraint on the marginal can rule the degenerate solution out.


Q3 Proof checker

We appreciate the suggestion! We will add Lean proofs to the repository.

Reviewer_cEY8OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper argues that “being predictable” is not the same as “being useful for planning.” In particular, it demonstrates that self-predictive world models may allocate capacity to encoding dynamic-but-trivial factors (such as backgrounds, clocks) that are already determined, rather than encoding task states determined by recent actions and observations. The proposed method, Contingent Self-Prediction (CSP), aims to reserve representational capacity for the variables that planning actually needs. CSP first uses earlier history to estimate the “already-known” part of the target representation, and then applies anti-collapse constraints only to the remaining part.

Critical strengths
  1. I appreciate that the paper provides a clear story from counterexample construction to method design and experimental validation.
  2. It is insightful to shift the criterion from “which features are most predictable” to “which features have only recently become predictable.”
Critical weaknesses
  1. The theoretical assumptions are more idealized than the experiments. More realistic distractor experiments are needed.
  2. The choice of Δ\Delta is an additional hyperparameter, but its effect is not studied. An ablation is needed.
Questions for authors
  1. The counterexample in Section 2 seems sound; my concern is narrower. For example, Lemma 3(ii) appears to claim that the optimum is attained whenever σK2\sigma_K^2 is an eigenvalue of T0T0∗T_0T_0^*. Does attainment require enough eigenvectors at the threshold?
  2. It is not clear on how to train m1m_1? More clarification (how) and demonstrations (why) are needed.
  3. Although the authors mention the limitation of Δ\Delta measurement, its ablation study is needed.
  4. It is not clear why CSP underperforms on clean PushT, and why its high position-probe R2R^2 (0.85) does not lead to better planning performance in textured PushT (3.3% success rate). More analysis is needed to explain these results.
  5. The experiments only evaluate synthetic, episode-constant textures. Have the authors tested CSP with more natural predictable distractors?
3 comments

We thank the reviewer for their thorough feedback.


W1, Q5 More distractors

We've added 2 more distractor types:

  • Temporally evolving non-background distractors. A new experiment in Sec 4.1 with an environment containing 3 deterministic clocks, 2 irrelevant and 1 that gates the dynamics of a task signal. The distractors are not static but fully predictable. We plot the linear probe R2R^2 during training in the new Fig. 2. LeWM fills its state with the clocks (R2R^2 0.6–0.8) and loses the signal (R2→0R^2 \to 0); CSP keeps the signal (R2>0.9R^2 >0.9), does not waste state capacity on the irrelevant clocks (R2≈0.01R^2 \approx 0.01), and retains the gating clock (R2>0.85R^2 >0.85) needed to predict the signal.

  • Moving backgrounds. A new scrolling background distractor benchmark, where the texture translates at a fixed velocity (Sec 4.4). CSP matches its static-distractor performance on all three environments while the baselines degrade. Results below.

New Table 2: Two-Room Scrolling Texture

Agent (α=0.1)SR (α=0.1)Agent (α=0.2)SR (α=0.2)
LeWM0.00.290.00.48
CSP1.00.961.00.96

New Table 2: PushT Scrolling Texture

Agent (α=0.1)Block rot (α=0.1)SR (α=0.1)Agent (α=0.2)Block rot (α=0.2)SR (α=0.2)
LeWM0.00.00.050.620.090.1
CSP0.980.90.840.980.910.9

W2, Q3 Δ\Delta Ablation

We have added this ablation (new Table 3). It compares the episode-start context with a context at a fixed Δ=10,30\Delta=10,30 before the history, on clean and distracted PushT. The learned state is unaffected by the choice and all probes are identical to the main setting, clean and distracted. The theory only requires that the task variables be contingent relative to the context, and any sufficiently old frames satisfy that. The difference appears at planning time. With a fixed Δ\Delta the predictor is trained on a single context lag, whereas during rollout the context’s lag grows. SR drops with this mismatch (0.88 → 0.74 at Δ=30\Delta=30). Conditioning on the episode start exposes the predictor to every lag during training and avoids having to advance the context during planning, which is why the main experiments use it. The revised paper is now explicit about this choice.

New Table 3: PushT Δ Ablation

Agent (α=0.0)Block pos (α=0.0)Block rot (α=0.0)SR (α=0.0)Agent (α=0.2)Block pos (α=0.2)Block rot (α=0.2)SR (α=0.2)
Δ=100.980.970.940.880.980.970.890.88
Δ=300.980.960.940.740.980.970.940.76

Q1 Attainment at the threshold

The reviewer is right. Attainment of the KK-dimensional optimum requires that the eigenvalues of T0T0∗T_0 T_0^\ast greater than or equal to σK2\sigma_K^2, counted with multiplicity, number at least KK. It is not enough for σK2\sigma_K^2 to be an eigenvalue if its eigenspace is too small to fill the remaining slots. We have restated Lemma 3(ii) in this form.


Q2 Training clarification

We have revised Sec 3 to better explain the role of mm:

  • How: mm estimates E[Ψ∣c]\mathbb{E}[\Psi \mid c], the context-predictable part of the embedding. It is a 2-layer transformer from the (detached) context embeddings to a KK-dim vector, trained by least squares, with its own optimizer. The contingent state Ψ−m\boldsymbol{\Psi} - m enters the whitening term and the predictor’s inputs and targets; mm receives no gradient from either. At planning time mm is computed once and applied to every rollout step and to the goal.
  • Why: The theory ranks features by what became predictable since the context, which requires subtracting what was predictable from the context. mm is that subtraction. We have included an m≡0m\equiv 0 ablation in the new Table 3 showing that performance collapses without it.

Q4 Underperformance on PushT

This was a general assessment of the submitted numbers. A new ablation (Table 3) shows that no constant λ\lambda is optimal on both clean and distracted data. Small λ\lambda keep clean performance but may not remove the distractor, while larger λ\lambda remove the distractor but cap the task variables. By annealing λ\lambda, from 1.5→0.0051.5 \to 0.005, identically across all environments, CSP’s distracted numbers match its clean numbers everywhere and the clean numbers improve. New results below.

Updated Table 1: Two-Room

Agent (α=0.0)SR (α=0.0)Agent (α=0.2)SR (α=0.2)
LeWM0.980.860.00.24
CSP1.00.961.00.96

Updated Table 1: PushT

Agent (α=0.0)Block rot (α=0.0)SR (α=0.0)Agent (α=0.2)Block rot (α=0.2)SR (α=0.2)
LeWM0.920.930.860.00.00.06
CSP0.980.910.880.970.880.92

Full updated results in Table 1 of the paper. The constant λ=0.05\lambda=0.05 row in new Table 3 (ablations) reproduces the initial submission’s results.

Thank you for the clarifications and additional experiments. They address my concerns. The paper is solid and informative, but I will keep my score unchanged.

We sincerely thank the reviewer for the feedback and for taking the time to engage with our paper. We are glad that the additional experiments and clarifications addressed your concerns.

Reviewer_LrBzOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper examines the tendency of latent self-predictive world models to represent predictable but task-irrelevant features. Further, this work shows theoretically that deterministic distractors can dominate self-predictive objectives inspite of whitening or Gaussian regularization.

The paper proposes Contingent Self-Prediction (CSP), which distinguishes recent features and actions from older observations. CSP is then used to selectively enforce non-collapse constraints to features unpredictable using just older context and observations.

The paper then examines this empirically in TwoRoom, PushT, Cube, and Reacher, where it shows improved retention of task information with a static background and even improved planning in some environments.

Critical strengths
  1. Instead of using task-specific controls like reward or simulator-state supervision, the proposed solution relies on the existing temporal structure in the observation space and environment.
  2. According to the empirical study, under static background noise, CSP retains substantial task information.
Critical weaknesses
  1. Empirical results do not demonstrate substantial improvement in planning.
Questions for authors
  1. How does CSP perform with temporally evolving but predictable distractors (such as a clock or calendar) given that they are irrelevant to the task?
1 comment

We sincerely thank the reviewer for their feedback.


W1 Planning improvement

This was a fair point regarding the submitted numbers and tracing it down led to the main change in the revision. The submission trained CSP with a constant regularization λ\lambda. A new ablation (Sec 4.5, Table 3) shows that small λ\lambda yields good clean performance but may fail to remove distractors, while larger λ\lambda removes the distractor but caps the task variables. By annealing λ\lambda identically across all environments CSP’s distracted numbers (α=0.2\alpha=0.2) now match its clean numbers (α=0.0\alpha=0.0) everywhere and the clean numbers improve, as shown below.

Updated Table 1: Two-Room

Agent (α=0.0)SR (α=0.0)Agent (α=0.2)SR (α=0.2)
LeWM0.980.860.00.24
CSP1.00.961.00.96

Updated Table 1: PushT

Agent (α=0.0)Block rot (α=0.0)SR (α=0.0)Agent (α=0.2)Block rot (α=0.2)SR (α=0.2)
LeWM0.920.930.860.00.00.06
CSP0.980.910.880.970.880.92

Updated Table 1: Reacher

Finger (α=0.0)q (α=0.0)SR (α=0.0)Finger (α=0.2)q (α=0.2)SR (α=0.2)
LeWM1.001.000.860.00.00.14
CSP1.001.000.881.01.00.88

Updated Table 1: Cube

Eff pos (α=0.0)Block pos (α=0.0)SR (α=0.0)Eff pos (α=0.2)Block pos (α=0.2)SR (α=0.2)
LeWM0.980.990.740.00.00.44
CSP0.990.980.780.990.80.7

The constant λ=0.05\lambda=0.05 row in new Table 3 (ablations) reproduces the initial submission’s results.


Q1 Temporally evolving, predictable distractors such as a clock

  • This is exactly the setting of the new synthetic experiment in Sec 4.1. The latent state contains three deterministic clocks. Two are irrelevant to everything, and the third flips the sign of the dynamics of the task signal, so it is predictable, predetermined, and relevant. We plot the linear probe R2R^2 during training in Fig 2. LeWM fills its state with the clocks (R2R^2 0.6–0.8) and loses the signal (R2→0R^2 \to 0); CSP keeps the signal (R2>0.9R^2 >0.9), spends nothing on the irrelevant clocks (R2→0.01R^2 \to 0.01), and retains the gating clock (R2>0.85R^2 >0.85) needed to predict the signal.

  • At the pixel level, the new scrolling-texture benchmark (Sec 4.4, Table 2) is the moving analogue. A background texture advances deterministically every frame, and CSP matches its static-texture performance on all three environments while baselines degrade. Results below.

New Table 2: Two-Room Scrolling Texture

Agent (α=0.1)SR (α=0.1)Agent (α=0.2)SR (α=0.2)
LeWM0.00.290.00.48
CSP1.00.961.00.96

New Table 2: PushT Scrolling Texture

Agent (α=0.1)Block rot (α=0.1)SR (α=0.1)Agent (α=0.2)Block rot (α=0.2)SR (α=0.2)
LeWM0.00.00.050.620.090.1
CSP0.980.90.840.980.910.9

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Changes

We thank the reviewers for their careful reading. All 3 reviews converged on the same points: the planning results did not yet show a clear advantage (LrBz, cEY8), several design choices were un-ablated (cEY8, YnBU), and the distractors were all of the same kind (all). We have addressed these with a new set of results, ablations and experiments.


1 Improved CSP results (LrBz, cEY8)

The submission trained CSP with a constant regularization λ\lambda, which yielded marginal planning gains. A new ablation (Sec 4.5, Table 3) shows why. A small λ\lambda keeps clean performance but may be insufficient to remove the distractors while large λ\lambda removes the distractors but caps the task variables. By annealing λ\lambda, identically across all environments, CSP’s distracted numbers (α=0.2\alpha=0.2) now match its clean numbers (α=0.0\alpha=0.0) everywhere and the clean numbers improve, as shown below.

Updated Table 1: Two-Room

Agent (α=0.0)SR (α=0.0)Agent (α=0.2)SR (α=0.2)
LeWM0.980.860.00.24
CSP1.00.961.00.96

Updated Table 1: PushT

Agent (α=0.0)Block rot (α=0.0)SR (α=0.0)Agent (α=0.2)Block rot (α=0.2)SR (α=0.2)
LeWM0.920.930.860.00.00.06
CSP0.980.910.880.970.880.92

Updated Table 1: Reacher

Finger (α=0.0)q (α=0.0)SR (α=0.0)Finger (α=0.2)q (α=0.2)SR (α=0.2)
LeWM1.001.000.860.00.00.14
CSP1.001.000.881.01.00.88

Updated Table 1: Cube

Eff pos (α=0.0)Block pos (α=0.0)SR (α=0.0)Eff pos (α=0.2)Block pos (α=0.2)SR (α=0.2)
LeWM0.980.990.740.00.00.44
CSP0.990.980.780.990.80.7

The constant λ=0.05\lambda=0.05 row in new Table 3 (ablations) reproduces the initial submission’s results.


2 Additional distractor experiments (LrBz, cEY8, YnBU)

We added new predictable distractors:

  • Predictable clocks. A new experiment with temporally evolving non-background distractors in Sec 4.1, corresponding to the clock/calendar case raised by LrBz. The environment contains 3 deterministic clocks, 2 irrelevant and 1 that gates the dynamics of a task signal. We plot the linear probe R2R^2 during training in the new Fig. 2. LeWM fills its state with the clocks (R2R^2 0.6–0.8) and loses the signal (R2→0R^2 \to 0); CSP keeps the signal (R2>0.9R^2 >0.9), spends nothing on the irrelevant clocks (R2→0.01R^2 \to 0.01), and retains the gating clock (R2>0.85R^2 >0.85) needed to predict the signal.

  • Moving backgrounds. A new scrolling background distractor benchmark, where the texture translates at a fixed velocity throughout the episode (Sec 4.4). CSP matches its static-distractor performance on all three environments while the baselines degrade. Results below.

New Table 2: Two-Room Scrolling Texture

Agent (α=0.1)SR (α=0.1)Agent (α=0.2)SR (α=0.2)
LeWM0.00.290.00.48
CSP1.00.961.00.96

New Table 2: PushT Scrolling Texture

Agent (α=0.1)Block rot (α=0.1)SR (α=0.1)Agent (α=0.2)Block rot (α=0.2)SR (α=0.2)
LeWM0.00.00.050.620.090.1
CSP0.980.90.840.980.910.9

3 Choice of context (YnBU, cEY8)

Two reviewers asked about how the context is set (fixed at the episode start) and requested an ablation of Δ\Delta. We added it to the new Table 3. Conditioning on frames a fixed lag Δ=10,30\Delta=10,30 before the history leaves the learned state mostly unchanged, as the theory predicts, since it requires only that the task variables be contingent relative to the context. The lag (Δ\Delta) matters only at planning time, where a fixed-lag predictor sees context lags it never saw during training and SR drops. Conditioning on the episode start (as in our main experiments) exposes the predictor to every lag during training and removes the mismatch, avoiding the need to advance the context during rollouts. The revised text now explains this.

New Table 3: PushT Δ Ablation

Agent (α=0.0)Block pos (α=0.0)Block rot (α=0.0)SR (α=0.0)Agent (α=0.2)Block pos (α=0.2)Block rot (α=0.2)SR (α=0.2)
Δ=100.980.970.940.880.980.970.890.88
Δ=300.980.960.940.740.980.970.940.76

4 Writing revisions (YnBU, cEY8)

  • Sec 2 now introduces the operator through its classical counterpart and ties each object to its world-model instantiation (YnBU)
  • Sec 3 now gives the full training procedure, including how the regressor is fit and why (YnBU, cEY8).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.