- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The manuscript identifies a failure mode of self-predictive world models: (in short) when the environment has a strong predictable factor, an optimal learned state of the history is entirely that factor and this can't be solved with standard non-collapse fixes. The proposed fix is to apply anti-collapse regularization only to parts of the state that were not foreseeable from history. Experiments that appear to validate the theoretical insights are conducted.
Critical strengths
- The problem the paper focuses on is very relevant for training world models. The theoretical work done in the paper is non-trivial and yields a proposed fix (admittedly under moderately strong assumptions, but maintaining enough generality to be useful).
Critical weaknesses
- The experiment departs from the theory in non-trivial ways. The context is fixed once (episode-start frames, both during training and at planning time) and reused unchanged across the whole 25-step rollouts. The context never slides forward as planning progresses, even though the theory's sliding-window picture assumes it would. This leaves me unsure the experiments are actually validating the theoretical insights (e.g., your Eq 11 in this case, could it not be simply doing "background subtraction" on the regularization term, which would explain your results?).
- The experiment "distractors" are highly artificial and of a single kind (textured backgrounds). This makes it hard to assess whether the theoretical formalism actually transfers to situations that are messier or with any other types of distractors.
Questions for authors
- The paper is generally well-written, but it required more effort than usual to keep the connection of the stated goals in mind while reading through the technical sections. The writing also leaves implicit the domains of key objects (e.g., the domains of , , in section 2) and how they relate to applied world models. Adding a little bit more exposition would make the manuscript more amenable to readers who are not deep into operator/spectral theory of stochastic processes.
- What are the most relevant works from pre-2020 literature? The operator-theoretic framework in Section 2 parallels classical subspace system identification works (e.g., Larimore, 1990; Van Overschee & De Moor, 1994, Akaike 1974, etc.). Is the specific collapse/impossibility result a genuinely new finding within existing frameworks, and why does the manuscript not discuss standard literature?
- (Optional, would strengthen but not required) Have you verified the main proofs with a proof checker?
We thank the reviewer for all the questions and feedback. We note that we've updated the main table with improved results on all 4 environments (see Summary of Changes).
W1 Context not sliding
The theory defines the contingent state for any set of old frames relative to which the task variables are contingent, not necessarily a window that advances with the rollout. While our experiments set the context to the first frames of the episode, we now ablate Δ in new Table 3 setting Δ=10,30. The representation is mostly unchanged. What the lag affects is planning (SR 0.88 → 0.74 from Δ=10→30). With a fixed lag the predictor only ever sees a context at Δ steps during training, while at rollout the context is fixed and its lag grows. Episode-start training exposes the predictor to every lag during training, so every rollout lag is in-distribution and the context does not need to slide. The paper is now explicit about why we made this choice.
New Table 3: PushT Δ Ablation
W1 Background subtraction
For a static texture, deflation by E[Ψ∣c] should be tantamount to a background subtraction in embedding space. However, CSP also removes predetermined content that is not static, as shown by the new experiments below, which better validate the theory.
W2 One kind of distractors
We added 2 more distractor types:
Temporally evolving non-background distractors. A new experiment in Sec 4.1 in an environment containing 3 deterministic clocks, 2 irrelevant and 1 that gates the dynamics of a task signal. The distractors are not static and fully predictable. We plot the linear probe R2 during training in the new Fig 2. LeWM fills its state with the clocks (R2 0.6–0.8) and loses the signal (R2→0); CSP keeps the signal (R2>0.9), does not waste state capacity on the irrelevant clocks (R2≈0.01), and retains the gating clock (R2>0.85) needed to predict the signal.
Moving backgrounds. A new scrolling background distractor benchmark, where the texture translates at a fixed velocity throughout the episode (Sec 4.4). CSP matches its static-distractor performance on all three environments while the baselines degrade.
New Table 2: Two-Room Scrolling Texture
New Table 2: PushT Scrolling Texture
Q1 Exposition
We have revised Sec 2 to ensure each object is given its non-operator, world-model meaning when it is introduced.
Q2 Pre-2020 literature
The operator Tf=E[f(future)∣past] and its SVD are indeed Akaike’s canonical correlation analysis (CCA) between past and future (1974), and CVA (Larimore) and N4SID (Van Overschee & De Moor) are its linear-Gaussian algorithms. They pick the state as the leading canonical directions of the past-future correlation, thus also ranking by predictability. The same idea also arises in spectral Predictive State Representations. The revised paper now introduces T this way and cites these works in Sec 2. While the predictability-based feature ranking principle is classical, our contributions are:
Our operator TΔf=E[f∣h,a]−E[f∣c] partials out an old context c and ranks by what became predictable since c.
The collapse result is about learned nonlinear features with an anti-collapse constraint. In the linear-Gaussian setting with a known state dimension, deterministic components are a separate, finite-dimensional block that the classical algorithms handle explicitly. With a learned nonlinear feature map, the predetermined content is a flat block of unbounded dimension (any function of a deterministic factor is perfectly predictable), and a finite budget of whitened features is filled by it at zero cost. Theorem 2 shows the optimum is a function of the predetermined factor alone.
The impossibility result targets how modern self-supervised methods prevent collapse by constraining the marginal law of the state (whitening, VICReg, SIGReg). Theorem 3 shows that for any target law Q there is a state that is exactly Q-distributed, fully predictable and task-blind. Thus, no constraint on the marginal can rule the degenerate solution out.
Q3 Proof checker
We appreciate the suggestion! We will add Lean proofs to the repository.