- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2(+1)
- Soundness
- 2
Read review
Paper summary
World models learn representations by predicting the future from data collected by some behavior policy, and that policy is usually described only by how often each action occurs. Recent theory for prediction-based representation learning says when such learning recovers the true state up to a rotation, but it leaves open whether two policies that look identical under the usual coverage summaries can lead to different learned representations. This paper argues that action frequencies leave out something important: the order in which actions are combined over time can decide which representation the learning objective prefers. In both cases the representation keeps all of the state information. What changes is whether the state appears in simple linear coordinates or in a bent, nonlinear form.
The authors study an idealized world with Gaussian states and noisy linear effects of actions. There they prove that one number decides the outcome: how much the least predictable linear direction of the state beats the most predictable quadratic feature in predictability. If this gap is positive, the best representation is a rotation of the state and nearly optimal ones stay close to it. If it is negative, a slightly bent but still invertible representation scores higher. They construct two policies that agree on action frequencies, on all statistics over any three time steps, and on all linear predictive structure, yet fall on opposite sides of the boundary. They also show that the gap can be estimated from paired simulated futures and used to design a policy whose gap is positive. In a small rendered-image version of the same world, a convolutional encoder learns the predicted linear or nonlinear coordinates in every paired run. Those coordinates persist after a world-model predictor is attached and trained, and they change which action sequence a simple planner picks.
Critical strengths
- Action order alone can change the learned coordinates, shown with all lower-order action statistics matched. Policies are normally summarized by how often each action occurs. The paper shows that the order in which actions are combined can, by itself, decide whether prediction-based learning recovers the state in simple linear coordinates or in a nonlinear form, even though both keep all of the state information.
- The theory correctly predicts which coordinates a trained CNN encoder learns. The outcome is decided by the sign of a single gap, which can be estimated from paired simulated futures and used to design a policy on the favourable side.
Critical weaknesses
-
The experiments use the theorem's own world and training objective, not standard world-model training. Every neural experiment runs in the exact two-dimensional Gaussian world. Every one also trains on two independent futures from the same reset state, an objective that needs a resettable simulator. The LeWorldModel stage starts from encoders already selected under this objective and keeps it as the main encoder loss (Table 5). §5 concedes that ordinary single-future training from scratch is untested. Theorem 1 does cover single-future training with a perfect predictor, so what is missing is evidence that practical training reaches the predicted solution. The title and abstract ("predictive objectives", "LeWorldModel") claim more than this shows.
The two action laws are also hand-built to match many statistics, and no ordinary policy (for example, one that tends to repeat rather than alternate actions) is shown to cross the boundary. So it is unclear whether the effect arises under behavior policies used in practice.
-
The planning experiment favours linear coordinates through the way it is scored. The planner picks the action sequence whose predicted representation is closest to the goal's representation in squared Euclidean distance. Regret is then measured by squared Euclidean distance in the true state (§4.3).
- When the representation is a rotation of the state, the two distances are equal. Under any other invertible map they differ.
- So the nonlinear coordinates learned under the negative-gap law must have higher regret even if they predict the future equally well. The authors themselves call this "surrogate geometry, not predictor incapacity".
- The regret gap of 0.293 therefore re-measures that the learned coordinates are nonlinear. It does not show a separate cost to the task.
-
The experiments section is hard to follow. Much of it records how runs were produced rather than what they show: development versus held-out seed pairs, which checkpoints were reused, and when labels were revealed (e.g. "three disclosed development pairs and then seven frozen held-out pairs, not a post-hoc subset"). It also uses several terms before defining them, such as "ordered word", "neutral fiber" and "surrogate geometry" (abstract, Secs. 3.4 and 4.3). This hides what each experiment tests, which matters because the experiments are where the paper must show that the effect reaches ordinary world-model training. The bookkeeping belongs in the appendix.
Questions for authors
- (Weakness 1: training objective) Suppose the encoder and the LeWorldModel predictor are trained together from random initialization, using only the standard single-future prediction loss and the same normalization. Do the two action laws still produce state coordinates and nonlinear coordinates, respectively?
- If they do, the title claim about predictive objectives would be supported, and my main concern would largely go away.
- If the difference disappears, the effect would be specific to the two-future objective, which needs a resettable simulator.
- (Weakness 1: hand-built action laws) Does any ordinary policy family with fixed action frequencies cross the zero-gap boundary, for example a Markov chain with a tunable tendency to repeat the previous action? How large is the gap when the state has more than two dimensions? If the sign flips only for laws engineered to match many statistics, the claim that behavior design matters for representation learning holds mainly in principle.
- (Weakness 2: planning metric) Can you report a downstream measure that does not automatically favour a representation that is a rotation of the state? Examples are planning with a cost learned in representation space, or task success under closed-loop replanning.
- A clear gap on such a measure would show a real task cost of the nonlinear coordinates.
- No gap would mean the planning result only restates that the learned coordinates differ.
8 comments
Response to Reviewer N7Su (2/3): Ordinary policies and higher-dimensional margins
Weakness 1: hand-built action laws — An ordinary repeat-versus-switch policy
Does any ordinary policy family with fixed action frequencies cross the zero-gap boundary, for example a Markov chain with a tunable tendency to repeat the previous action?
Yes. We evaluated precisely this Markov family. The first action is uniform; each subsequent action repeats its predecessor with probability p. Every action marginal remains uniform. In two dimensions, the population margin is 0.0341 at p = 0.3 and -0.0453 at p = 0.7, with both signs certified by exact rational arithmetic (Section 3.6, Table 1; Appendix M.1).
The ordinary policy also changes low-order correlations and linear predictability. It establishes boundary crossing in a familiar policy family, while the matched five-action construction separately isolates changes beyond those summaries.
Weakness 1: hand-built action laws — Margins beyond two dimensions
How large is the gap when the state has more than two dimensions?
The crossing persists in coupled four- and eight-dimensional examples. At coupling 0.5:
| State dimension | Margin at p = 0.3 | Margin at p = 0.7 |
|---|---|---|
| 4 | 0.0078 | -0.0692 |
| 8 | 0.0045 | -0.0789 |
The full coupling scan is reported in Appendix M.1, including stronger coupling where both endpoints are negative. These are population operator calculations, complementary to image training. They show that the policy and dynamics jointly determine the boundary, rather than a dimension-independent guarantee.
Response to Reviewer N7Su (3/3): Planning metric, task success, and readability
Weakness 2: planning metric — What the planning regret establishes
The regret gap of 0.293 therefore re-measures that the learned coordinates are nonlinear. It does not show a separate cost to the task.
We agree that the fixed-cost ranking experiment does not establish a task-independent disadvantage of nonlinear coordinates. It measures how coordinate geometry changes action ranking under a fixed Euclidean goal cost. Section 4.3 now makes this interpretation explicit: regret is 0.073 versus 0.366, with paired gap 0.293 (95% CI [0.250, 0.336]) across seven paired runs (Figure 4; Appendix N.3). Nonlinearity alone does not force worse ranking for every task.
We separately test capacity-dependent accessibility. For jointly learned single-future encoders, nonlinear readouts reduce the first two affine state-recovery gaps from 0.500 and 0.501 to 0.088 and 0.156; the third pair is linearly recoverable under both laws (Section 4.2, Figure 3; Appendix N.1, Table 7). This supports the finite-model consequence without treating readout accuracy as task success.
Weakness 2: planning metric — An independent task measure under closed-loop replanning
Can you report a downstream measure that does not automatically favour a representation that is a rotation of the state?
Examples are planning with a cost learned in representation space, or task success under closed-loop replanning.
We added the closed-loop task-success evaluation you suggested, in the public PushT simulator. Success uses the physical position-and-angle criterion, independently of latent coordinates. With 50 controls and replanning every 25 controls, results on the same 32 tasks are:
| Model / collection | Terminal success |
|---|---|
| Public pretrained checkpoint | 13 / 32 |
| From scratch: independent actions | 0 / 32 |
| From scratch: autoregressive actions | 0 / 32 |
The public checkpoint also achieves 20 first hits. The matched scratch models use the native 192-dimensional architecture and SIGReg; the 32 tasks are evaluation cases, not training repeats. This establishes image-based task feasibility, while a matched collection-policy improvement in task success is not yet established. The public checkpoint has a different training history and cannot supply that causal comparison (Section 4.5; Appendix N.4 and O.5).
7. Experimental organization and terminology
Much of it records how runs were produced rather than what they show:
It also uses several terms before defining them, such as "ordered word", "neutral fiber" and "surrogate geometry"
We rewrote Section 4 around the scientific comparisons: joint learning, state accessibility, action choice, behavior design, and closed-loop evaluation. The Introduction defines an action word as an ordered action sequence. The abstract and experimental narrative use direct descriptions instead of the cited shorthand; model and training details are consolidated in Appendix O. Figures 2-5 present the corresponding findings and comparisons. Thank you for identifying what made the experimental argument difficult to follow.
Response to Reviewer N7Su (1/3): Joint single-future learning and claim scope
Thank you for the concrete tests and for recognizing the matched-policy result. We have uploaded a revised manuscript. We answer the single-future question here, ordinary policies and dimension in our second comment, and the planning metric and presentation in our third.
Weakness 1: training objective — Joint single-future training from random initialization
Yes, in two of three paired initializations. We jointly train a random image CNN and the released LeWorldModel predictor core. Each example contains one starting image, five ordered actions, and one noisy future image. Training uses squared prediction error with differentiable covariance normalization, without state labels, replica loss, or a preselected encoder. The two laws share initial weights within each pair.
At 24,000 encoder updates, affine state-recovery R-squared on shared evaluation states is:
In the two separating pairs, quadratic features explain about 38% of negative-law representation variance, versus less than 0.4% under the positive law. The third pair recovers state under both laws. All pairs and both evaluated checkpoints are reported (Sections 4.1-4.2, Figures 2-3; Appendix N.1, Table 7). The table uses shared evaluation; Figure 2 uses the separate raw diagnostic specified in Appendix O.2.
Weakness 1: training objective — Population theory and the practical training claim
The new experiment supplies a practical discovery test; the theorem remains a population-optimum statement. The predictor core is adapted to complete five-action sequences and a two-dimensional embedding, using covariance normalization. This tests the whitening-based setting, rather than claiming the full native 192-dimensional SIGReg recipe. The Abstract and Section 4.1 now identify the adaptation, single-future training, and two-of-three outcome explicitly; Appendix O.1 gives the architecture. The result shows discovery is possible under joint single-future optimization, with initialization dependence.