- Rating
- 3
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies representation selection in end-to-end feature-predictive world models, where the prediction loss updates both the predictor and its future-feature target encoder. It formalizes “predictability bias”: under finite capacity and stated structural assumptions, optimizing prediction can favor predictable but task-irrelevant factors. The authors analyze how explicit target supervision changes this allocation and identify conditions under which it improves alignment between the native planning score and task cost. Controlled image experiments and an action-conditioned TwoRoom study test the representation effect. Finally, using simulator-state supervision during training, the paper reports improved mean closed-loop success on Cube, PushT, TwoRoom, and Reacher with the original latent-goal planner unchanged at deployment.
Critical strengths
-
Mechanism is isolated in a tractable setting. Theorem 1 states its finite-capacity and factor-separation assumptions, and the exact-whitening CNN varies temporal persistence without changing the single-frame image distribution. The reversal of factor preference and the supervision sweep support the proposed selection mechanism in this controlled regime.
-
The downstream comparison is unusually well controlled. The four-task study uses matched initializations, held-out roots, CEM random streams, and the unchanged deployment score, as well as paired seed-level results are reported. The authors also report negative or mixed diagnostics rather than treating improved success as proof of a single planner mechanism.
Critical weaknesses
-
The theoretical control link is not established for the deployed model. Theorem 1 and Proposition 2 analyze whitened representations, whereas the native checkpoints have anisotropic covariance. The monotonic score-correlation result in Proposition 3 further requires a quadratic projection reward, a target operator proportional to the task-cost operator, and an isotropic Gaussian endpoint residual. These are distinct assumptions, and the latter conditions are not verified in the four-task study. Table 14 tests a changed candidate score, but does not measure native score–physical-cost alignment on common candidates. The theory therefore identifies a possible control pathway, not an established explanation for Table 2.
-
The closed-loop gain is not causally tied to correcting predictability bias. The action-conditioned TwoRoom test does find greater position energy with true than random labels (Appendix B.3, Table 6). Yet random labels also attenuate the persistence response, the observed-latent probes do not resolve a true-label advantage over random labels, and this 500-update intervention does not establish better candidate selection or control. Fixed-candidate tests using other checkpoints likewise show no consistent ranking advantage. These results do not negate the ten-epoch four-task success gains, but leave their proposed representation-selection mediator unverified.
Questions for authors
-
Some questions in Weaknesses.
-
The authors systematically rule out several candidate mediators (fixed-candidate ranking, final-pool selection, endpoint distance, velocity swap) yet fail to identify a shared mediator. What do you think the "unmeasured common mediator" is most likely to be? Is there a plan, e.g., structural interventions on the latent space, or experiments explicitly varying a single factor's energy, to further close this chain?
-
Table 14 changes the candidate score on the unified checkpoints, while Appendix C tests fixed candidates using different checkpoints. For the exact Table 2 checkpoints, can the authors evaluate identical action pools from identical initial states and goals, and report how each model’s unchanged native score ranks physical task cost alongside a measure of retained task state?
-
In the matched action-conditioned TwoRoom persistence test, how does the factor-energy response change if prediction gradients into the future-target branch are stopped via stop-gradient or the target is supplied by a moving-average encoder? Please specify which encoder update paths remain active, since a shared online encoder can still change through the input branch.
-
The native factor test uses a fixed 192-dimensional latent, so it does not isolate how much the proposed selection effect depends on available capacity. With otherwise matched TwoRoom persistence interventions at several latent widths, how do background and position energies, state readability, and the effect of target supervision change?
5 comments
Response to questions:
Q3:For the exact Table 2 checkpoints, can the authors evaluate identical action pools from identical initial states and goals, and report how each model’s unchanged native score ranks physical task cost alongside a measure of retained task state?
Ans for Q3: We thank the reviewer for this valuable suggestion. Under identical initial states, goals, and candidate action pools, we evaluated the rank correlation between the native latent score and the ground-truth simulator task cost. In PushT and TwoRoom, the method simultaneously exhibits lower state readout error and superior native ranking compared to . In Cube, the improvement in state readout is not accompanied by a clear gain in ranking correlation; whereas in Reacher, while the ground-truth encoding readout improves slightly, both predicted readout error and ranking correlation degrade. These results suggest that the mere linear decodability of task-relevant information from ground-truth observation encodings does not suffice to guarantee accurate forward prediction or downstream ranking alignment.
| Task | Native Score–True Cost Correlation ↑ | Ground-Truth Future Encoding State Error ↓ | Predicted Encoding State Error ↓ |
|---|---|---|---|
| Cube | 0.0352 → 0.0397 | 0.1105 → 0.0536 | 0.1108 → 0.0759 |
| PushT | 0.3821 → 0.6650 | 0.3734 → 0.1471 | 0.3853 → 0.2431 |
| TwoRoom | 0.1826 → 0.2472 | 0.0700 → 0.0057 | 0.1680 → 0.1453 |
| Reacher | 0.7071 → 0.6601 | 0.6517 → 0.6287 | 0.6383 → 0.6572 |
Q4: In the matched action-conditioned TwoRoom persistence test, how does the factor-energy response change if prediction gradients into the future-target branch are stopped via stop-gradient or the target is supplied by a moving-average encoder? Please specify which encoder update paths remain active.
Ans for Q4: We thank the reviewer for this thoughtful comment. Under direct backpropagation, the latent prediction loss propagates gradients simultaneously through the predictive branch and the target-side future-frame encoder. Applying stop-gradient (SG) only truncates the target-side gradient of this prediction loss, while preserving the gradients flowing through the context encoder, the predictor, and the online representation's SIGReg regularization. For our method , the gradients from both the ground-truth state readout loss and the predicted state loss with respect to the online encoder are also retained. We compared three alternative configurations on the TwoRoom task—direct backpropagation to the target side, stop-gradient (SG), and an EMA target encoder—against our proposed method across four seeds. denotes the difference in the relative background energy metric between high and low environmental background persistence training conditions; a larger value indicates a stronger response induced by persistence changes.
| Method | Background Persistence Response |
|---|---|
| Original Method O | 0.7051 |
| O + stop-gradient | 0.3026 |
| O + EMA + stop-gradient | 0.2749 |
| Our Method J | 0.2325 |
Both SG and EMA successfully attenuate the background persistence response, with EMA yielding a relatively minor additional change over plain SG. Our method likewise achieves a lower mean response. While gradient truncation and target smoothing prevent encoders from overfitting to rapidly varying background noise, their inherent limitation lies in being agnostic to task requirements. While suppressing background clutter, they also risk discarding critical task-relevant environmental dynamics. In contrast, our method achieves superior performance while remaining directly targeted toward task-relevant states.
Q5: With otherwise matched TwoRoom persistence interventions at several latent widths, how do background and position energies, state readability, and the effect of target supervision change?
Ans for Q5: We thank the reviewer for this valuable comment. To further investigate the effect of latent capacity, we supplemented our TwoRoom evaluation with a multi-dimensional comparison across three latent dimensions, using four seeds per configuration under identical settings:
| Latent Dimension | Background Energy Share | Position Readout NRMSE ↓ |
|---|---|---|
| 96 | 1.61% → 20.76% | 0.18910 → 0.00643 |
| 192 | 11.85% → 37.77% | 0.08260 → 0.00619 |
| 384 | 11.01% → 14.79% | 0.04675 → 0.00533 |
| Latent Dimension | O Success Rate | J Success Rate | J − O [95% Paired CI] |
|---|---|---|---|
| 96 | 81.50% | 91.25% | +9.75 [5.77, 13.73] percentage points |
| 192 | 80.00% | 91.25% | +11.25 [7.07, 15.43] percentage points |
| 384 | 79.00% | 90.25% | +11.25 [7.72, 14.78] percentage points |
As the representation dimensionality of the unsupervised baseline increases from 96 to 384, the ground-truth state readout NRMSE drops from 0.18910 to 0.04675, confirming that scaling up latent capacity improves linear state decodability. However, closed-loop success rates do not increase synchronously. Meanwhile, our method consistently achieves performance gains of +9.75, +11.25, and +11.25 percentage points across all three dimensionalities. These findings demonstrate that within this experimental regime, merely increasing representation capacity enhances passive information decodability without securing downstream control gains.
Response to weaknesses:
Ans for W1&Q2: As we explicitly noted when analyzing our theoretical boundaries, the whitening assumption in our theoretical framework is specifically utilized to establish exact mathematical identities for information decomposition. In practical deployments, strict whitening no longer holds for real models. However, the core mechanisms revealed by our theory, namely, the capacity upper bound under finite capacity and the information capacity reallocation mechanism, continue to govern real-world behavior. Thus, the whitening assumption merely affects the explicit analytical form of the fine-grained mathematical decomposition, without undermining our core conclusions regarding representation selection and predictive bias.
To provide a clearer mechanistic analysis, we added a fixed-candidate-pool evaluation to examine the properties of the supervised variables among selected trajectories. Under identical conditions, we used the native scoring function to select the top 30 elite action sequences and compared the average true terminal distance to the goal:
Across all four tasks, the elite candidates selected by J achieve a consistently lower mean terminal distance than those selected by O. Because both models evaluate the exact same pool of candidate actions, this difference directly reflects superior alignment with the full task state in the candidates prioritized by the native latent scorer.
These findings provide direct empirical evidence for how task supervision governs planning selection, establishing a logical chain: state supervision enhances the decodability of task-relevant states, which enables the latent scoring function to select action candidates closer to the target goal, ultimately translating into higher task completion success rates.
Ans for W2: To thoroughly investigate the impact of supervision quality, we conducted additional analytical experiments and closed-loop evaluations comparing random labels against true supervision across four random seeds.
The closed-loop evaluation results demonstrate that random labels fail to reproduce the performance gains achieved by true state supervision:
Furthermore, our mechanistic comparison on the TwoRoom task across four seeds reveals that true supervision consistently achieves lower observation state readout error (NRMSE), lower 5-step-ahead (H5) predicted state readout error, and superior alignment with true downstream costs:
These empirical findings support our core claim: both the presence of genuine task-state information and its precise alignment with observations are critical factors driving effective representation learning and candidate selection. Random labels fail to enhance state decodability, and any marginal mean improvements observed on certain tasks are likely attributable to mild implicit regularization rather than true task guidance.