- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper asks whether the component that dominates a latent world model's prediction error (MSE) is the one whose repair most improves action selection. From identical physical starts, the authors execute multiple candidate action sequences and exactly decompose the endpoint latent MSE into a center error (the component shared by all candidates, i.e., the deviation of the candidate-pool mean) and an action-relative response error (each candidate's deviation from the pool mean). The response error is further split into direction and scale: Q = (1−c²) + (g−c)². On this basis, the paper designs two oracle diagnostics that use realized futures: the P/E comparison — scoring identical candidates by the cost of predicted outcomes (P) versus encoded realized outcomes (E), localizing whether physical ordering is lost at the representation/goal-metric stage or at the prediction stage; and the C/S repair comparison — replacing only the center (C) or only the responses (S) with ground truth, to compare repair priority.
Experiments focus on LeWM, with a new-start confirmation on four additional model families across four tasks: center error dominates MSE in 14/16 model–task cells, yet in six of these cells S yields better physical ranking and elite quality than C while leaving more residual MSE. The P/E study shows that for LeWM, encoding realized outcomes raises the physical Spearman from 0.464/0.193 to 0.975/0.631 on Reacher (with a render-matched goal)/PushT, while on Cube even E is uninformative. A 72-run objective study shows that the scale–direction identity allows squared error to reward response shrinkage — an effect clearly observed on PushT at long horizons, while the other tasks improve direction — and incremental closed-loop CEM gains remain unconfirmed.
Critical strengths
- A rigorous diagnostic framework and experimental design. Paired candidates restored from identical complete physical states, the exact center/response decomposition, and the P/C/S/E 2×2 grid separate error magnitude from decision effect and measure each independently. The separation of exploratory samples from confirmatory new starts, together with family-wise multiplicity correction, strengthens the credibility of the conclusions.
Critical weaknesses
-
Limited information content and practical utility of the central finding. C imports a single true pool-mean vector, whereas S imports the true responses of all K candidates; the two oracles differ greatly in information budget, no equal-information control is provided anywhere in the paper, and an advantage of S for ranking is partly expected. Moreover, none of the paper's trainable interventions appears to yield a confirmed closed-loop gain. Since these diagnostics cannot predict in advance which settings deserve response repair, their guidance for model design and for improving actual performance is limited.
-
Poor readability and limited external validity. The text has a very high terminology density (P/C/S/E, SHAPE/AMP, Q/g/c, direction floor/amplitude excess), and parts are difficult to follow. Experimentally, all models score the same candidate pool proposed by LeWM's CEM rather than candidates from each model's native planner, and evaluation is limited to four simulated benchmarks from the LeWM suite — both restricting generalization of the conclusions to realistic planning settings.
Questions for authors
-
The SHAPE/AMP oracles already separate response configuration from energy. Could you add oracle controls that interpolate the information content between AMP and SHAPE/S — e.g., a rank-1 low-rank response repair, or repairing only the principal response direction? This would answer how much true response information is needed for the S advantage to appear.
-
On Cube the E cost has near-zero physical correlation. Does this stem from insufficient encoder representations, goal-observation mismatch, or the physical proxy (block position) itself? Could ordering be recovered with a goal-conditioned metric or an alternative physical proxy?
-
All incremental closed-loop CEM contrasts are unresolved. Have you conducted a power analysis indicating the minimum effect size detectable with 256 episodes? With more episodes, could the response-level improvements convert into a confirmable closed-loop gain?