- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The proposed method is built upon LeWorldModel and aim to suggest a modulation of the latent space. The idea is to learn several orthogonal basis in order to project the latent vector into different sub-spaces. Then different gains are computed from the corresponding projected latents and the action to modulate each sub-latent differently. They prove that their method achieve a gain-basis coupling meaning that cross-subspace perturbation affect the modulation operator depending on cross-subspace gain differences (which are empirically high). Finally they use their modulation method in different environments and compute success rate and action probing.
Critical strengths
None
Critical weaknesses
Empirical results show no clear improvement relatively to related state of the art methods as the suggested method always fall within the same confidence interval of almost all the baseline
Questions for authors
- When and why should someone use your method rather than SD-Jepa or Sub-Jepa ?
- Section 3.1: experiments reported in Figure 4b compare success rate between the learning setting () and a settings never seen during training (). The training setting will obviously outperforms the rest, how can you conclude any meaningful property ?
- Section 3.2: I don't understand why a high error in leave-one-out interventions is a desired property, could you explain ?
- Appendix A.2: " giving " is oddly specific, where does it come from ?
- Appendix A.2: what is the purpose of the regularization of ? What happens without it ? With its parameterization I thought that was already bounded
7 comments
[1] Zhao et al. (2026). Sub-JEPA: Subspace Gaussian Regularization for Stable End-to-End World Models. arXiv:2605.09241.
[2] Yu et al. (2026). QQWorld: Quantile-Quantile Matching for World Model Regularization. arXiv:2607.28415.
[3] Maes, L., Le Lidec, Q., Scieur, D., LeCun, Y., & Balestriero, R. (2026). LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixels. arXiv:2603.19312.
[4] Perez et al. (2018). FiLM: Visual Reasoning with a General Conditioning Layer. AAAI 2018. arXiv:1709.07871.
Q1: I struggle to understand why the linear accessibility of one specific action of one specific environment represents a meaningful improvement. The theoretical and experimental findings on the Gain-Basis coupling are interesting but are done relatively to the Frozen-Haar method, not to the baseline (as far as I know) which limits its impact. How can you extend your conclusion that "the perspective from which a latent world model views its representation matters" to the other methods of the literature ?
Q2: Ok, I understand better what you are achieving with this Figure
Q3: I understand that you aim to show that there is no redundancy in subspace correction. However positive LOO can also be interpreted as a low generalization capability, indeed this intervention reminds me of dropout in neural networks where you purposely require the neural networks to have great performance with partially hidden internal representation in order to improve generalization. Is Leave-one-out metric commonly used to measure redundancy ? I can understand that low redundancy could imply positive LOO, however the reciprocal is not obvious. Are you sure that positive LOO is not caused by an undesired property ? The analysis would be easier with an information-based metric rather than prediction-based
Q4: Did you mean ? Ok, it was not clear to me that was learned, I thought that it was fixed.
Q5: What do you mean with "keeps the gates selective" ? What prevents to have with all the having the same value ? Did you do the same hyper-parameters (e.g. SIGReg coefficient) search for the setting without budget control than with budget control ? Could you report confidence interval ?
Clarification on Empirical Improvement and Experimental Interpretation
We thank the reviewer for the careful reading and specific questions.
Ours obtains the highest macro success rate among all locally evaluated methods (87.25%), +8.00 pp over LeWorldModel and +3.17 pp over Sub-JEPA, and improves over LeWorldModel on all four tasks. The error bars in Figure 3 are standard deviations across six CEM planner seeds, not confidence intervals, consistent with the evaluation and reporting protocol of Sub-JEPA (Zhao et al., 2026, Table 1) and QQWorld (Yu et al., 2026, Table 1), so they describe planner variability and overlap does not by itself indicate equal performance. The advantage is also consistent across new controlled comparisons: three additional baselines trained for the rebuttal (Q=I, capacity-matched FiLM, Non-Orthogonal) reach 84.25--84.58% macro success rate, all below Ours. Following the original LeWorldModel paper (Maes et al., 2026, Table 5), which evaluates robustness by training PushT across three independent seeds, we also did so. Ours achieves an average success rate of 94.34%, compared with 91.78% for FiLM and 91.33% for (Q=I).
Our method is particularly promising for rotation-sensitive manipulation tasks such as PushT. Figure 6 shows the clearest improvement in the linear accessibility of T-object rotation, while Figure 3 shows our largest control performance margin over competing baselines on PushT. Unlike Sub-JEPA and SD-JEPA, which use predefined subspaces, our method jointly learns where to modulate (Q) and how strongly (adaptive gains). Furthermore, beyond the choice between methods, the main contribution includes the finding itself: the perspective from which a latent world model views its representation matters, and learning it improves both control (Section 2.2) and the accessibility of the learned dynamics (Section 3.3). We hope this finding can also inspire other latent representation learning methods.
Learning the perspective matters because it determines where the gains apply:
Figure 4b does not compare λ=1 against λ<1. It compares two bases at the same λ, on the same checkpoint with identical gains. At λ=0 the gains are equal and the two bases give identical results, as Theorem 1 predicts; the gap opens only as gain differences are restored. To address the reviewer's concern, we repeated the same intervention on a separately trained Frozen Haar checkpoint, whose basis was fixed throughout training:
Replacing the basis costs 15.34 pp when the basis was learned and 3.33 pp when it was not. Familiarity with the training setting therefore explains only a small part of the gap; the rest reflects the coupling between differentiated gains and the learned directions.
Leave-one-out ablation measures the marginal predictive contribution of each subspace correction. We remove one correction while keeping the other seven active, and measure the resulting increase in prediction error: ΔkLOO=E(without k)−E(all). A positive value means that the correction contributes additional predictive value even when combined with the other seven, rather than being redundant. Thus, the desired property is a positive error increase upon removal, not a high prediction error itself. All 8 learned corrections have positive contributions, compared with 6/8 for Frozen Haar. We will clarify this interpretation in Section 3.2.
It is a learned parameter, not a chosen hyperparameter: bα=−1.41528 is the value in the reported PushT checkpoint, and α=σ(bα)=0.19540 is the resulting bound on the residual gain. We will state its origin and round the displayed value.
The bound and the budget do different jobs:
The parameterization bounds each gain, 1<gt,k<1+α, but does not prevent all gates from saturating together. The budget controls their mean activation (target ρ=0.3), which keeps the gates selective. Removing it and retraining three PushT models with matched seeds:
Without the budget, mean activation rises to 0.762, gain differentiation (CV) falls, and success rate drops by 2.22 pp. The bound limits the maximum gain; the budget limits average gate usage.