- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper points out that SIGReg, the anti-collapse regulariser in the LeWM world model, only pushes the latent at each time step toward and says nothing about how consecutive latents relate. As a result, a latent that is constant along each trajectory still reaches the global minimum of the LeWM loss. The paper proposes T-SIGReg, which replaces SIGReg with a target for the joint law of a window of consecutive latents. The target is , where is a stationary autocorrelation profile. This target is enforced with the same Epps–Pulley test on 1-D projections that SIGReg uses. The main methodological contribution is treating the choice of projection directions as an experimental design problem. Separable directions make the test sensitive to errors across time without the dilution of full-sphere directions.
Critical strengths
- The diagnosis is clear and correct.
- Treating the distribution of projection directions as an experimental design, and summarising what a design can see by the spectrum of one small matrix , is a nice idea. It makes the blind spots of a sliced test computable before training.
Critical weaknesses
W1. Scope and evaluation are too narrow.
- The method is only shown as a patch to SIGReg inside LeWM. No other world model is compared (e.g. DINO-WM, PLDM, DreamerV3, TD-MPC2). The path law is never applied to another base model, e.g. as a replacement for PLDM's VICReg-based terms. So it is unclear where the method stands among world models, or whether it helps training for anything beyond LeWM.
- Small benchmarks, no real-world data. Four small simulated suites inherited from LeWM. OGBench appears only through single-cube manipulation, where 23–53% of episodes start with the goal already satisfied (Table 9). Only one window length is tested (). No real-world data.
W2. The motivating failure is not shown to happen, and it is unclear if the gain is tied to the path law. The paper claimed a trajectory-constant latent is a global minimiser, but there's no experiment suggesting SIGReg-trained models suffer from collapse (e.g. Fig.1.). It would need further ablation study to demonstrate if the problem is real and how the proposed path regularization is help to prevent it.
W3. The writing is hard to follow. Dense prose, many coined terms ("world clock", "the ladder"), and most definitions and caveats deferred to a 35-page appendix make the main argument hard to follow.
Questions for authors
- How does T-SIGReg compare with existing world models (e.g. DINO-WM, PLDM, TD-MPC2) under the same protocol? Can you report results under LeWM's original evaluation protocol? How does the method do on harder OGBench tasks (Visual Scene, multi-cube, Puzzle) and on real-world video?
- Can you show a setting where a SIGReg-trained model actually ends up with a degenerate or trajectory-constant latent, and T-SIGReg prevents it?
- Would the path law improve any base model other than LeWM?
Response to Reviewer jECd
Thank you for a careful and direct review. Your concerns were fair: the original submission asked the reader to accept a failure it did not show, on a narrow evaluation, in prose that was hard to follow. We took all three seriously. The paper has been rewritten from the abstract to the conclusion, and each of your questions became a set of experiments that are now its main results. Changes that respond to your review are in pink. Numbers are planning success (%).
W1 and Questions 1 and 3: scope
Does the path law improve a base model other than LeWM? Yes. As you suggested, we replaced the VICReg-based regulariser of PLDM by T-SIGReg, and we did the same in Fast-LeWM. Mean over the four environments (Table 1):
T-SIGReg improves each world model in each of the four environments. Table 1 also compares with Sub-JEPA, AC-MTM, TC-LeWM and temporal straightening.
DINO-WM and LeWM's original protocol. Appendix Table 9 reports every goal distance, and its shaded column is LeWM's original setting (goal 25 steps away). In that setting, T-SIGReg against SIGReg in LeWM gives 100.0 against 86.7 on TwoRooms, 82.8 against 71.8 on Cube, 89.3 against 92.2 on Push-T and 80.2 against 81.7 on Reacher. So at this short distance T-SIGReg is clearly ahead in two environments and slightly behind in two. The gain grows with the distance to the goal (Figure 5). DINO-WM, from its public checkpoints, reaches 100.0, 80.0 and 75.7 on TwoRooms, Cube and Push-T. We evaluate it only in this setting, because one planning step with it takes about 130 times longer than with LeWM (Appendix D.1).
Harder OGBench tasks. We added cube-double, cube-triple, scene and puzzle-3x3 (Appendix E, Table 15). With an AR(1) profile at ρ = 0.97, fixed before seeing any result on these tasks, T-SIGReg improves on the original regulariser of all three world models in all four tasks. For example, in LeWM: 52.4 against 40.8 on cube-double and 50.0 against 44.1 on cube-triple. With the measured profile, the configuration of Table 1, T-SIGReg is below the AR(1) profile on these tasks and below the original regulariser in three of the twelve cases. We report both rows.
Episodes that start at the goal. Without them the gain on Cube holds and grows: 87, 52 and 66 for T-SIGReg against 76, 35 and 51 for SIGReg at goals 15, 45 and 75 steps away (Appendix D.5).
Window length. We sweep it in Appendix C.5. T-SIGReg stays above SIGReg at every length on TwoRooms, Cube and Reacher. On Push-T it needs LeWM's default of three context frames and falls below SIGReg with shorter windows.
What we did not do. We do not compare with reward-based agents such as DreamerV3 and TD-MPC2, which solve a different problem (our setting has no reward), and we have no real-world video. The conclusion states both limits.
W2 and Question 2: is the collapse real, and is the gain tied to the path law?
A setting where SIGReg collapses and T-SIGReg does not (Section 5.2, Figure 1a–d). We render Reacher over a random background that is fixed within each episode. Under SIGReg the latent stops moving within an episode: only 10⁻⁵ of its variance lies within a window, against 0.36 on clean Reacher, and a linear probe no longer recovers the arm. Planning succeeds in 5.6% of episodes, below a random policy (11.3%). Under T-SIGReg the probe recovers the shoulder angle and planning succeeds in 84.9%. The other two world models show the same pattern: PLDM goes from 6.5 to 84.3 and Fast-LeWM from 7.8 to 60.5. On Cube with a natural video in the background, LeWM goes from 6.6 to 47.0 (Appendix E).
Is the gain tied to the path law? Three controls (Section 5.3). T-SIGReg prescribing no temporal structure (R = I) falls below SIGReg in all four environments. SIGReg matched to the variance T-SIGReg reaches does not improve on SIGReg. Slowness and curvature penalties help in some environments and hurt in others.
W3: writing
You were right, and we treated the writing as seriously as the experiments. The paper no longer leads with the theory. The abstract and introduction state the two problems in plain terms and show them in Figure 1. A new Section 3.1 defines the problem, the world model, the planner and SIGReg in the body, and Section 3.2 explains step by step why SIGReg allows both problems. "World clock" and "the ladder" are gone, and the body no longer depends on the appendix for definitions.
Each configuration is one training run, and we state this in the paper. We would be glad if you could look at the revised version, and if any step of the argument is still hard to follow, we would like to know and will fix it.