- Rating
- 4(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes Commute-Time-Preserving World Models (CTWM), a JEPA-based reconstruction free world model that jointly learns an encoder and latent dynamics model. The main motivation of the paper is that the latent spaces of prior JEPA-based world modeling works do not accurately reflect the commute times in the actual environment. While the scaled spectral embedding of the transition graph’s Laplacian has this property but is intractable, and thus this motivates the use of self-supervised learning methods to learn these embeddings. They argue that prior works usually incorporate regularization terms that encourage isotropic representations which inherently removes eigenvalue-dependent scaling, which is a necessary component for representing commute times properly.
The proposed architecture builds upon LeWM with residual prediction and a log-determinant regularization term. The residual predictor is trained to predict the difference between successive embeddings and incorporates a regularizer term to penalize large predictor outputs. The goal of the residual predictor to predict the displacement between latents informative and predictive while keeping temporally correlated latents close together. The log-determinant regularization term helps prevent global collapse. Prior works have opted to use methods like VCReg yet these methods impose a hard threshold on feature variance which limits their ability to represent commute-time accurately.
The main contributions of the paper are proposing their CTWM architecture that is based on LeWM/JEPA world modeling as well as providing theoretical foundations on why different regularizers may result in different representations, and more specifically, how log-determinant regularization is consistent with scaled Laplacian representation unlike VCReg and orthonormality constraints.
To validate their direction, they do a small scale empirical experiment to verify the regularization choice as well as test on 6 different goal-reaching continuous RL environments. On the small scale setting, they create a torus with MNIST-rendered states, where CTWM matches the ground truth and ALLO subspaces at the proposed ceiling and beats VICREG, and their orthonormal + residual prediction ablation. On the goal-reaching benchmarks, CTWM outperforms or at least matches the performance of LeWM on all experiments. An analysis on a discretized version of the pointmaze benchmark further suggests that CTWM's latent distances follow commute-time geometry better than LeWM.
Critical strengths
The authors present strong motivation for learning reconstruction free world models such that the latent geometry is useful for planning. This is a highly relevant area of research as prior reconstruction free/JEPA-based works have often been advocated for their efficient planning in terms of time and performance. The authors' research direction is well explained and supported with clean theoretical formulations that explain the role of the regularizer as well as provide evidence on why a log-determinant regularizer is more optimal for planning than the choices used in prior works. The authors also present 3 separate experiments sections that each answer different questions such as importance of the regularizer choice, overall success rate performance relative to LeWM on goal reaching tasks, and whether CTWM faithfully represents commute-time geometry.
Critical weaknesses
W1. The paper's main claim is that CTWM outperforms the prior work because it preserves commute times while prior regularizers for JEPA models do not. However, the experiments do not cleanly establish this link because the regularizer test uses a different baseline than the planning test. The torus experiment that checks whether each regularizer recovers the correct laplacian subspace compares CTWM against ALLO, VICreg and the their WM-GDO ablation. However, in the planning experiments CTWM is only compared against LeWM which uses SIGreg and not VICreg. Since they are both different regularization objectives, the torus results do not show that SIGreg fails to recover the correct scaling and it also remains unclear if a VICreg-based JEPA world model such as PLDM would perform on the goal reaching experiments. This matters because in the original LeWM paper, PLDM outperforms LeWM on Two-room, which is one of the environments where CTWM's long-horizon gains are larger. Additionally, there is no ablation of the slowness term introduced through residual prediction. While WM-GDO shows that replacing log-determinant with a different constraint degrades the latent geometry, no experiment tests log-determinant without the slowness penalty or what happens when the penalty is applied directly to the encoder. The paper does not clearly show that residual prediction is necessary or how much the slowness term contributes in practice.
W2. There are no comparisons to other planning methods which limits the paper's claims. LeWM is the only baseline for the goal reaching experiments, but as highlighted above, LeWM sometimes underperforms other similar works. The paper also does not compare against related methods that target improving the planning of JEPA-based models. For example, Temporal Straightening was cited and seems to be close to CTWM but is not compared against. Comparisons against temporal straightening would be a valuable baseline and other works would significantly reinforce the paper's claims that commute-time preserving latent representations are necessary.
Questions for authors
Q1. Why was the latent dimension reduced from 192 to 64? I am curious on if this choice helps CTWM's performance or CTWM outperforms LeWM despite the reduction in size. There wasn't any argument for this reduction but it is a big reason on why CTWM can claim half the parameter count. Are there any results on CTWM at D = 192 and LeWM at D = 64 to justify the statement CTWM outperforms LeWM at half the parameter count?
Q2. Why does CTWM have pretty much flat curves based on epochs across all the six goal reaching tasks? I find it a bit weird that CTWM pretty much performs the same regardless of how many epochs. The performance delta from epoch 1 versus epoch 10 looks below 10% on all tasks and virtually 0 on Cube, Two-Room, and Pointmaze. There isn't a mention on whether CTWM results in faster learning over LeWM or something else is going on. Could you analyze or give an explanation on what is happening at epoch 1 and the early epochs for there to be much flatter curves than LeWM?
Q3. What happens with log-determinant if there slowness penalty is higher or nonexistent? Alternatively, what happens when you apply a slowness penalty to LeWM?
Q4. Why do you route the slowness penalty through the predictor rather than applying it directly to the encoder? Is there an empirical difference?