- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
Building on existing spectral representation learning and kernel alignment frameworks, this paper proposes SpecWM, a joint training approach for visual latent world models that combines spectral alignment with action-conditioned dynamics prediction. It investigates how different teacher kernels shape learned representations through theoretical analysis and linear probing, and evaluates their impact on downstream planning. Across three visual manipulation environments, OGBench Cube, OGBench Scene, and CALVIN, the temporal and quasimetric variants improve CEM planning success over LeWM by approximately 21 percentage points on average.
Critical strengths
- The proposed joint objective, combining spectral alignment with action-conditioned latent prediction, achieves consistent improvements in CEM planning success over LeWM across three visual manipulation environments, with an average gain of approximately 21 percentage points using temporal and quasimetric kernels.
- The paper empirically investigates how different teacher kernels influence the information preserved in learned representations through linear probing. The results show that kernel choice affects which physical quantities are linearly recoverable, and that the self-supervised temporal kernel can preserve substantial physical state information without requiring ground-truth state supervision.
Critical weaknesses
- The planning experiments are limited to three visual manipulation environments, without systematically evaluating generalization to unseen scenes, task compositions, or environment configurations. It remains unclear whether the learned representations can generalize beyond the training environments, particularly given the substantial amount of environment-specific training data required.
- The connections between spectral decomposition, contrastive representation learning, and successor measures have been extensively studied in prior work, as partly acknowledged by the authors. Closely related matrix-factorization objectives and multi-step transition operators have also been established in previous studies [1, 2]. The main technical novelty of this paper therefore appears to lie in combining spectral alignment with action-conditioned latent dynamics prediction. The related work section could further clarify the distinction from these earlier approaches.
References:
[1] Wu et al. (2018). The Laplacian in RL: Learning Representations with Efficient Approximations. https://arxiv.org/abs/1810.04586
[2] Tan et al. (2024). Contrastive Learning Is Spectral Clustering on Similarity Graph. https://arxiv.org/abs/2303.15103
Questions for authors
- How well do the teacher kernels reflect the state relationships needed for goal-directed planning? While the paper compares temporal and quasimetric kernels, it remains unclear how well the learned similarities reflect goal-conditioned reachability or planning costs. In particular, temporal distances can be affected by pauses, detours, revisits, and varying execution speeds, which do not necessarily change the underlying reachability structure. Could the authors investigate how such discrepancies affect the learned representations and planning performance?
- How does the proposed method perform on longer-horizon planning tasks? The planning experiments in Appendix C.3 use relatively short prediction horizons (25 environment steps for OGBench and 50 for CALVIN), with frequent replanning. For longer horizons, errors in recursive latent dynamics prediction may compound, while inaccuracies or biases in the teacher-defined similarities may lead to incorrect rankings of candidate action sequences. Could the authors evaluate planning performance over longer horizons and investigate how these two sources of error affect planning reliability?
3 comments
Thank you for your review and your support. As the rebuttal phase draws to a close, please let us know if any question remains; we'd be happy to answer.
Rebuttal Response
Thank you to the authors for the thoughtful responses and additional experiments. W1 and the related work concern in W2 are largely addressed, although my assessment of limited technical novelty remains unchanged. Q1 is partially addressed, but correlations with reachability weaken at longer distances and across episodes. Q2 is partially addressed, but long-horizon performance remains limited, and the contributions of teacher-related and dynamics errors remain unclear. Therefore, I maintain my weak accept recommendation.
Thank you for your thoughtful review. We are glad you found the analysis of what each teacher kernel preserves informative and the planning results compelling. Below are responses to each weakness.
We thank the reviewers for this point. We’ve updated our CALVIN evaluation to train on environments A, B, and C, and evaluate on environment D (Appendix B.1). We find that all methods see a drop in performance on environment D, but SpecWM still outperforms LeWM, and thus display better generalization.
We agree - while spectral methods have explored connections to RL, their instantiation in latent world model remains, to the best of our knowledge, novel. As suggested, we have further expanded our related works to not just mention related studies, but also how this work positions with respect to them.
Thank you for suggesting this analysis, which we are happy to provide. The answer to your question depends on the kernel used: while the temporal kernel should capture temporal distances under the behavioral policy, the quasimetric kernel should relate to optimal temporal distances. While both should correlate with reachability, we expect correlation to be stronger for the latter. We perform the evaluation you suggested in Appendix B.3, and find that the temporal-kernel similarity correlates with distance under the behaviour policy to a Spearman of 0.81 and with the true reachability to a 0.70. Hence, the model learns the behavioural somewhat more strongly than the optimal one, as predicted by Theorem 4. As temporal distance and reachability are in practice often correlated, learned similarities remain nevertheless informative for reachability.
Thank you for the suggestion. We have updated the paper with CEM performance at longer horizons and a plot of prediction and ranking error over horizons in Appendix B.2. We observe that for SpecWM, CEM performance deteriorates gradually, likely stemming from weaker ranking of progress than prediction as compared to LeWM.
Thank you for suggesting these additional analyses and ablation, which we believe have further improved our submission. Please, let us know if any questions remain: we are happy to address them through the rest of the rebuttal phase.