Scope
While abstract and introduction suggest much broader scope, every theoretical result assumes noiseless observations of a deterministic, invertible, measure-preserving system, which is essentially a study of unitary Koopman group (very idealized setting usually far from practical use cases of JEPA). The authors say “under standard assumptions” and “dynamical data”, and introduction motivates the work with observations that are “a noisy, partial, and often nonlinear view” of the system. But then the theory excludes noise, studies a linear observation map, and unitarity is only mentioned in the appendix notation. This is not a minor shift in scope, since under process or observation noise, Lem A.1 fails (the window is not anymore a function of the state), the loss acquires an irreducible variance term so that main strategy of this paper (zero loss collapse in Thm 3.3, delay-embedding result of Thm 3.5, and point/continuous dichotomy of Thm 3.11) is failing completely. I would also like to note that (see bellow), even within the deterministic scope, the non-degenerate case of Thm 3.3 is attained only by the torus regime, since for mixing systems no non-constant finite-dimensional invariant subspace exists. So, in the empirical part of the paper: (1) the two stochastic regimes are therefore not “stress tests” of the presented theory but fully lie outside of it, and (2) all regimes are trained jointly on a non-ergodic mixture. To conclude, the authors should either restrict the claims accordingly and state the correct scope explicitly in the abstract and introduction, or extend the analysis to Markov transfer operators carefully reviewing novelties w.r.t. the operator-learning literature of the next paragraph (which I believe is a significant revision of the work).
Positioning and missing literature
The paper frames several questions as new, despite the fact that they have been already studied Koopman and spectral representation-learning literature. This includes what a linear latent predictor learns, how to avoid collapse, how to constrain the predictor for measure-preserving systems, and how to verify learned spectra. While notion of JEPA became recently popularized (and the name was coined recently), the work that predates it is significant and is largely not cited here. So the contribution is hard to assess and in my opinion the novelty is overstated. To help the authors position their work better, I tried to group the missing work by topic and briefly explain why each group matters.
-
Learning Koopman/transfer-operator representations without collapse. The paper’s central observation is that a linear latent predictor learns an approximately invariant subspace and that an unnormalized loss collapses. This has been discussed in deep Koopman representation learning literature, see e.g. VAMPnets [1, 2] and DPNets [3] where collapse is excluded by construction and does not need an add-on regularizer, or operator-regression formulations [4] that make the “Deep EDMD” of Remark A.4 precise, including the projection error. In my opinion these methods are the natural baselines for Sec 5, and they should be discussed when presenting VICReg as the remedy. Beyond that, non-contrastive SSL with linear predictors has been analyzed in [5], and VICReg-type objectives have been related to spectral embeddings in [6], and I believe that the paper should explain how its collapse analysis (Remark A.3, Section 5.1) relates to these results. Finally, Lusch et al. is cited, but the authors do not say that it already parameterizes the operator with rotation-scaling blocks.
-
Low-rank-approximation and contrastive objectives as spectral learning. Another relevant line of work is the one that learns the leading singular functions of a conditional expectation operator (stochastic version of Koopman operator called also transfer operator) with neural networks. It minimizes a low-rank approximation loss that needs no whitening or matrix inversion. See e.g. NestedLoRA [7, 8], work [9] that shows how the same objective underlies SSL with a spectral decomposition as its minimizer, NCP [10] that discusses covariance type regularizations and proves with guarantees for inference and uncertainty quantification, and finally [11] that frames SSL explicitly as evolution operator learning for high-dimensional dynamics, which is also this paper’s framing. So, together these works directly impact the paper’s question “what should a non-collapsing predictive SSL objective recover?” but also come with operator-theoretic guarantees covering the stochastic and non-normal setting (in my opinion more typical in practice), which this paper’s theory excludes.
-
Unitary predictors and verification of learned spectra. Constraining the predictor to be unitary for measure-preserving systems is the idea of mpEDMD [12], and ResDMD [13] certifies learned eigenpairs through residuals and handles continuous spectrum. On the other hand spectral learning rates and spurious eigenvalues of stochastic systems are also analyzed in [4]. These works bear directly on the claims of frequency recovery. I think they are needed to tell true eigenvalues from pseudo-eigenvalues and to show which part of the block-orthogonal design is new.
Theoretical issues (measure related) I find that following issues significantly impact the paper and that they need to be addressed correctly:
(1) Proving Thm 3.3, step “using the ergodic theorem to equate the expectation over the data
distribution with the spatial integral over the invariant measure” in Thm A.2 is not done correctly. It is unclear which theorem from Ergodic theory and assumptions under which this can be done are missing. According to my understanding, the only case that satisfies all the hypotheses of Thm 3.3 in a non-trivial way is torous.
(2) Ass 3.1 says “an ergodic invariant probability measure”, but Lorenz and Rossler have infinitely many, and depending on which one is ment the intended the content cis affected: (i) for a fixed point, L2(μ) is one-dimensional, so the linear-independence hypothesis fails for even two functions, (ii) for a periodic orbit, L2(μ) is finite-dimensional, and the content becomes trivial. Since “Spatial averages equal empirical averages” holds for Lebesgue-a.e. initial condition in the basin only when the measure is physical, it is my understanding that the authors intended the SRB measure for these systems, But in that case, this needs to be assumed explicitly, and note that its existence is not established for the hyperchaotic Rossler.
(3) While Def 3.4 refers to “the attractor supporting μ”, the proof of Thm 3.5 sets A=suppμ, which differ in general, and importantly mA and dA change with the choice of μ (e.g. for a periodic-orbit measure they are 0). Because the loss is a μ-expectation, what matters is injectivity on a set of full μ-measure, not on all of A. Moreover, Thm 3.5 I think has issues in use of Sauer–Yorke–Casdagli, handling of prevalent family of observables, and mistaking correlation dimension with the box dimension.
To conclude
My current score reflects the main tension within the paper: (A) keep the theoretical setting (I believe that current technical issues can be resolved in minor revision) and significantly reduce the scope and impact of the paper, or (B) aim intended impact and do a proper theory that studies the representation collapse in practical setting (noisy and stochastic data from multiple systems) which in my opinion would need significant rewriting (in a sense is quite different paper to write). In both cases I see the possibility of paper meeting the acceptance bar unlikely. That said, I acknowledge the authors path and do believe that this topic is interested and that it deserves a good in-depth research that can bring benefits to JEPA community.
References:
- [1] Mardt, Pasquali, Wu, Noé. VAMPnets for deep learning of molecular kinetics. Nat. Commun. 2018.
- [2] Wu, Noé. Variational approach for learning Markov processes from time series data. J. Nonlinear Sci. 2020.
- [3] Kostic et al. Learning invariant representations of time-homogeneous stochastic dynamical systems. ICLR 2024.
- [4] Kostic et al. Sharp spectral rates for Koopman operator learning. NeurIPS 2023.
- [5] Tian, Chen, Ganguli. Understanding self-supervised learning dynamics without contrastive pairs. ICML 2021.
- [6] Balestriero, LeCun. Contrastive and non-contrastive self-supervised learning recover global and local spectral embedding methods. NeurIPS 2022.
- [7] Ryu et al. Operator SVD with neural networks via nested low-rank approximation. ICML 2024.
- [8] Jeong et al.. Efficient parametric SVD of Koopman operator for stochastic dynamical systems. NeurIPS 2025.
- [9] HaoChen et al. Provable guarantees for self-supervised deep learning with spectral contrastive loss. NeurIPS 2021.
- [10] Kostic et al. Neural conditional probability for uncertainty quantification. NeurIPS 2024.
- [11] Turri et al.. Self-supervised evolution operator learning for high-dimensional dynamical systems. ICLR 2026.
- [12] Colbrook. The mpEDMD algorithm for data-driven computations of measure-preserving dynamical systems. SIAM J. Numer. Anal. 2023.
- [13] Colbrook and Townsend. Rigorous data-driven computation of spectral properties of Koopman operators for dynamical systems. Comm. Pure Appl. Math. 2024.
We thank the reviewer for these comments. Its main point is fair: our experiments check the predictor's rotation angles, not the encoder's coordinates, and a claim about the representation should be tested on the representation. We focused on the angles because each one has an exact theoretical value that also shifts in a known way with the prediction horizon. This gave us a direct test against ground truth, including for frequencies that do not appear in the observed signal. A correct spectrum in the predictor does not, however, show by itself that the coordinates carry the phase.
There is a theoretical link between the two. If a rotation block predicted its own future exactly, its two coordinates would form a Koopman eigenfunction, and for a torus or a limit cycle that eigenfunction is the physical phase up to an offset and a scale. Matching frequencies together with a small prediction error inside the block would therefore imply matching phases. We did not measure that error block by block. Comparing each block's phase with the true phase, as the reviewer suggests, is the test that could settle the question.
The identity block does mostly encode which system a window comes from. This is expected when several systems are pooled: the quantities that stay constant over time are the system labels, so the non-rotating block learns them. It is also why we did not use regime classification as evidence of within-system dynamics.
On the unconstrained JEPA, we agree this is the right control, and the paper does not include it. We reported how close its eigenvalues are to the unit circle but did not compare their frequencies with the targets, so it may well carry similar information. We chose the block form because it assigns each frequency to a fixed pair of coordinates, which makes the interpretation more direct. With a dense predictor, the frequencies would first have to be extracted from its eigendecomposition and then matched to coordinates. In the comparison the paper does contain, the block structure gives similar regime separability and roughly a third of the prediction error. The structure helps, then, but the paper fails to show that it is necessary.
The reviewer's two suggestions are to compare block phases with true phases, and to compare the unconstrained predictor's eigenvalues with the targets. These are what this work needs to go from a correct spectrum in the predictor to a representation that demonstrably tracks the dynamics. We are grateful to the reviewer for making that gap so clear.