Representational Collapse and Spectral Recovery in Koopman-Structured Time-Series JEPAs

Joint-embedding predictive architectures (JEPAs) are an effective framework for self-supervised learning on sequential data, but their latent representations remain difficult to interpret, particularly when a dynamical system generates the data. We study predictive self-supervised learning through Koopman operator theory and delay embeddings. Extending prior analyses of the eigenvalue-one case, we show that, under standard assumptions, training a linear latent predictor amounts to learning a finite-dimensional approximation to a Koopman-invariant subspace. Absent variance and covariance constraints on the latent space, both unconstrained and norm-preserving predictors converge toward degenerate representations rather than well-organized ones; this collapse, rather than the loss of oscillatory structure, is the dominant failure mode of predictive self-supervised learning on dynamical data. Standard variance-covariance regularization suffices to prevent it, and a Koopman-structured predictor adds interpretability: its learned rotation angles can be compared directly against the spectral content of the underlying system. We relate context length, latent dimension, and prediction horizon to observability, spectral capacity, and predictability. On synthetic dynamical systems, the learned representations recover the true oscillation frequencies, including combination frequencies not explicitly targeted by the model, while exhibiting the collapse and capacity trade-offs predicted by our analysis.