- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper extends the identifiability analysis of distribution-matched JEPAs from Euclidean latent spaces to latent variables supported on embedded Riemannian manifolds. The authors consider positive pairs generated by stationary diffusion and study when minimizing alignment loss under exact distribution matching recovers the latent variables up to a linear transformation.
The main theoretical contribution is a characterization of linearly admissible latent worlds through conditions on the stationary density, embedding geometry, and spectral ordering of the weighted Laplacian. The paper establishes sufficient and conditional necessary results for exact linear recovery, derives an approximate-recovery bound controlled by the separation between linear and nonlinear spectral modes, and identifies a geometric obstruction to obtaining a larger spectral gap. In particular, uniformly distributed spherical latents admit exact linear recovery and a stronger relative spectral separation than Gaussian latents.
Experiments on Gaussian, spherical, toroidal, and high-dimensional Clifford-torus worlds investigate the effects of matching or mismatching the target geometry. The results generally favor geometrically matched targets, although optimization sensitivity is observed in the toroidal setting.
Critical strengths
1. A meaningful theoretical extension of JEPA identifiability.
The paper addresses a substantive limitation of existing Euclidean identifiability analyses: many natural latent variables, including directions and periodic configurations, live on non-Euclidean manifolds. By formulating identifiability in terms of latent geometry and the spectrum of the associated diffusion operator, the work provides a principled explanation of when non-Gaussian, manifold-supported targets can support linear recovery. The characterization of optimal representations up to orthogonal symmetries is particularly valuable because it connects the choice of target distribution to a concrete identifiability guarantee.
2. A useful stability analysis supported by controlled experiments.
The approximate-recovery result in Theorem 4.4 goes beyond exact global optima by relating deviations from linear recovery to the separation between linear and nonlinear spectral modes. Together with the radial obstruction in Theorem 4.5 and the Gaussian–sphere comparison in Corollary 4.6, this gives an informative geometric interpretation of representation stability. The experiments across several latent geometries, including high-dimensional Clifford tori, provide empirical support for the importance of matching the representation target to the latent geometry. I view the combination of identifiability theory and controlled geometric comparisons as the main reason to support acceptance.
Critical weaknesses
1. Insufficient positioning relative to classical geometric analysis.
The geometric compatibility conditions in Definition 4.1 appear closely related to well-established results in differential geometry. In particular, the Gaussian-weighted density and embedding condition correspond, up to scaling, to the self-shrinker equation in mean curvature flow, while the associated drift Laplacian has been extensively studied (e.g., Colding and Minicozzi, 2012). In the constant-radius setting, the common coordinate-eigenfunction condition is closely related to Takahashi's theorem on minimal immersions into spheres (1966).
These connections do not invalidate the JEPA identifiability results, whose formulation in terms of distribution-preserving alignment and spectral optimality remains valuable. However, the paper does not sufficiently distinguish which geometric properties are inherited from classical results and which conclusions are genuinely new consequences for representation learning. This omission makes it difficult to assess the precise extent of the mathematical novelty. The framing of moving "beyond Gaussian" would also benefit from clarification, since the admissible density in Definition 4.1 remains a Gaussian potential restricted to the manifold, becoming uniform on constant-radius manifolds.
2. A gap between the theoretical guarantees and the empirical evidence.
The main guarantees assume that the representation takes values on the same manifold as the true latent variables, exactly matches the latent distribution, and reaches an optimal or near-optimal alignment objective. This is a useful theoretical setting, but its applicability to practical JEPAs with unknown latent geometry, finite-weight regularization, and potentially higher-dimensional representations is limited.
There are also concerns about how strongly the experiments validate the theoretical conclusions. Most notably, Table 1 reports near-perfect matched torus recovery averaged over only the five successful runs out of ten, rather than reporting all runs as the primary statistic. The remaining runs perform substantially worse, indicating meaningful optimization instability. The stronger Gaussian-versus-spherical stability guarantee is not directly tested under controlled excess alignment loss, and the high-dimensional Gaussian and Clifford-torus comparisons use different distribution regularizers in their main settings.
These limitations do not negate the theoretical contribution, but they weaken the empirical evidence for the practical advantages implied by the analysis.
Questions for authors
-
Connection to classical geometric results. Could the authors clarify the relationship between Definition 4.1 and the self-shrinker/drift-Laplacian literature, as well as Takahashi's theorem for minimal immersions into spheres? Which components of Theorems 4.2–4.5 are novel beyond these geometric identities and the spectral arguments inherited from Klindt et al. (2026)? A more explicit distinction would help assess the theoretical contribution.
-
Scope of the matched-geometry assumption. The theory assumes that the representation space is the true latent manifold and that its target distribution is known. What implications, if any, survive when the embedding dimension exceeds the intrinsic latent dimension and the representation is regularized toward a full-dimensional Gaussian or spherical target? This seems important for relating the results to practical JEPA architectures.
-
Optimization stability and experimental reporting. Could the authors report the unconditional mean and standard deviation over all ten matched torus runs in the main results? It would also be helpful to analyze whether unsuccessful runs fail to recover one of the two angular coordinates, as suggested in Section 6.1. Can the authors provide direct evidence distinguishing optimization failure from other sources of poor recovery?
-
Empirical validation of the spectral-gap prediction. Corollary 4.6 predicts a tighter approximate-recovery guarantee for spherical than Gaussian latents at a fixed linear-mode correlation. Could the authors compare matched Gaussian and spherical worlds at controlled levels of excess alignment loss and examine whether recovery errors reflect this predicted advantage? This would more directly test the stability result rather than only the benefits of target matching.