acceptodds
Under review as a conference paper at ICLR 2027

BEYOND GAUSSIAN WORLDS: LATENT GEOMETRY MATTERS FOR JEPAS

Abstract

Recent Joint-Embedding Predictive Architectures (JEPAs) prevent representation collapse by constraining learned representations to follow a prescribed target distribution, such as an isotropic Gaussian or the uniform distribution on a hypersphere. Klindt et al. (2026) showed that, under their Euclidean assumptions, matching a Gaussian target can recover Gaussian latent variables up to a linear transformation, and that the Gaussian is the unique distribution with this guarantee. We extend their analysis to latent variables supported on embedded Riemannian manifolds and derive conditions on the latent geometry and positive-pair dynamics under which alignment and exact distribution matching guarantee linear recovery. In particular, when the latent variables are uniformly distributed on a sphere and the representations are matched to the same spherical distribution, every optimal representation recovers the latent state up to an orthogonal transformation. This shows that Gaussian uniqueness is not a universal property of distribution-matched JEPAs: non-Euclidean latent geometries can admit other linearly recoverable distributions. We further derive an approximate-recovery bound that is strictly tighter for the spherical world than for the Gaussian world. Experiments on Gaussian, spherical, and toroidal latent spaces show that geometrically compatible targets yield better linear recovery when optimization succeeds, whereas mismatched targets distort the latent structure. This advantage persists in high-dimensional Clifford-torus worlds.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
JEPA, GEOMETRY

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_t3saOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper extends the identifiability analysis of distribution-matched JEPAs from Euclidean latent spaces to latent variables supported on embedded Riemannian manifolds. The authors consider positive pairs generated by stationary diffusion and study when minimizing alignment loss under exact distribution matching recovers the latent variables up to a linear transformation.

The main theoretical contribution is a characterization of linearly admissible latent worlds through conditions on the stationary density, embedding geometry, and spectral ordering of the weighted Laplacian. The paper establishes sufficient and conditional necessary results for exact linear recovery, derives an approximate-recovery bound controlled by the separation between linear and nonlinear spectral modes, and identifies a geometric obstruction to obtaining a larger spectral gap. In particular, uniformly distributed spherical latents admit exact linear recovery and a stronger relative spectral separation than Gaussian latents.

Experiments on Gaussian, spherical, toroidal, and high-dimensional Clifford-torus worlds investigate the effects of matching or mismatching the target geometry. The results generally favor geometrically matched targets, although optimization sensitivity is observed in the toroidal setting.

Critical strengths

1. A meaningful theoretical extension of JEPA identifiability.

The paper addresses a substantive limitation of existing Euclidean identifiability analyses: many natural latent variables, including directions and periodic configurations, live on non-Euclidean manifolds. By formulating identifiability in terms of latent geometry and the spectrum of the associated diffusion operator, the work provides a principled explanation of when non-Gaussian, manifold-supported targets can support linear recovery. The characterization of optimal representations up to orthogonal symmetries is particularly valuable because it connects the choice of target distribution to a concrete identifiability guarantee.

2. A useful stability analysis supported by controlled experiments.

The approximate-recovery result in Theorem 4.4 goes beyond exact global optima by relating deviations from linear recovery to the separation between linear and nonlinear spectral modes. Together with the radial obstruction in Theorem 4.5 and the Gaussian–sphere comparison in Corollary 4.6, this gives an informative geometric interpretation of representation stability. The experiments across several latent geometries, including high-dimensional Clifford tori, provide empirical support for the importance of matching the representation target to the latent geometry. I view the combination of identifiability theory and controlled geometric comparisons as the main reason to support acceptance.

Critical weaknesses

1. Insufficient positioning relative to classical geometric analysis.

The geometric compatibility conditions in Definition 4.1 appear closely related to well-established results in differential geometry. In particular, the Gaussian-weighted density and embedding condition correspond, up to scaling, to the self-shrinker equation in mean curvature flow, while the associated drift Laplacian has been extensively studied (e.g., Colding and Minicozzi, 2012). In the constant-radius setting, the common coordinate-eigenfunction condition is closely related to Takahashi's theorem on minimal immersions into spheres (1966).

These connections do not invalidate the JEPA identifiability results, whose formulation in terms of distribution-preserving alignment and spectral optimality remains valuable. However, the paper does not sufficiently distinguish which geometric properties are inherited from classical results and which conclusions are genuinely new consequences for representation learning. This omission makes it difficult to assess the precise extent of the mathematical novelty. The framing of moving "beyond Gaussian" would also benefit from clarification, since the admissible density in Definition 4.1 remains a Gaussian potential restricted to the manifold, becoming uniform on constant-radius manifolds.

2. A gap between the theoretical guarantees and the empirical evidence.

The main guarantees assume that the representation takes values on the same manifold as the true latent variables, exactly matches the latent distribution, and reaches an optimal or near-optimal alignment objective. This is a useful theoretical setting, but its applicability to practical JEPAs with unknown latent geometry, finite-weight regularization, and potentially higher-dimensional representations is limited.

There are also concerns about how strongly the experiments validate the theoretical conclusions. Most notably, Table 1 reports near-perfect matched torus recovery averaged over only the five successful runs out of ten, rather than reporting all runs as the primary statistic. The remaining runs perform substantially worse, indicating meaningful optimization instability. The stronger Gaussian-versus-spherical stability guarantee is not directly tested under controlled excess alignment loss, and the high-dimensional Gaussian and Clifford-torus comparisons use different distribution regularizers in their main settings.

These limitations do not negate the theoretical contribution, but they weaken the empirical evidence for the practical advantages implied by the analysis.

Questions for authors
  1. Connection to classical geometric results. Could the authors clarify the relationship between Definition 4.1 and the self-shrinker/drift-Laplacian literature, as well as Takahashi's theorem for minimal immersions into spheres? Which components of Theorems 4.2–4.5 are novel beyond these geometric identities and the spectral arguments inherited from Klindt et al. (2026)? A more explicit distinction would help assess the theoretical contribution.

  2. Scope of the matched-geometry assumption. The theory assumes that the representation space is the true latent manifold and that its target distribution is known. What implications, if any, survive when the embedding dimension exceeds the intrinsic latent dimension and the representation is regularized toward a full-dimensional Gaussian or spherical target? This seems important for relating the results to practical JEPA architectures.

  3. Optimization stability and experimental reporting. Could the authors report the unconditional mean and standard deviation over all ten matched torus runs in the main results? It would also be helpful to analyze whether unsuccessful runs fail to recover one of the two angular coordinates, as suggested in Section 6.1. Can the authors provide direct evidence distinguishing optimization failure from other sources of poor recovery?

  4. Empirical validation of the spectral-gap prediction. Corollary 4.6 predicts a tighter approximate-recovery guarantee for spherical than Gaussian latents at a fixed linear-mode correlation. Could the authors compare matched Gaussian and spherical worlds at controlled levels of excess alignment loss and examine whether recovery errors reflect this predicted advantage? This would more directly test the stability result rather than only the benefits of target matching.

Reviewer_HWuJOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper analyze the latent geometry for JEPAs. They extend Klindt et al.(2026)'s Euclidean assumption to Riemannian manifolds. They theoretically derive the conditions on manifold worlds under which alignment and exact distribution matching revoer the latent state up to a linear transformation. They also validate that compatible target geometries can improve recovery using synthetic data under different distributions.

Critical strengths

I appreciate the theoretical analysis that help us better understand the latent geometry of JEPAs. Section 3 is well written. It provides a simple and clear background description.

Critical weaknesses

The authors did EXTREMELY few experiments. In the main paper, the experiment section is less than 2 pages and only 1 table is about the experiment results. What is worse, only synthetic data is used and there is no comparison against existing SSL methods. In LeJEPA, which this paper heavily cited, they applied pretraining on ImageNet-1k, ImageNet-100, ImageNet-10, etc, and validate the effectiveness on multiple tasks including DTD、Aircraft、Cars、CIFAR-10, etc. I think Section 4 and 5 should be merged and simplified and leave space for experiments on more practical datasets.

In the theory part, the admissible world is very narrow (Definition 4.1). It has to be an isotropic Gaussian distribution over M\mathcal{M}. Considering that the datasets are toy and the empirical results are too few, the narrow admissible world becomes very fatal, indicating that the analysis can be unpractical in a large sense.

Questions for authors
  • From line 741 to 746, only five runs achieve near perfect linear recovery. R2R^2 of the other failed trials is only 0.597. The authors explain that the other five trials with lower scores reflect an optimization failure. Is it possible that optimization failure also exists with other latent world assumptions? If so, table 2 can be misleading. For the Tours-Tours case, the authors only average over success trials. But for all the other cases (Sphere-Tours, Gaussian-Tours), they average over all trials, inlcuding sucess trials and failure trials.

  • In line 431, the authors mentiond "Five out of ten runs achieve near-perfect linear recovery". However, in line 477, the authors mentioned "For the toroidal world, the matched target reaches near-perfect recovery in two of three run". Is this a mismatch?

Reviewer_cqYCOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper extends the linear-identifiability analysis of distribution-matched JEPAs from Euclidean Gaussian latent spaces to distributions supported on embedded Riemannian manifolds, with positive pairs generated by stationary Langevin diffusions. The authors introduce conditions on the latent density, embedding geometry, and positive-pair dynamics under which alignment combined with exact distribution matching recovers the latent state up to an orthogonal symmetry of the latent world. They further derive an approximate-recovery bound controlled by the linear–nonlinear spectral gap, show that the uniform sphere has a larger relative spectral gap than the Gaussian, and evaluate the framework on Gaussian, spherical, toroidal, and high-dimensional Clifford-torus worlds.

Critical strengths

S1.

The theoretical contribution is clear and coherent. The paper connects density compatibility, geometric compatibility, and spectral ordering within a unified framework for exact and approximate linear recovery. The forward identifiability result, approximate-recovery bound, and radial-obstruction result provide a meaningful extension beyond the Euclidean Gaussian setting. In particular, showing that the uniform sphere is linearly identifiable demonstrates that Gaussian uniqueness does not hold universally once non-Euclidean latent geometries are considered.


S2.

The empirical evaluation includes a useful high-dimensional control. In the Clifford-torus experiment, the matched target is compared not only with a �0��0�-dimensional Gaussian but also with a �1��1�-dimensional Gaussian whose output dimension matches the canonical Clifford embedding. The matched target remains superior across dimensions, indicating that output dimensionality explains only part of the observed advantage.

Critical weaknesses

W1.

Part of the low-dimensional matched–mismatched comparison in Table 1 is confounded by output dimension. The Gaussian, spherical, and toroidal targets are represented in �0��0�, �1��1�, and �2��2�, respectively, while recovery is measured using an affine probe to the ambient latent coordinates. For an isotropic �3��3�-dimensional latent state and a �4��4�-dimensional representation, the explained variance of an affine decoder is bounded by approximately �5��5�. Several reported mismatched results nearly coincide with this rank ceiling:

  • Sphere world / Gaussian target: �6��6� versus �7��7�
  • Torus world / Gaussian target: �8��8� versus �9��9�
  • Torus world / Sphere target: �10��10� versus �11��11�

Therefore, the magnitude of these performance drops cannot distinguish geometric distortion from dimensionality loss. This does not invalidate the main conclusion, especially given the high-dimensional �12��12�-Gaussian control, but it weakens the interpretation of the low-dimensional comparison. Dimension-matched controls or reporting relative to the affine rank ceiling would make the evidence substantially more convincing.


W2.

The scope of the theoretical characterization should be stated more precisely. Theorem 4.2 is a conditional converse: it assumes that the first nonconstant eigenspace has dimension �13��13�, that the spectral lower bound is attained, and that every distribution-preserving minimizer is already linear in the ambient coordinates. Moreover, the theory assumes exact distribution matching in the matched population setting, whereas the experiments use finite-weight regularization and the mismatched experiments are explicitly outside the theorem’s scope. These assumptions are reasonable for the stated analysis, but the paper should describe the result as a characterization under the specified spectral and optimization assumptions rather than as a fully general characterization of manifold latent worlds. This is mainly a scope and presentation issue rather than a flaw in the forward identifiability result.


W3.

The �14��14� result in Table 1 averages only the five successful runs among ten, while the remaining five obtain �15��15�. The full ten-run statistics, median, and success rate should be the primary result. The claim that the unsuccessful runs reflect poor local minima rather than a limitation of the matched geometry is plausible, but the current evidence establishes correlation with higher invariance loss rather than conclusively identifying the cause. There is also an inconsistency in the main paper: Section 6.1 reports five successful runs out of ten, while the Conclusion states that near-perfect recovery occurs in two of three runs. This should be corrected.

Questions for authors

Q1.

How is the multi-output �0��0� in Table 1 aggregated? Could the authors add dimension-matched controls or report each result relative to the affine rank ceiling �1��1�? How would this affect the interpretation that mismatched targets distort or lose latent information?


Q2.

Could the authors clarify the scope of the term “characterization,” especially given the assumptions of Theorem 4.2 and the exact distribution-preservation requirement? A clearer separation between the proved matched-population result and the mismatched empirical stress tests would be helpful.


Q3.

Could the authors report all ten Torus-to-Torus runs as the primary result, including the mean, standard deviation, median, success rate, and per-seed values? What additional evidence supports attributing the unsuccessful runs specifically to optimization failure?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.