acceptodds
Under review as a conference paper at ICLR 2027

Uniformity Is Relative: A Maximum-Entropy Principle Across Representation Geometries

15 views

Abstract

A fundamental objective of self-supervised learning is to learn representations that preserve semantic consistency while occupying the representation space in a sufficiently dispersed yet non-concentrated manner. Existing notions of uniformity are tied to particular representation spaces, leaving unclear how to define an appropriate reference distribution across different representation spaces. We resolve this by defining uniformity as maximum entropy with respect to intrinsic Riemannian volume under a fixed geodesic scale. This makes uniformity a joint property of geometry and scale, recovering the canonical spherical and Euclidean references while yielding a distinct curved-space target. Meanwhile, tangent-space Gaussian transplantation misses curvature-dependent volume growth, leaving an irreducible Kullback–Leibler divergence gap even after scale retuning. A radial-angular decomposition yields tractable minibatch objectives, and experiments verify the predicted curvature-dependent mismatch in learned geometry, reveal its effects on radial and hierarchical organization, and demonstrate complementary roles for intrinsic uniformity and cross-view alignment.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Self-supervised learning, Representation uniformity, Representation Geometry, Maximum entropy, Hyperbolic representation learning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_5FGnOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper defines a uniformity reference in a constant-curvature representtion space as the maximum-entropy distribution, relative to Riemannian volume, under a budget on the squared geodesic distance from a reference point (Eq. 4). The solution is an isotropic Gibbs density (Eq. 5), which recovers the uniform measure on the sphere (when the budget is inactive). A tangent-space Gaussian pushed through the exponential map is shown to differ from this reference by a KL gap that scale retuning cannot remove. KL is then split into radial and angular terms (Prop. 4). Experiments covered a numerical check of the gap, hyperbolic node classification, an ImageNet-100 comparison, and an alignment/MaxEnt study

Critical strengths

The derivations in section 5 look (mostly) correct. The authors are at least candid that the effect size of the gap appears to be very modest (L437-439).

Critical weaknesses

The mathematics appears correct but disconnected from the experiments and the experiments do not support the paper's central claim that the intrinsic reference is the better one to train towards. Overall, the framing of novelty and scope is in my opinion not empirically or theoretically supported. Three problems seem to me individually serious: (1) the node-classification evidence is confounded; (2) the learned-representation evidence appears largely circular and rests on a label-informed initialisation; and (3) the gap is never shown to matter at the scales used. I am also concerned about novelty (4).

  1. The method (Eq. 16, no InfoNCE or negatives) is compared with full methods using other objectives, and no row runs the same pipeline with the tangent reference or K = 0. K, rho and alpha are not reported, and tuning is per dataset for the proposed method only. Margins are within noise on three of four datasets: PubMed accuracy, PubMed Macro-F1, and CiteSeer balanced accuracy.

  2. Section 7.3 is close to tautological and starts from a labelled initialisation. Eq. 13 matches radii to the chosen reference's quantiles, showing only that the optimiser reaches its target. Stress and Top-1 results are weak, with overlapping error bars in Figure 4c and no seed count. The ResNet-34 is supervised on ImageNet-1K (L415-417), and Appendix D has no tangent condition (why not?). In 7.4, D_rad and D_ang are the training losses (L462-463), so their rise without MaxEnt holds by construction.

  3. The gap is never shown to matter at the scales used. Integrating Eq. 7 at d = 16, I obtain KL gaps of 0.155 nats at (K, rho) = (-3, 6), 0.019 at (-3, 2), 0.005 at (+1, 2) and 0.0008 at (-1, 2), so the gap is appreciable only for tau >= 5, where Eq. 11 no longer holds. I also don't put a lot of stock in the boundary-drift remark at the end of section 5.

  4. The Gibbs form is the Riemannian normal, Prop. 2 is a change of variables, and Prop. 1 holds by construction, since E[r^2] is exactly the budget whose maximiser in R^d is the Gaussian. The surrogates in Eqs. 13-14 are not linked to Prop. 4, and no native uniformity loss is compared.

Questions for authors
  1. Which (K, rho, alpha) were used for Table 1? Please add rows with the identical pipeline under the tangent reference and under K = 0, and state the tuning budget given to the baselines.

  2. How does the gap at each setting in 7.3 compare with the minibatch noise floor of the loss (Figure 8), and how many seeds support Figure 4?

  3. Can the tangent vs. intrinsic comparison be repeated from a random or SSL-pretrained initialisation, with the tangent scale tuned?

  4. Why the second-moment budget, and how is o justified for K > 0? At rho >= rho_unif, does the objective match the standard Wang & Isola loss in the same pipeline?

5 comments
AuthorsOpenReview ↗

Author response [1/5]

We are grateful for the reviewers’ insightful comments.

We respectfully wish to clarify a possible misunderstanding regarding our central thesis. As stated in our title and contributions, our central goal is not to establish the performance superiority of the intrinsic reference over tangent-space approximations, but to develop a unified theoretical framework for SSL uniformity across different curvatures based on the intrinsic maximum-entropy principle. To this end, we pursue two complementary directions: (1) characterizing the gap between the intrinsic and tangent-space references as a formally testable geometric signature of this generalized notion (Sec. 5, Exps. 7.1/7.3); and (2) investigating whether intrinsic maximum entropy can serve, like classical uniformity, as an effective cross-curvature metric and regularizer for SSL representations (Sec. 4, Exps. 7.2/7.4). Although a substantial portion of our mathematical analysis focuses on the intrinsic–tangent gap, its purpose is to establish a formal, observable distinction supporting the proposed generalized notion of uniformity. Our central claim is that a more general, geometry-consistent notion of uniformity can be rigorously formulated and meaningfully applied across representation geometries.

A useful analogy for clarifying the scope of our contribution is the transition from the classical mechanical description of gravity to its geometric formulation in curved spacetime. In this transition, the anomalous perihelion precession of Mercury, though exceedingly small in magnitude, provided crucial observable evidence distinguishing the two theoretical frameworks. Its significance lies not in the magnitude of the discrepancy, but in revealing the underlying geometric structure that the classical description could not fully capture. Similarly, the intrinsic–tangent KL gap in our work is intended to provide a quantifiable signature of the geometric distinction, rather than to serve as a measure of SOTA performance improvement.

Point-by-Point Responses to Experimental Concerns

W1/Q1: Hyperparameter Settings, Tuning Fairness, and Controlled Ablations

1. Hyperparameter Configuration and Tuning Protocol

We select the hyperparameters based on validation performance through grid search, including curvature KK, scale budget ρ\rho, regularization weight α\alpha, number of slicing directions MM, and number of radial bins BB. Table 1 summarizes their search spaces and the selected configurations for each dataset, together with the dimensionless curvature–scale parameter τ=∣K∣ρ\tau=|K|\rho.

Table 1. Dataset-specific hyperparameter settings for the controlled ablation. The dimensionless scale is defined as τ=∣K∣ρ\tau=|K|\rho.

DatasetKKρ\rhoα\alphaMMBBτ\tau
Grid search range[−3,3][-3,3], step 0.50.5[0.1,2][0.1,2], step 0.10.1[0.25,2][0.25,2], step 0.250.25{64,128}\{64,128\}{2,4,8}\{2,4,8\}Derived
Airport−0.5-0.52.02.00.50.512881.01.0
PubMed−1.0-1.01.51.51.01.012841.51.5
CiteSeer−1.0-1.00.90.91.251.256480.90.9
Cora−2.0-2.01.01.00.50.512822.02.0

2. Controlled Comparison under the Table 1 Pipeline

To isolate the effect of reference selection, we additionally conduct a three-arm ablation on the same four datasets under the identical Eq. (16) training pipeline: (i) the Euclidean control (K=0K=0); (ii) the tangent transplant reference; and (iii) the Intrinsic MaxEnt reference. For the curved-space branches, curvature KK and scale budget ρ\rho are fixed; the K=0K=0 branch uses the flat-limit version of the same training pipeline, with all other hyperparameters unchanged. All three branches share the same two-layer GCN encoder, data augmentations, optimizer settings, five random seeds and linear-probe selection protocols. The results are reported below:

Table 2. Controlled three-arm comparison of node-classification accuracy (%). Results are reported as mean ±\pm sample standard deviation over five random seeds.

ReferenceAirportPubMedCiteSeerCora
K=0K=078.75±1.2078.75\pm1.2080.13±0.7280.13\pm0.7270.40±1.3570.40\pm1.3580.40±1.0980.40\pm1.09
Tangent transplant78.71±1.4678.71\pm1.4679.34±0.7679.34\pm0.7670.30±1.3770.30\pm1.3780.17±1.2180.17\pm1.21
Intrinsic MaxEnt (Ours)78.91±1.3878.91\pm1.3880.66±0.7580.66\pm0.7571.40±1.5071.40\pm1.5082.26±1.2882.26\pm1.28
AuthorsOpenReview ↗

Author response [2/5]

3. The Tuning Budget of the Baselines

For each baseline, hyperparameters are first selected using validation accuracy from an initial seed; the selected configuration is then fixed, and the test mean and sample standard deviation are reported over five additional seeds. All supervised baselines use Adam with a learning rate of 0.010.01, weight decay of 5×10−45\times10^{-4}, and dropout of 0.50.5, with model selection based on validation accuracy. GCN, GAT, and GraphSAGE use PyG convolutional layers with two-layer architectures and 64-dimensional hidden representations; GAT uses four attention heads. HGCN and HIE use the Poincaré ball with learnable curvature. These encoder configurations are held fixed across datasets. GRACE, COSTA, and HGCL also use fixed encoder configurations, with the linear-classifier regularization parameter CC selected on the validation set. For HyperGCL, we evaluate 20 configurations over five seeds, varying curvature, loss weights, data augmentations, representation dimension, and learning rate.

Q2/W3: Signal-to-Noise Comparison and Seed Count

First, the KL gap and the Wasserstein surrogate losses in Figure 8 use different metrics, so their numerical magnitudes are not directly comparable. The two references share the same uniform angular target; their discrepancy is entirely radial. To assess this discrepancy on the scale of the training objective relative to minibatch sampling noise, we compute the normalized radial Wasserstein gap W22(qtan,qK,ρ)/ρW_2^2(q_{\mathrm{tan}},q_{K,\rho})/\rho, the population counterpart of Eq. (13). Under the configurations used in Section 7.3, we then independently sample 4,096 minibatches from the intrinsic reference and compare the radial gap against the zero-perturbation sampling noise floor. The results are as follows:

Table 3. Calibration of the radial gap against minibatch sampling noise. All numerical values are scaled by 10−410^{-4}.

(K,ρ)(K, \rho)Radial gapNoise floorGap / floor ratio
(−3,6)(-3, 6)28.1828.180.90±0.530.90 \pm 0.5331.3831.38
(−3,2)(-3, 2)4.594.591.39±0.831.39 \pm 0.833.293.29
(−1,2)(-1, 2)0.240.241.70±1.021.70 \pm 1.020.140.14
(0,2)(0, 2)001.80±1.081.80 \pm 1.0800
(+1,2)(+1, 2)1.381.381.53±0.931.53 \pm 0.930.900.90

For (−3,6)(-3,6) and (−3,2)(-3,2), the radial gap exceeds the single-minibatch noise floor by factors of 31.3831.38 and 3.293.29, respectively, consistent with the separation in radial-tail ratio and normalized stress in Figure 4. For (−1,2)(-1,2) and (+1,2)(+1,2), the gap is below or close to the noise floor, and the two reference conditions yield nearly overlapping curves. Thus, we acknowledge that the effect is scale-dependent, while the results show that it can influence learned representation geometry when curvature effects are substantial.

Figure 4 uses three paired model seeds for each configuration; error bars denote the sample standard deviation across those seeds. Five (K,ρ)(K,\rho) configurations and two reference distributions yield 30 final checkpoints in total.

Regarding W3's concern about the validity range of the asymptotic approximation in Eq. (11), it is important to distinguish the exact result from its near-flat expansion. Under the assumptions of Proposition 3, Δ(K,ρ,d)>0\Delta(K,\rho,d)>0 holds for nonzero curvature, whereas Δ=Cdτ4+o(τ4)\Delta=C_d\tau^4+o(\tau^4) describes only the local behavior as τ→0\tau\to0. For larger τ\tau, the KL gap is computed by numerical integration of the full radial densities rather than by the fourth-order approximation. Thus, the breakdown of the asymptotic approximation at larger scales does not invalidate the exact gap result.

Regarding the boundary-drift remark at the end of Section 5, we agree that our experiments do not directly establish this phenomenon. The remark was intended to motivate possible implications of curvature-dependent volume growth rather than present boundary drift as an experimentally established effect. We have therefore revised the statement to keep it within the scope of the demonstrated geometric result:

Line 275: "This highlights the fundamental role of curvature-dependent volume growth in shaping the radial organization of hyperbolic representations."

AuthorsOpenReview ↗

Author response [3/5]

Q4: Theoretical Justification and Classical Recovery

1. Motivation for the Geodesic Second-Moment Constraint

Our goal is to extend uniformity from specific representation spaces to Riemannian manifolds of different curvatures while recovering canonical uniformity references under the appropriate conditions. We therefore adopt a geodesic second-moment constraint, motivated by canonical recovery properties and a consistent interpretation of representation scale across geometries. In particular, this constraint has three useful properties:

(i) Canonical recovery: In Euclidean space, it exactly recovers the isotropic Gaussian reference, remaining consistent with existing SSL theory. (ii) Explicit scale control: The budget ρ\rho constrains the expected squared geodesic distance from the reference point, so the maximum-entropy reference depends on both curvature and the permitted dispersion scale. (iii) Isolating curvature effects: Matching the second-moment budgets of the intrinsic and tangent-space references controls for differences in radial scale, allowing the effect of curvature-dependent Riemannian volume growth to be examined.

This choice also builds on prior work. Mathieu et al. (2019) distinguish the Riemannian normal, defined using squared geodesic distance, from the wrapped normal obtained through the exponential map, and discuss the maximum-entropy interpretation of the former. We use this established distributional construction to derive uniformity references across constant-curvature representation spaces.

2. Selection of ρ\rho

For K>0K>0, the budget ρ\rho controls geodesic dispersion around the reference point. Its interpretation is governed by the geometric threshold ρunif(K,d)\rho_{\mathrm{unif}}(K,d), the second geodesic moment of the uniform distribution on the hypersphere. When ρ<ρunif(K,d)\rho<\rho_{\mathrm{unif}}(K,d), the constraint is active and yields a scale-constrained maximum-entropy reference. When ρ≥ρunif(K,d)\rho\geq\rho_{\mathrm{unif}}(K,d), the constraint is inactive and the reference exactly recovers the uniform distribution on the hypersphere.

Experimentally, ρ\rho is a hyperparameter controlling representation scale: in node classification, it is chosen from a predefined grid using validation performance, while the ImageNet-100 controlled comparison fixes ρ\rho to examine geometric differences across curvatures. For example, at K=1K=1 and d=16d=16, ρunif≈2.530\rho_{\mathrm{unif}}\approx2.530; hence, the experimental choice ρ=2\rho=2 corresponds to an active scale constraint.

3. Connection to Wang & Isola (2020)

When K=1K=1 and ρ≥ρunif\rho\geq\rho_{\mathrm{unif}}, the two formulations recover the same reference distribution but use different finite-sample surrogates. A brief proof follows.

For K>0K>0, let σK=dVK/VK\sigma_K=dV_K/V_K be the normalized Riemannian volume measure, where VKV_K is the total manifold volume. By the non-negativity of KL divergence, for any distribution μ\mu,

H(μ)=log⁡VK−DKL(μ∥σK)≤log⁡VK,H(\mu)=\log V_K-D_{\mathrm{KL}}(\mu\|\sigma_K) \leq \log V_K,

with equality if and only if μ=σK\mu=\sigma_K. Thus, σK\sigma_K is the unique maximum-entropy distribution.

Define ρunif(K,d)=EσK[r2]\rho_{\mathrm{unif}}(K,d)=\mathbb{E}_{\sigma_K}[r^2]. When ρ≥ρunif(K,d)\rho\geq\rho_{\mathrm{unif}}(K,d), the distribution σK\sigma_K satisfies the second-moment constraint; hence,

ρ≥ρunif(K,d)⟹νK,ρ=σK,λ=0.\rho\geq\rho_{\mathrm{unif}}(K,d) \Longrightarrow \nu_{K,\rho}=\sigma_K,\quad\lambda=0.

In particular, for K=1K=1, Wang & Isola (2020, Proposition 1) show that the uniform distribution on the hypersphere is also the unique minimizer of their Gaussian-potential uniformity objective:

ν1,ρ=σ1=arg⁡min⁡μ∈P(Sd)Eu,v∼μ[e−t∥u−v∥22],t>0.\nu_{1,\rho}=\sigma_1=\arg\min_{\mu\in\mathcal P(\mathbb S^d)}\mathbb E_{u,v\sim\mu}\left[e^{-t\|u-v\|_2^2}\right],\quad t>0.

Therefore, the two formulations recover exactly the same population-level optimal reference distribution, but use different finite-sample surrogates: Wang & Isola employ a pairwise Gaussian potential, whereas we use radial quantile matching and conditional angular sliced-Wasserstein matching.

Reviewer_T58iOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper aims to evaluate and optimise the uniformity of learned representations across representation spaces of varying curvature in the setting of self-supervised learning. Specifically, this work defines uniformity as maximum entropy with respect to intrinsic Riemannian volume under a fixed geodesic scale, so that the resulting measure adjusts to varying curvatures and embedding scales. Performance is evaluated across 4 node classification tasks in comparison with GNNs and self-supervised hyperbolic baselines, specifically HGCL (Hyperbolic Graph Contrastive Learning). Other experiments compare the tangent-space transplanted distribution with the intrinsic MaxEnt reference, and investigate the relation between alignment and the intrinsic MaxEnt objectives during training.

Critical strengths

Scope and clarity of the contributions. The aim of the paper is clearly articulated and described in terms of some of the related works across the first two sections.

Critical weaknesses

Limited practical performance advantages. The comparison in Table 1 could be improved by not just highlighting the best performing model, but highlighting the datasets for which there is a critical difference between MaxEnt and the baseline models. As it stands, considering uncertainty in the results, MaxEnt seems to perform similarly to baseline methods on PubMed, CiteSeer and Cora, only showing substantial performance gains on the Airport dataset. The Disease dataset is mentioned in the code submission as another option, but results on this dataset aren’t reported. Similarly, the results on ImageNet-100 in Figure 4 only compare with one alternative baseline, over which MaxEnt shows limited performance benefits in terms of accuracy. Overall, these limited reported demonstrated performance gains over existing models is my main concern. While other experiments compare the tangent-space transplanted distribution with the intrinsic MaxEnt reference, and investigate the relation between alignment and the intrinsic MaxEnt objectives during training, it would be useful to provide further concrete evidence on the utility of MaxEnt for practical applications. For example, the node classification results, or the results on ImageNet could be extended to consider further datasets or baseline models. Similarly, the utility of MaxEnt over other uniformity objectives could further be evaluated and demonstrated in practice across applications beyond node prediction tasks.

Computational efficiency. The costs, time complexity, and computational efficiency of the proposed method is not reported on. Doing so, and also providing an empirical evaluation of runtime and memory usage compared to the baselines in Table 1 would strengthen the paper.

Minor comments.

  • The contents of the paper appear to be heavily AI generated, and the authors disclose substantial LLM usage and assistance in the AI use statement. A more detailed clarification on exactly which of the mathematical claims were formulated and derived using AI, which of the experiments were conducted with AI assistance, whether the text was initially generated by LLMs, and what specific type of AI assistance was used for the disclosed purposes would clarify the contributions.
  • Some of the intervals in Figure 4 are difficult to read and overlap with the markers. Ideally the legend should also specify what the intervals refer to (e.g. standard deviations calculated over specific data splits).
  • The background of Figure 5 is not white, which is distracting.
  • Based on the heavy review load and unfamiliarity with some of the background theory, I am finding it difficult to review the theoretical contributions. The correctness of the theoretical statements has not been carefully checked.
Questions for authors
  • Commenting on your method's practical advantages, could you provide extended results on a wider range of classification datasets to demonstrate robust and clear performance gains of your method?
  • Could you extend your evaluation towards graph classification tasks or alternatively, other applications that could demonstrate the utility of your proposed uniformity loss over alternative approaches?
  • Could you report on the time complexity of your method as it scales with dataset or graph sizes and for example evaluate runtimes and memory usage empirically compared to the other models reported in Table 1?
3 comments
AuthorsOpenReview ↗

Author Response [1/3]

W1/Q1/Q2: Practical Utility, Cross-Task Generalization, and Scope of Contribution

1. Comparison with Other SSL and Uniformity Objectives Beyond Node Classification

To directly evaluate the practical utility of Intrinsic MaxEnt beyond node classification and against alternative uniformity and SSL objectives, we conducted a controlled five-objective comparison on STL-10: Intrinsic MaxEnt, tangent transplant, SimCLR (InfoNCE), Wang–Isola, and VICReg. The Wang–Isola branch optimizes its native hyperspherical uniformity loss, jointly with its standard alignment term on unit-normalized representations:

Lunif=log⁡Eu,v[exp⁡(−t∥u−v∥22)]\mathcal{L}_{\mathrm{unif}} = \log \mathbb{E}_{u,v} \left[ \exp\left(-t \lVert u - v \rVert_2^2\right) \right]

Within each preset configuration, all five methods share the same randomly initialized ResNet-18 encoder, projection dimension, data augmentations, and optimizer settings, trained from scratch for 50 epochs and evaluated with an identical frozen-backbone linear probe. The three columns vary projection dimension, augmentation strength, and optimization settings:

Table 2-1. Linear-probe test accuracy (%) on STL-10 across five SSL objectives from random initialization.

ObjectiveWeak (d=4)Standard (d=16)Strong (d=64)
Intrinsic MaxEnt57.7572.9870.28
Tangent transplant56.2874.3070.84
SimCLR (InfoNCE)52.0373.0675.94
Wang–Isola54.8973.5945.11
VICReg57.1970.1641.36

As shown in Table 2-1, Intrinsic MaxEnt performs competitively with established SSL objectives—including the native hyperspherical uniformity loss of Wang–Isola—in a visual representation setting beyond node prediction. Notably, whereas Wang–Isola and VICReg degrade sharply under the strong, higher-dimensional configuration, Intrinsic MaxEnt maintains stable accuracy across all three regimes.

2. Extension to Graph-Level Classification

Our uniformity regularizer operates solely on learned embeddings and does not rely on node-specific graph structures; thus, it applies directly to graph-level, node-level, and image-level representations.

To verify its practical transferability beyond transductive node classification, we evaluate inductive graph classification on two standard molecular and protein graph benchmarks, MUTAG and PROTEINS. We compare against all Table 1 baselines, adapting node-level methods with graph-level readout where applicable. Specifically, "GC" denotes graph-level adaptation: GRACE, COSTA, and HyperGCL use a shared GIN encoder and pool node embeddings into graph representations before computing their respective objectives, whereas HGCL retains its node-level HPC loss and then pools the embeddings for graph classification. For HGCL, the joint variant keeps its supervised task loss while the HPC-only variant is purely self-supervised. All methods share identical five-fold splits, a 100-epoch training budget, and validation-only model selection, with each fold averaged over two model seeds. For Intrinsic MaxEnt, we set K=−3 and ρ=6. Results are reported in Table 2-gc:

Table 2-gc. Inductive graph classification accuracy (%) on MUTAG and PROTEINS, reported as mean ± sample SD across five folds.

TrainingSpaceMethodMUTAGPROTEINS
SupervisedEuclideanGCN81.59 ± 6.0274.53 ± 3.15
EuclideanGAT83.75 ± 3.8375.56 ± 2.50
EuclideanGraphSAGE79.46 ± 5.6075.42 ± 2.92
HyperbolicHNN80.26 ± 5.8574.21 ± 2.74
HyperbolicHGNN81.88 ± 4.1075.02 ± 3.16
HyperbolicHGCN73.91 ± 5.6073.59 ± 2.95
HyperbolicHGAT80.56 ± 4.9174.66 ± 3.94
HyperbolicHIE75.49 ± 6.2874.03 ± 1.82
HyperbolicHGCL (joint)80.82 ± 5.4876.28 ± 3.52
Self-supervisedEuclideanGRACE-GC84.80 ± 4.0475.38 ± 3.26
EuclideanCOSTA-GC85.87 ± 3.2374.48 ± 3.52
HyperbolicHGCL (HPC-only GC)82.71 ± 2.6574.35 ± 2.49
HyperbolicHyperGCL-Laplace-GC85.86 ± 4.6675.52 ± 2.89
HyperbolicHyperGCL-Normal-GC84.54 ± 4.2675.02 ± 3.00
HyperbolicIntrinsic MaxEnt (Ours)86.68 ± 3.5275.70 ± 3.19

As shown in Table 2-gc, Intrinsic MaxEnt achieves the highest mean accuracy among all self-supervised methods, reaching 86.68 ± 3.52% on MUTAG and 75.70 ± 3.19% on PROTEINS, and remains competitive with all supervised baselines, including jointly supervised HGCL on PROTEINS. The results confirm that Intrinsic MaxEnt transfers cleanly to inductive graph-level tasks.

AuthorsOpenReview ↗

Author Response [2/3]

3. Clarification of Our Central Thesis and Table 1 Interpretation

We appreciate the reviewer's careful reading. Although the improvements over strong baselines on PubMed, CiteSeer, and Cora are incremental once variance is accounted for, our goal is not SOTA-driven benchmarking. More broadly, we wish to clarify that our central objective is not to claim universal downstream accuracy superiority over task-specific SSL pipelines, but to formulate a unified, geometry-consistent variational principle for SSL uniformity across constant curvatures and to identify and verify the resulting structural gap between intrinsic and tangent-space references. Accordingly, we pursue two complementary goals: (1) characterizing the intrinsic–tangent gap as a formally testable geometric signature of curvature-dependent volume growth (Sec. 5, Exps. 7.1/7.3); and (2) demonstrating that intrinsic maximum entropy functions as a viable, well-behaved cross-curvature metric and regularizer (Sec. 4, Exps. 7.2/7.4). Much like the anomalous perihelion precession of Mercury in curved spacetime—whose scientific value lay not in its magnitude but in revealing geometric structure invisible to the flat classical description—the significance of Intrinsic MaxEnt lies in exposing and operationalizing this underlying geometric distinction rather than chasing SOTA margins.

W2/Q3: Computational Complexity, Scalability, and Empirical Costs

1. Asymptotic Complexity and Graph-Size Scaling

For a graph with N nodes and E edges, given fixed feature dimensions, slice count M, and radial bins B, one training step of Intrinsic MaxEnt has a sorting-based time complexity of O(E + N log N) and O(N) additional loss workspace. Reference quantiles are precomputed and cached, and the objective avoids constructing an N × N pairwise similarity matrix.

To evaluate scalability empirically, we use synthetic graphs with 128-dimensional node features, a GCN hidden width of 128, and 128-dimensional embeddings, fixing M=64, B=4, and mean degree 8. All measurements use an RTX 6000D in FP32. Scaling N by 4× from 65,536 to 262,144 increases training time from 27.44 to 96.59 ms/epoch (3.52×) and peak allocated GPU memory from 1,073.4 to 4,080.3 MiB (3.80×), confirming approximately linear scaling in both time and memory.

2. Computational Efficiency Against All Baselines

We benchmarked Intrinsic MaxEnt alongside all 13 Table 1 baselines across Airport, Cora, CiteSeer, and PubMed on the same RTX 6000D GPU using FP32 precision and their respective Table 1 configurations. Each result was averaged over three random seeds, with five warmup epochs followed by 25 CUDA-synchronized timed epochs per seed. Validation and linear probing were excluded from the timing measurements. Table 2-2 reports the results on PubMed, the largest graph in Table 1.

Table 2-2. Training runtime and peak allocated GPU memory on PubMed (19,717 nodes) for all 14 methods from Table 1.

Methodms/epochGPU MiB
GCN3.97 ± 0.02230.7
GAT7.16 ± 0.24585.3
GraphSAGE2.54 ± 0.49393.1
HNN6.64 ± 1.11227.9
HGNN10.52 ± 0.47393.1
HGCN9.66 ± 0.30301.2
HGAT16.11 ± 0.22621.1
HIE10.83 ± 0.03306.7
GRACE10.97 ± 0.01361.6
COSTA10.95 ± 0.03441.7
HGCL34.20 ± 0.072343.9
HyperGCL-Laplace16.92 ± 0.30304.7
HyperGCL-Normal16.52 ± 0.14355.8
Intrinsic MaxEnt (Ours)15.18 ± 0.52518.1

On PubMed, Intrinsic MaxEnt runs in 15.18 ± 0.52 ms/epoch with 518.1 MiB peak memory. It is faster than HyperGCL-Normal (16.52 ms), HyperGCL-Laplace (16.92 ms), and HGAT (16.11 ms), while using less memory than GAT (585.3 MiB) and HGAT (621.1 MiB), and over 2× faster with 4.5× less memory than HGCL (34.20 ms, 2343.9 MiB).

Responses to Minor Comments

1. Clarification on AI Assistance

We thank the reviewer for requesting a more granular breakdown. Consistent with our AI Use Statement (p. 10), we clarify each point below:

(i) Mathematical claims and derivations: Definition 1 and Propositions 1, 2, and 4 (along with their derivations in Appendices B.1, B.2, and B.4) were formulated or organized by the authors, with AI used at most for standardizing parts of the notation and presentation. For Proposition 3 (Appendix B.3), AI provided partial ideas and intermediate derivation steps, all of which were independently checked and verified by the authors.

(ii) Experiments: All experimental designs and training/evaluation runs (Sections 7.1–7.4 and Appendices C–E) were designed, executed, and validated by the authors. AI coding assistants were used strictly for auxiliary engineering tasks, specifically boilerplate logging, portions of the post-hoc analysis and synthetic perturbation scripts (e.g., Appendix E), and matplotlib visualization code, all of which were reviewed line by line and verified by the authors.

AuthorsOpenReview ↗

Author Response [3/3]

(iii) Text authoring: The manuscript was not initially generated by LLMs. All sections, technical arguments, and experimental discussions were originally drafted by the authors; LLMs were used for preliminary literature discovery and subsequent language and LaTeX polishing of author-written drafts. All cited references were independently verified by the authors.

(iv) Conceptual illustrations: All schematic figures in the paper (Figures 1, 2, and 6) were manually created by the authors in PowerPoint without any AI involvement.

2. Figure 4 Error-Bar Visibility

All three panels include error bars showing ± one sample standard deviation across three paired seeds on fixed data splits. Their visibility differs because of their lengths relative to the markers. To quantify this, we mapped the original numerical values to their corresponding lengths in the rendered figure. The resulting nonzero error bars have total heights of 0.42–4.23 pt in (a) and 0.002–4.20 pt in (b), compared with 5.53–25.83 pt in (c), while the markers are 3.7 pt in size. Consequently, many error bars in (a) and (b) are obscured by the markers. We will clarify this and the error-bar definition in the figure legend.

3. Figure 5 Background Consistency

Figure 5 retained a light background from the plotting style. We have replaced it with a white background in the revised version to improve readability and visual consistency.

Reviewer_VAyjOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

In self-supervised learning, "uniformity" has been defined separately for each space. On the hypersphere it means the uniform measure, in Euclidean space it means an isotropic Gaussian, in hyperbolic space people push a tangent-space Gaussian through the exponential map. The authors try to put all three under one principle. The uniformity reference is the maximum-entropy distribution with respect to the space's own Riemannian volume, under a fixed geodesic second-moment budget $E

r2r^2

\le \rho$. So uniformity depends jointly on curvature KK and scale ρ\rho.

Critical strengths
  • The τ4\tau^4 gap result is solid. People already suspected that wrapped normals don't match intrinsic geometry. This paper says exactly how much: the mismatch can't be fixed by retuning the scale, it grows as �0�4�0�^4 near flat space, and it keeps growing in hyperbolic space. That gives practitioners a rule of thumb. If τ\tau is small, tangent-space Gaussians are fine; if τ\tau is large, they aren't.
  • The unifying framing is clear. Recovering the spherical uniform and Euclidean Gaussian references from one principle is tidy. It explains why the scale constraint is inactive on the sphere but required in flat and hyperbolic space.
Critical weaknesses
  • As far as I know, the theory described is mostly not new. Maximum entropy under a second-moment constraint giving a Riemannian normal is classical (Pennec 2006; Said et al. 2014). Mathieu et al. (2019) already used Riemannian normals as hyperbolic priors and compared them with wrapped normals. The two canonical recoveries are close to immediate, and the radial–angular split is the KL chain rule. What's new is the SSL framing and the asymptotic analysis of the gap.

Pennec, X. (2006). Intrinsic statistics on Riemannian manifolds: Basic tools for geometric measurements. Journal of Mathematical Imaging and Vision, 25(1), 127–154.

Said, S., Bombrun, L., & Berthoumieu, Y. (2014). New Riemannian priors on the univariate normal model. Entropy, 16(7), 4015–4031.

Mathieu, E., Le Lan, C., Maddison, C. J., Tomioka, R., & Teh, Y. W. (2019). Continuous hierarchical representations with Poincaré variational auto-encoders. In Advances in Neural Information Processing Systems (NeurIPS), 32.

Questions for authors
  • What τ\tau do real hyperbolic SSL systems actually operate at? This decides whether the gap matters in practice.
  • The baselines in Table 1 are far below their published results. The authors' own method (78.91 / 82.26 / 71.40 / 80.66) is below HyperGCL's published numbers on every dataset. It "wins" only against these re-run, much weaker baselines. Would the authors mind commenting on this?
2 comments
AuthorsOpenReview ↗

Author Response [1/2]

W1: Theoretical Novelty and Contributions

We thank the reviewer for this thoughtful assessment and wish to clarify the scope of our theoretical contribution. Consistent with your observation, our central goal is not to claim the discovery of classical geometric distributions or isolated information-theoretic identities, such as the maximum-entropy characterization of the Riemannian normal (Pennec, 2006; Said et al., 2014), its use as a hyperbolic VAE prior compared against wrapped normals (Mathieu et al., 2019), or the KL chain rule, but rather to establish a unified variational framework for SSL uniformity across constant curvatures based on the intrinsic maximum-entropy principle. To this end, we pursue two complementary directions: (1) characterizing the intrinsic–tangent gap as a formally testable geometric signature that cannot be eliminated by scale retuning, quantified through its exact τ⁴ asymptotic scaling (Prop. 3, Sec. 5, Exps. 7.1/7.3); and (2) operationalizing this generalized notion of uniformity into an effective cross-curvature metric and differentiable regularizer for SSL representations (Secs. 4 and 6, Exps. 7.2/7.4).

Within this unified framing, the classical results serve as necessary building blocks rather than standalone claims of novelty. While Mathieu et al. (2019) study Riemannian and wrapped normals in the context of hyperbolic VAEs, our Proposition 1 brings compact (K>0) and non-compact (K≤0) SSL geometries under a single geodesic second-moment constraint whose activity transitions naturally at ρ_unif. Similarly, although Proposition 4 follows directly from the KL chain rule, its purpose is operational: it provides the exact radial–angular decomposition that connects the ideal but computationally intractable minibatch KL objective to our differentiable, minimizer-preserving Wasserstein surrogates (Eqs. 13–14). In short, these classical results serve as the mathematical foundation upon which our SSL framing, asymptotic gap analysis, and tractable cross-curvature surrogates are built.

Q1: Practical Curvature–Scale Regimes

Practical operating regimes depend on both the task and the radial spread of learned embeddings. In our graph benchmarks, validation-selected budgets fall in τ=|K|ρ∈[0.9,2.0], where curvature effects remain modest. In the hyperbolic ImageNet-100 experiments (τ∈{2,6,18}), mean scale alone understates the curvature experienced by outer-shell samples. Reporting both the realized mean scale τ̂ and the 95th-percentile tail scale τ₀.₉₅=|K|Q₀.₉₅(r)² (intrinsic / tangent): at target τ=2, τ̂ is 1.40 / 1.40 with τ₀.₉₅=2.67 / 2.70; at τ=6, τ̂ is 3.64 / 3.68 with τ₀.₉₅=7.15 / 7.59; and at (K,ρ)=(-3,6) (τ=18), τ̂ is 6.57 / 7.06 while τ₀.₉₅ reaches 15.07 / 17.93. Thus, outer samples enter substantially stronger curvature regimes than the mean indicates.

Accordingly, the practical impact of the gap is regime-dependent: small near τ≈1, but measurable when embeddings extend into larger τ. At (K,ρ)=(-3,6), two exploratory analyses on the paired runs show effects beyond simple radial rescaling: (i) matching class-prototype second moments still preserves a 6.1% reduction in WordNet stress (0.3920 → 0.3683); and (ii) in a controlled intervention holding directions and radial ranks fixed on frozen features, native-distance retrieval R@1 increases by 1.95 percentage points (36.85% → 38.80%).

Q2: Discrepancies with Published HyperGCL Results

We thank the reviewer for highlighting this discrepancy. We acknowledge that the HyperGCL results in Table 1 fall below the accuracy reported in the original paper. As we could not locate an official public implementation of HyperGCL, we independently reproduced the method based on the paper's description. However, without the original code and dataset-specific configurations, it is difficult to ensure exact consistency in data preprocessing, dataset splits, linear evaluation, and model selection.

To further investigate this discrepancy and assess the relative performance of both methods under a unified evaluation protocol, we conducted an exploratory experiment on Airport. Specifically, HyperGCL-Normal and Intrinsic MaxEnt used identical data splits, raw continuous node features, StandardScaler normalization, validation-based model selection, and five random seeds, with both trained for 400 epochs. Under this unified protocol, HyperGCL-Normal improved from 72.89±1.82% in our initial reproduction to 81.34±1.00%, while Intrinsic MaxEnt achieved 81.09±1.66%, yielding comparable performance. These results indicate that preprocessing and evaluation protocols substantially affect reproduction outcomes and can partially account for the discrepancy with the originally reported 94.55% accuracy, although the remaining gap cannot be fully explained.

AuthorsOpenReview ↗

Author Response [2/2]

This exploratory comparison suggests that, while the absence of official code prevents exact reproduction of the published results, a controlled comparison under an identical evaluation protocol remains feasible. Under this protocol, Intrinsic MaxEnt performs competitively with the reproduced HyperGCL-Normal baseline. We will clarify the scope and limitations of the Table 1 comparison, together with the sources of the observed discrepancy, in the revised manuscript.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.