- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper defines a uniformity reference in a constant-curvature representtion space as the maximum-entropy distribution, relative to Riemannian volume, under a budget on the squared geodesic distance from a reference point (Eq. 4). The solution is an isotropic Gibbs density (Eq. 5), which recovers the uniform measure on the sphere (when the budget is inactive). A tangent-space Gaussian pushed through the exponential map is shown to differ from this reference by a KL gap that scale retuning cannot remove. KL is then split into radial and angular terms (Prop. 4). Experiments covered a numerical check of the gap, hyperbolic node classification, an ImageNet-100 comparison, and an alignment/MaxEnt study
Critical strengths
The derivations in section 5 look (mostly) correct. The authors are at least candid that the effect size of the gap appears to be very modest (L437-439).
Critical weaknesses
The mathematics appears correct but disconnected from the experiments and the experiments do not support the paper's central claim that the intrinsic reference is the better one to train towards. Overall, the framing of novelty and scope is in my opinion not empirically or theoretically supported. Three problems seem to me individually serious: (1) the node-classification evidence is confounded; (2) the learned-representation evidence appears largely circular and rests on a label-informed initialisation; and (3) the gap is never shown to matter at the scales used. I am also concerned about novelty (4).
-
The method (Eq. 16, no InfoNCE or negatives) is compared with full methods using other objectives, and no row runs the same pipeline with the tangent reference or K = 0. K, rho and alpha are not reported, and tuning is per dataset for the proposed method only. Margins are within noise on three of four datasets: PubMed accuracy, PubMed Macro-F1, and CiteSeer balanced accuracy.
-
Section 7.3 is close to tautological and starts from a labelled initialisation. Eq. 13 matches radii to the chosen reference's quantiles, showing only that the optimiser reaches its target. Stress and Top-1 results are weak, with overlapping error bars in Figure 4c and no seed count. The ResNet-34 is supervised on ImageNet-1K (L415-417), and Appendix D has no tangent condition (why not?). In 7.4, D_rad and D_ang are the training losses (L462-463), so their rise without MaxEnt holds by construction.
-
The gap is never shown to matter at the scales used. Integrating Eq. 7 at d = 16, I obtain KL gaps of 0.155 nats at (K, rho) = (-3, 6), 0.019 at (-3, 2), 0.005 at (+1, 2) and 0.0008 at (-1, 2), so the gap is appreciable only for tau >= 5, where Eq. 11 no longer holds. I also don't put a lot of stock in the boundary-drift remark at the end of section 5.
-
The Gibbs form is the Riemannian normal, Prop. 2 is a change of variables, and Prop. 1 holds by construction, since E[r^2] is exactly the budget whose maximiser in R^d is the Gaussian. The surrogates in Eqs. 13-14 are not linked to Prop. 4, and no native uniformity loss is compared.
Questions for authors
-
Which (K, rho, alpha) were used for Table 1? Please add rows with the identical pipeline under the tangent reference and under K = 0, and state the tuning budget given to the baselines.
-
How does the gap at each setting in 7.3 compare with the minibatch noise floor of the loss (Figure 8), and how many seeds support Figure 4?
-
Can the tangent vs. intrinsic comparison be repeated from a random or SSL-pretrained initialisation, with the tangent scale tuned?
-
Why the second-moment budget, and how is o justified for K > 0? At rho >= rho_unif, does the objective match the standard Wang & Isola loss in the same pipeline?
5 comments
Author response [2/5]
3. The Tuning Budget of the Baselines
For each baseline, hyperparameters are first selected using validation accuracy from an initial seed; the selected configuration is then fixed, and the test mean and sample standard deviation are reported over five additional seeds. All supervised baselines use Adam with a learning rate of , weight decay of , and dropout of , with model selection based on validation accuracy. GCN, GAT, and GraphSAGE use PyG convolutional layers with two-layer architectures and 64-dimensional hidden representations; GAT uses four attention heads. HGCN and HIE use the Poincaré ball with learnable curvature. These encoder configurations are held fixed across datasets. GRACE, COSTA, and HGCL also use fixed encoder configurations, with the linear-classifier regularization parameter selected on the validation set. For HyperGCL, we evaluate 20 configurations over five seeds, varying curvature, loss weights, data augmentations, representation dimension, and learning rate.
Q2/W3: Signal-to-Noise Comparison and Seed Count
First, the KL gap and the Wasserstein surrogate losses in Figure 8 use different metrics, so their numerical magnitudes are not directly comparable. The two references share the same uniform angular target; their discrepancy is entirely radial. To assess this discrepancy on the scale of the training objective relative to minibatch sampling noise, we compute the normalized radial Wasserstein gap , the population counterpart of Eq. (13). Under the configurations used in Section 7.3, we then independently sample 4,096 minibatches from the intrinsic reference and compare the radial gap against the zero-perturbation sampling noise floor. The results are as follows:
Table 3. Calibration of the radial gap against minibatch sampling noise. All numerical values are scaled by .
| Radial gap | Noise floor | Gap / floor ratio | |
|---|---|---|---|
For and , the radial gap exceeds the single-minibatch noise floor by factors of and , respectively, consistent with the separation in radial-tail ratio and normalized stress in Figure 4. For and , the gap is below or close to the noise floor, and the two reference conditions yield nearly overlapping curves. Thus, we acknowledge that the effect is scale-dependent, while the results show that it can influence learned representation geometry when curvature effects are substantial.
Figure 4 uses three paired model seeds for each configuration; error bars denote the sample standard deviation across those seeds. Five configurations and two reference distributions yield 30 final checkpoints in total.
Regarding W3's concern about the validity range of the asymptotic approximation in Eq. (11), it is important to distinguish the exact result from its near-flat expansion. Under the assumptions of Proposition 3, holds for nonzero curvature, whereas describes only the local behavior as . For larger , the KL gap is computed by numerical integration of the full radial densities rather than by the fourth-order approximation. Thus, the breakdown of the asymptotic approximation at larger scales does not invalidate the exact gap result.
Regarding the boundary-drift remark at the end of Section 5, we agree that our experiments do not directly establish this phenomenon. The remark was intended to motivate possible implications of curvature-dependent volume growth rather than present boundary drift as an experimentally established effect. We have therefore revised the statement to keep it within the scope of the demonstrated geometric result:
Line 275: "This highlights the fundamental role of curvature-dependent volume growth in shaping the radial organization of hyperbolic representations."
Author response [3/5]
Q4: Theoretical Justification and Classical Recovery
1. Motivation for the Geodesic Second-Moment Constraint
Our goal is to extend uniformity from specific representation spaces to Riemannian manifolds of different curvatures while recovering canonical uniformity references under the appropriate conditions. We therefore adopt a geodesic second-moment constraint, motivated by canonical recovery properties and a consistent interpretation of representation scale across geometries. In particular, this constraint has three useful properties:
(i) Canonical recovery: In Euclidean space, it exactly recovers the isotropic Gaussian reference, remaining consistent with existing SSL theory. (ii) Explicit scale control: The budget constrains the expected squared geodesic distance from the reference point, so the maximum-entropy reference depends on both curvature and the permitted dispersion scale. (iii) Isolating curvature effects: Matching the second-moment budgets of the intrinsic and tangent-space references controls for differences in radial scale, allowing the effect of curvature-dependent Riemannian volume growth to be examined.
This choice also builds on prior work. Mathieu et al. (2019) distinguish the Riemannian normal, defined using squared geodesic distance, from the wrapped normal obtained through the exponential map, and discuss the maximum-entropy interpretation of the former. We use this established distributional construction to derive uniformity references across constant-curvature representation spaces.
2. Selection of
For , the budget controls geodesic dispersion around the reference point. Its interpretation is governed by the geometric threshold , the second geodesic moment of the uniform distribution on the hypersphere. When , the constraint is active and yields a scale-constrained maximum-entropy reference. When , the constraint is inactive and the reference exactly recovers the uniform distribution on the hypersphere.
Experimentally, is a hyperparameter controlling representation scale: in node classification, it is chosen from a predefined grid using validation performance, while the ImageNet-100 controlled comparison fixes to examine geometric differences across curvatures. For example, at and , ; hence, the experimental choice corresponds to an active scale constraint.
3. Connection to Wang & Isola (2020)
When and , the two formulations recover the same reference distribution but use different finite-sample surrogates. A brief proof follows.
For , let be the normalized Riemannian volume measure, where is the total manifold volume. By the non-negativity of KL divergence, for any distribution ,
with equality if and only if . Thus, is the unique maximum-entropy distribution.
Define . When , the distribution satisfies the second-moment constraint; hence,
In particular, for , Wang & Isola (2020, Proposition 1) show that the uniform distribution on the hypersphere is also the unique minimizer of their Gaussian-potential uniformity objective:
Therefore, the two formulations recover exactly the same population-level optimal reference distribution, but use different finite-sample surrogates: Wang & Isola employ a pairwise Gaussian potential, whereas we use radial quantile matching and conditional angular sliced-Wasserstein matching.
Author response [1/5]
We are grateful for the reviewers’ insightful comments.
We respectfully wish to clarify a possible misunderstanding regarding our central thesis. As stated in our title and contributions, our central goal is not to establish the performance superiority of the intrinsic reference over tangent-space approximations, but to develop a unified theoretical framework for SSL uniformity across different curvatures based on the intrinsic maximum-entropy principle. To this end, we pursue two complementary directions: (1) characterizing the gap between the intrinsic and tangent-space references as a formally testable geometric signature of this generalized notion (Sec. 5, Exps. 7.1/7.3); and (2) investigating whether intrinsic maximum entropy can serve, like classical uniformity, as an effective cross-curvature metric and regularizer for SSL representations (Sec. 4, Exps. 7.2/7.4). Although a substantial portion of our mathematical analysis focuses on the intrinsic–tangent gap, its purpose is to establish a formal, observable distinction supporting the proposed generalized notion of uniformity. Our central claim is that a more general, geometry-consistent notion of uniformity can be rigorously formulated and meaningfully applied across representation geometries.
A useful analogy for clarifying the scope of our contribution is the transition from the classical mechanical description of gravity to its geometric formulation in curved spacetime. In this transition, the anomalous perihelion precession of Mercury, though exceedingly small in magnitude, provided crucial observable evidence distinguishing the two theoretical frameworks. Its significance lies not in the magnitude of the discrepancy, but in revealing the underlying geometric structure that the classical description could not fully capture. Similarly, the intrinsic–tangent KL gap in our work is intended to provide a quantifiable signature of the geometric distinction, rather than to serve as a measure of SOTA performance improvement.
Point-by-Point Responses to Experimental Concerns
W1/Q1: Hyperparameter Settings, Tuning Fairness, and Controlled Ablations
1. Hyperparameter Configuration and Tuning Protocol
We select the hyperparameters based on validation performance through grid search, including curvature K, scale budget ρ, regularization weight α, number of slicing directions M, and number of radial bins B. Table 1 summarizes their search spaces and the selected configurations for each dataset, together with the dimensionless curvature–scale parameter τ=∣K∣ρ.
Table 1. Dataset-specific hyperparameter settings for the controlled ablation. The dimensionless scale is defined as τ=∣K∣ρ.
2. Controlled Comparison under the Table 1 Pipeline
To isolate the effect of reference selection, we additionally conduct a three-arm ablation on the same four datasets under the identical Eq. (16) training pipeline: (i) the Euclidean control (K=0); (ii) the tangent transplant reference; and (iii) the Intrinsic MaxEnt reference. For the curved-space branches, curvature K and scale budget ρ are fixed; the K=0 branch uses the flat-limit version of the same training pipeline, with all other hyperparameters unchanged. All three branches share the same two-layer GCN encoder, data augmentations, optimizer settings, five random seeds and linear-probe selection protocols. The results are reported below:
Table 2. Controlled three-arm comparison of node-classification accuracy (%). Results are reported as mean ± sample standard deviation over five random seeds.