- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper addresses geometric distortion in existing hyperbolic kernel methods, where mapping hyperbolic representations into an RKHS does not necessarily preserve the original Riemannian geometry.
To address this issue, the authors introduce Riemannian isometric kernels, whose canonical feature maps preserve the Riemannian metric of the input manifold. They establish theoretical conditions for Riemannian isometry and construct four positive-definite kernels: RILog, RIBin, RIExp, and RIInt. They also prove that the kernel-induced distances are monotonically increasing functions of the original geodesic distance.
The proposed methods are evaluated through geometric stress analysis and four downstream tasks: zero-shot learning, few-shot learning, graph node classification, and semantic textual similarity. The experimental results show that the proposed kernels generally outperform existing hyperbolic kernel methods.
Critical strengths
The paper provides a mathematically well-motivated approach to constructing hyperbolic kernels that preserve intrinsic Riemannian geometry. The authors establish sufficient conditions for Riemannian isometry and construct four positive-definite kernels satisfying these conditions. The theoretical development is a meaningful contribution.
The authors evaluate their methods across multiple tasks, including zero-shot learning, few-shot learning, graph node classification, and semantic textual similarity. The proposed kernels achieve competitive or superior performance across a broad range of benchmarks, suggesting their practical usefulness.
Critical weaknesses
1. About style
First, the font sizes in Tables 1–5 appear to have been noticeably reduced compared to the main text. This seems to violate the official ICLR 2027 formatting guidelines, which explicitly state: “Do not change any aspects of the formatting parameters in the style files” and “do not change font sizes (except perhaps in the References section).”
I strongly encourage the authors to revise these tables to comply with the official formatting requirements. If the tables do not fit within the available space, the authors should consider wrapping text onto multiple lines, restructuring the tables, or splitting them into smaller tables rather than reducing the font size.
2. The geometric preservation claim is not directly validated by the experiments.
The main motivation of this paper is to eliminate geometric distortion introduced by kernel mappings. However, the geometric stress experiment in Section 5.1 compares embeddings obtained from separately trained models using different similarity measures. This evaluates how similar the learned distance structures are, rather than directly measuring the geometric distortion introduced by the kernel mapping itself.
Consequently, although the proposed methods achieve lower stress values, these results do not directly demonstrate that the proposed kernels preserve the original Riemannian geometry better than existing kernels.
3. The relationship between Riemannian isometry and downstream performance improvements remains unclear.
Although the proposed kernels achieve promising results on several downstream tasks, the experiments do not establish whether these improvements are attributable to Riemannian isometry or simply to the different distance functions used during training.
In particular, the kernel-induced distances are monotonic transformations of the geodesic distance, and different transformations can influence optimization and predictive performance independently of exact Riemannian isometry.
Therefore, additional controlled comparisons would be necessary to clarify the specific practical advantage of the proposed isometric property.
4. Please check the mathematical notation
There appear to be several notation issues in the main text. Could the authors please carefully check and correct the following?
-
Equations (2)–(3) (pp. 4–5): The local coordinates are introduced, but the partial derivatives are written with respect to . The notation should be clarified and made consistent.
-
Theorem 2 (p. 5, lines 213–220): The definition of includes the term . Although follows from the definition of , explicitly stating this convention would improve clarity.
-
RIExp (p. 6, lines 235–238): The valid range of the parameter is not explicitly specified. The authors should clarify that , as required by the construction in Theorem 2.
-
Theorem 3 (p. 6, lines 255–258): The notation is used, although and . Should this instead be ?
Although the main ideas of the paper are understandable, the inconsistent notation causes considerable confusion and makes the paper unnecessarily difficult to follow. The authors are therefore encouraged to carefully review and correct the notation issues mentioned above.
In addition, the authors are encouraged to carefully check the appendix for similar notation issues and ensure consistency throughout the paper.
Questions for authors
1. Regarding the geometric stress experiment in Section 5.1
As I understand it, the authors compare the distances between embeddings obtained from separately trained models, rather than directly evaluating the distortion introduced by the kernel mapping. Is this interpretation correct? If so, could the authors explain how this experiment directly supports the geometric preservation claim?
I believe a more direct experiment would be useful. For example, the authors could compare the original Riemannian metric with the kernel-induced metric at sampled points, or compare curve lengths before and after the kernel mapping. These evaluations are theoretically feasible and would provide more direct evidence for the central claim of the paper.
2. Regarding the downstream experiments
Could the authors provide controlled experiments to clarify whether the performance improvements are specifically due to Riemannian isometry? For example, comparing the proposed kernels with non-isometric transformations of the same kernel distances under matched training conditions would help distinguish the effect of geometric preservation from that of changing the distance function.
3. Regarding the differences between RILog and RIBin
According to the kernel definitions, the squared kernel distances of RIBin approach those of RILog as the parameter t approaches zero. Appendix E states that t = 0.001 is used, and Appendix D.4 reports the same curvature c = 0.001 for both methods on AWA1 and AWA2. Under the clipping procedure described in Appendix D.1, their distances should therefore be extremely close. However, Table 2 reports a substantial difference between RILog and RIBin, particularly on AWA2 (76.3 versus 63.7 in harmonic mean). Could the authors clarify the reason for this discrepancy and provide results across multiple random seeds?
5 comments
3. Geodesic curve-length preservation.
A direct consequence of Riemannian isometry is that the length of every smooth curve is preserved under the canonical feature map. We numerically evaluate this property using the kernel-induced RKHS distance.
We consider a 5-dimensional Poincaré ball with curvature parameter and generate 500 random geodesics. The hyperbolic radial distance of each starting point from the origin is sampled uniformly from , and the geodesic length is sampled uniformly from . We uniformly sample points along each geodesic.
For each kernel , denoting the canonical map as , the RKHS curve length is approximated by the sum of consecutive RKHS distances:
We compare it with the original hyperbolic geodesic length ( is the geodesic distance of the Poincaré ball) using the absolute relative error
\frac{L_{\mathcal H_k}^{(N)}}{L_{\mathbb H}}-1 \right|.$$ We evaluate RIBin ($t=1$), RIInt, RIExp ($\lambda=1$), and RILog (with the origin as the reference point). The results are: | Kernel | Mean ratio | MAE | RMSE | Max error | |---|---:|---:|---:|---:| | RIBin | 0.999971 | 2.9e-5 | 3.8e-5 | 8.4e-5 | | RIInt | 0.999954 | 4.6e-5 | 6.0e-5 | 1.34e-4 | | RIExp | 0.999954 | 4.6e-5 | 6.0e-5 | 1.34e-4 | | RILog | 0.999989 | 1.1e-5 | 1.5e-5 | 3.4e-5 | Here, MAE, RMSE, and maximum error are computed from the absolute relative errors across the 500 geodesics. To examine the effect of numerical discretization, we further repeat the experiment with $N\in\{5,10,20,50,100,200,500\}$ points per geodesic, using the same set of 500 geodesics. The MAE consistently decreases as the sampling density increases. For example, for RIBin, it decreases from $6.03\times10^{-2}$ at $N=5$ to $2.9\times10^{-5}$ at $N=200$ and $4.6\times10^{-6}$ at $N=500$. The empirical convergence slopes, obtained by fitting the log-MAE against $\log(1/(N-1))$ for $N\geq20$, are approximately 2 for all four kernels. This is consistent with the second-order convergence of polygonal approximations to the arc length of a smooth curve. **4. Angle preservation.** We further evaluate angle preservation, another direct consequence of the preservation of the Riemannian metric. We generate 100 random non-degenerate geodesic triangles in a 3-dimensional Poincaré ball with $c=0.01$. At a selected vertex $p$ of each triangle, we consider two geodesic directions and construct short geodesic segments along them, controlled by a segment parameter $\epsilon$. Let $a_\epsilon$ and $b_\epsilon$ denote the endpoints of these segments. The original hyperbolic angle is computed using the hyperbolic law of cosines. $$\theta_{\mathbb H}=\arccos \left( \frac{ \cosh\left(\sqrt{c}\,d_{\mathbb H}(p,a_\epsilon)\right) \cosh\left(\sqrt{c}\,d_{\mathbb H}(p,b_\epsilon)\right)- \cosh\left(\sqrt{c}\,d_{\mathbb H}(a_\epsilon,b_\epsilon)\right) }{ \sinh\left(\sqrt{c}\,d_{\mathbb H}(p,a_\epsilon)\right) \sinh\left(\sqrt{c}\,d_{\mathbb H}(p,b_\epsilon)\right) } \right).$$ The corresponding angle between the RKHS feature-space displacement vectors is computed from the kernel-induced distances using the polarization identity: $$\theta_{\mathcal H_k}= \arccos \left( \frac{ d_k^2(p,a_\epsilon) +d_k^2(p,b_\epsilon) -d_k^2(a_\epsilon,b_\epsilon) }{ 2d_k(p,a_\epsilon)d_k(p,b_\epsilon) } \right),$$ where $d_k(x,y)=\sqrt{k(x,x)+k(y,y)-2k(x,y)}$. We report the absolute angular error in degrees. At $\epsilon=0.005$, the results are: | Kernel | Mean error (°) | Std (°) | Median error (°) | Max error (°) | |---|---:|---:|---:|---:| | RIBin | 0.003618 | 0.007003 | 0.000921 | 0.035390 | | RIInt | 0.005543 | 0.010988 | 0.001387 | 0.055865 | | RIExp | 0.005543 | 0.010989 | 0.001387 | 0.055870 | | RILog | 0.001707 | 0.003039 | 0.000438 | 0.014861 | The angular errors further decrease as $\epsilon$ becomes smaller. In particular, halving $\epsilon$ from 0.01 to 0.005 reduces the mean angular errors by approximately a factor of four, consistent with the finite-segment approximation effect. The original stress experiment evaluates the geometric consistency of learned representations after training, whereas the new curve-length and angle experiments directly evaluate geometric quantities induced by the canonical feature maps, independently of neural network training. The theoretical equality of the kernel-induced and original Riemannian metrics guarantees exact local metric preservation; the additional numerical experiments provide complementary empirical verification of its geometric consequences. We will revise Section 5.1 to clarify the distinct purposes of these evaluations and avoid interpreting representation-level stress as direct evidence of Riemannian isometry.W3 & Q2. Ablation Study on the Riemannian Isometry Condition
Response:
We thank the reviewer for this important question. We agree that it is necessary to determine whether the downstream improvements arise from the Riemannian isometry condition or merely from the distance transformations used by our kernels.
To address this, we conduct a controlled ablation study that removes the Riemannian isometry condition while keeping the distance-transformation functions unchanged. Specifically, for each proposed RI kernel, we construct a Euclidean counterpart (EN) by replacing the hyperbolic geodesic distance with the Euclidean distance. This allows us to isolate the contribution of the underlying Riemannian geometry.
Let . We define
\cosh\left(\sqrt{c}\|x-y\|_2\right).$$ The four EN counterparts are defined as follows: **ENBin:** \$k_{\mathrm{EN}}^{\mathrm{Bin}}(x,y) =\frac{1}{ct}r_{\mathrm{EN}}(x,y)^{-t}, \ t>0.\$ **ENExp:** \$k_{\mathrm{EN}}^{\mathrm{Exp}}(x,y) =\frac{1}{c\lambda} \exp\left[ -\lambda\left(1-r_{\mathrm{EN}}(x,y)^{-1}\right) \right], \ \lambda>0.\$ **ENInt:** \$k_{\mathrm{EN}}^{\mathrm{Int}}(x,y) =\frac{1}{c\left(1+\log r_{\mathrm{EN}}(x,y)\right)}. \$ **ENLog:** \$ \begin{aligned} k_{\mathrm{EN}}^{\mathrm{Log}}(x,y) =\frac{1}{c}\big[-\log r_{\mathrm{EN}}(x,y)+\log r_{\mathrm{EN}}(x_0,x) +\log r_{\mathrm{EN}}(x_0,y) \big], \end{aligned} \$ where $x_0$ is the reference point used in the corresponding RI construction. The RI and EN counterparts use the same scalar distance transformations. Specifically, define $$r_{\mathrm{RI}}(x,y)= \cosh\left(\sqrt{c}\,d_{\mathbb H}(x,y)\right).$$ Their squared kernel-induced distances can be written as $$d_{\mathrm{RI}}^2(x,y)= f(d_{\mathbb H}(x,y)), \qquad d_{\mathrm{EN}}^2(x,y)= f(\|x-y\|_2).$$ The corresponding scalar functions are: **Binomial:** \$f_{\mathrm{Bin}}(s) =\frac{2}{ct} \left[ 1-\cosh^{-t}(\sqrt{c}s) \right].\$ **Exponential:** \$f_{\mathrm{Exp}}(s) =\frac{2}{c\lambda} \left[ 1-\exp\left( -\lambda\left(1-\operatorname{sech}(\sqrt{c}s)\right) \right) \right].\$ **Integral:** \$f_{\mathrm{Int}}(s) =\frac{2}{c} \frac{\log\cosh(\sqrt{c}s)} {1+\log\cosh(\sqrt{c}s)}. \$ **Logarithmic:** \$f_{\mathrm{Log}}(s) =\frac{2}{c}\log\cosh(\sqrt{c}s). \$ Thus, each RI–EN pair uses exactly the same scalar function $f$ and corresponding parameters; only the input distance differs. We use $t=0.001$ for the binomial construction and $\lambda=0.001$ for the exponential construction, following our original experimental settings. All four EN constructions induce the same local tensor: $$g_{\mathrm{EN}}(x)=I_n.$$ In contrast, the Poincaré ball metric is $$g_{\mathbb H}(x)= \frac{4}{(1-c\|x\|_2^2)^2}I_n.$$ Consequently, $$g_{\mathrm{EN}}(x)\neq g_{\mathbb H}(x), \qquad \forall x\in\mathbb{B}_c^n.$$ The EN counterparts therefore do not satisfy the Riemannian isometry condition. They provide non-isometric counterparts that retain the same scalar distance transformations while removing geometric preservation. We keep all other hyperparameters and experimental configurations identical between RI and EN, including the backbone, optimization settings, training protocol, and evaluation procedure. The curvature parameters are also matched to the corresponding RI experiments: $c=0.001$ for RIBin, RIExp, and RIInt, and $c=0.01$ for RILog. Results are reported as mean accuracy with 95% confidence intervals, and the performance gap is calculated as RI minus EN. **Table 1: Classification accuracy (%) on CUB (5-way 1-shot) with 95% confidence interval.** | Method | Bin | Exp | Int | Log | |:--|--:|--:|--:|--:| | RI (isometric) | 67.3 ± 0.2 | 67.3 ± 0.2 | 65.8 ± 0.2 | 68.6 ± 0.2 | | EN (non-isometric) | 61.8 ± 0.2 | 62.8 ± 0.2 | 63.2 ± 0.2 | 63.5 ± 0.3 | | RI − EN | **+5.5** | **+4.5** | **+2.6** | **+5.1** | **Table 2: Classification accuracy (%) on miniImageNet (5-way 1-shot) with 95% confidence interval.** | Method | Bin | Exp | Int | Log | |:--|--:|--:|--:|--:| | RI (isometric) | 55.2 ± 0.2 | 54.3 ± 0.2 | 54.5 ± 0.2 | 56.3 ± 0.2 | | EN (non-isometric) | 48.9 ± 0.2 | 49.3 ± 0.2 | 49.0 ± 0.2 | 47.9 ± 0.2 | | RI − EN | **+6.3** | **+5.0** | **+5.5** | **+8.4** | The results show that removing the Riemannian isometry condition consistently degrades downstream performance across all four kernel constructions on both datasets. Since the scalar distance transformations and all other experimental configurations remain unchanged, these differences cannot be attributed simply to different distance-transformation functions. Instead, they demonstrate the performance benefit of preserving the underlying hyperbolic geometry through the Riemannian isometry condition. ## Q3. Difference Between RILog and RIBin We thank the reviewer for this comment. As the same concern was raised by Reviewer p2Zq, we refer the reviewer to our response to Q2 in A2 for detailed analysis and additional experimental results.
We sincerely thank the reviewer for the thoughtful and detailed feedback. We particularly appreciate the reviewer’s recognition of the mathematical motivation and theoretical contributions of our work. The comments have helped us identify several aspects that require clearer presentation and stronger empirical validation.
We try our best to address each concern below.
W1 & W4. Formatting Issues and Mathematical Notation
Response: We thank the reviewer for pointing out the formatting and notation issues. We agree that both the presentation of the tables and the consistency of the mathematical notation should be improved.
Regarding the formatting, we will revise Tables 1–5 to comply with the official ICLR formatting requirements by restoring the default font settings of the official style file. For tables that do not fit within the available space, we will restructure the columns, wrap long entries, or split the tables into smaller ones rather than reducing the font size.
Regarding the mathematical notation, we will revise the relevant equations and definitions to improve clarity and consistency. Specifically, we will:
Clarify the distinction between manifold points and their local coordinate representations in Equations (2)–(3). In particular, xi and yj denote the i-th and j-th coordinate components of the local chart representations of x and y, respectively; the partial derivatives are indicated explicitly by ∂.
Explicitly define the zeroth-order kernel term in Theorem 2 as the constant kernel k0(x,y)≡1.
State the valid parameter range for RIExp explicitly, including λ>0.
Correct and standardize the distance notation in Theorem 3, clearly distinguishing distances on the original manifold, the embedded submanifold, and the RKHS.
We will also review the appendix to ensure consistent notation throughout the manuscript.
W2 & Q1. Direct Validation of Riemannian Geometry Preservation
Response: We thank the reviewer for this insightful comment. We agree that our original geometric stress experiment does not directly measure the geometric distortion introduced by the canonical feature map. We clarify its motivation below and introduce additional experiments to directly evaluate geometric preservation.
1. Clarification of the motivation and scope of the original stress experiment.
The original stress experiment was designed to investigate whether different similarity measures lead to learned representations whose pairwise distance structures remain consistent with the intrinsic hyperbolic geometry. The motivation is to assess the extent to which the geometry of the learned representations deviates from the reference hyperbolic representation after neural network training and optimization. In this sense, the experiment provides a representation-level evaluation of geometric consistency across different similarity measures.
However, we acknowledge that the reviewer is correct: since the embeddings are obtained from separately trained models, the stress values reflect both the effects of the similarity measure and the representation learning process. Therefore, this experiment does not isolate the distortion introduced by the canonical feature map itself, and should not be interpreted as a direct validation of the Riemannian isometry property. We will clarify this distinction in the revised manuscript.
2. Why we do not numerically compare the Riemannian metric tensors.
The local metric preservation property is already established analytically in our paper. Specifically, for a smooth kernel k, the Riemannian metric induced by its canonical feature map is
\left. \frac{\partial^2 k(x,y)} {\partial x^i\partial y^j} \right|_{y=x}.$$ For the Poincaré ball, the original Riemannian metric is $$h_{ij}(x)= \frac{4\delta_{ij}} {(1-c\|x\|^2)^2}.$$ Our theoretical results establish that the proposed kernels satisfy $$g^k_{ij}(x)=h_{ij}(x)$$ at every point in the domain. Together with the required smoothness and embedding conditions, Theorem 1 establishes that the canonical feature map is a Riemannian isometry. Consequently, the equality of the two metric tensors is an exact theoretical result, rather than a numerical property that needs to be established through sampling. Numerically evaluating the metric tensors at sampled points would only provide a finite-precision verification of an equality already guaranteed by the theorem. Instead, we provide additional numerical experiments on curve lengths and angles, which offer complementary and more intuitive evaluations of the geometric consequences of this isometry.