acceptodds
Under review as a conference paper at ICLR 2027

Klein Neural Networks++

Abstract

Hyperbolic neural networks (HNNs) have shown significant advantages in representing hierarchical and tree-structured data through their exponential volume growth. However, existing methods rely almost exclusively on the Poincare ball and Lorentz hyperboloid, while the Klein model, another classical representation, has long been unexplored. The Klein model has Euclidean line-segment geodesics and a closed-form Einstein midpoint, where the latter avoids the iterative Frechet mean required for batch normalization. In this work, we systematically explore its modeling potential and build a Klein neural network framework of closed-form operators, namely Klein MLR, Klein Busemann MLR, Klein fully connected layer, Klein Busemann fully connected layer, and Klein GyroBN. Klein demonstrates strong training stability under feature clipping, curvature mis-specification, and gradient scaling, where the Poincare and Lorentz references diverge. The closed-form Einstein midpoint in Klein GyroBN offers a favorable trade-off between computational efficiency and performance against iterative Frechet means. Across numerical stability, image classification, graph link prediction, graph node classification, and EEG decoding, the proposed Klein layers match or improve upon their Poincare, Lorentz, and PV counterparts. Klein MLR heads further achieve the fastest runtime among hyperbolic counterparts.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
Hyperbolic neural networks, Klein Model, Riemannian deep learning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_fKfmOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper assembles Klein-coordinate versions of MLR/BMLR classifiers, FC/BFC layers, and GyroBN into a hyperbolic neural-network toolkit. It derives closed-form expressions and evaluates predictive performance, numerical behavior, and runtime against other hyperbolic implementations. The central claim is that the Klein realization offers computational and numerical advantages while maintaining competitive accuracy.

Critical strengths
  1. The manuscript is carefully organized, with clear figures, comparison tables, and extensive supplementary explanations.
  2. The evaluation reports multiple seeds, significance tests, and runtime measurements, providing useful detail beyond headline accuracy.
Critical weaknesses
  1. Limited substantive contribution and insufficient theoretical support

W1 — Methodological novelty. Sections 3.1–3.3 largely instantiate existing signed-distance, Busemann, and FC constructions in Klein coordinates. The authors explicitly position the contribution at the framework level, but assembling these counterparts does not establish a substantive new learning mechanism or a compelling methodological advance.

W2 — Theory does not establish the claimed advantages. The derivations establish closed forms and geometric correspondences, rather than comparative stability or optimization guarantees. Proposition E.2 provides only special-case equivalences and a local midpoint approximation; it does not explain the claimed training advantages, and its applicability to the actual GyroBN experiments is not established.

2. The experiments do not convincingly isolate the promised advantages

W3 — Stability diagnostics do not compare the same geometric objective. Table 4 differentiates the squared Euclidean norm of each model's coordinates, not a common intrinsic loss; gradient differences therefore need not indicate numerical instability. Moreover, Table 15 reports only 0.99% Klein accuracy without clipping, versus 0.5% chance: avoiding NaNs is not equivalent to effective learning.

W4 — Predictive gains are limited and confounded. CIFAR improvements over the strongest hyperbolic baseline are not significant (Table 27). Some graph gains are significant, but Section D.3.2 additionally gives Klein/PV GyroBN and varies activation placement/type, preventing clean attribution to the geometry and contradicting the boundary-type-only interpretation of Table 25.

W5 — Efficiency claims exceed the evidence. Table 26 shows only approximately 1.1–1.2x speedup over BMLR-L, the fastest competing head, without establishing an end-to-end training benefit. Section E.2.1 gives the full Klein BFC minus BFC-L cost as −m+2n+3-m + 2n + 3, so the broadly stated lowest-FLOP claim is conditional, not general.

Additional technical errors and corrections:

  • Equation (27), p. 20: Under the stated forward map, the inverse must be xs/(−K∗xt)x_s / (\sqrt{-K} * x_t), not xs/xtx_s / x_t; the printed pair is consistent only when K=−1K = -1.
  • Equation (30), p. 20: For K=−1K = -1, p=(1/2,0)p = (1/2, 0), and v=(1,0)v = (1, 0), the formula returns (−5/4,0)(-5/4, 0), whose squared Klein norm at pp is 25/925/9 rather than 11. It therefore fails to preserve the tangent norm.
  • Running-mean update above Algorithm 1, p. 38: For K=−1K = -1 and identical one-dimensional means mur=mub=1/2mu_r = mu_b = 1/2, the printed update gives 8/178/17, not 1/21/2. It cannot be the stated binary Einstein midpoint.
  • Section E.1, p. 33: The zero-score horosphere should satisfy B=b/αB = b/\alpha, not B=−b/αB = -b/\alpha, given the score definition u=−α∗B+bu = -\alpha * B + b.
Questions for authors
  1. Regarding W1: What is the principal methodological result beyond transporting existing constructions into Klein coordinates? Why is the resulting framework more than an incremental combination of known operators?
  2. Regarding W2: Which result directly supports a comparative stability or optimization advantage? Were Equations (27), (30), and the running-mean update implemented as printed, and would their correction affect the reported results?
  3. Regarding W3: Do the stability differences persist with matched intrinsic radii, a common geometric loss, and controlled projection policies? What evidence separates effective learning from merely returning finite values?
  4. Regarding W4: Do the graph gains persist with matched normalization, activation type/placement, and tuning budgets? How is Table 25 a boundary-type-only comparison given the differences documented in Section D.3.2?
  5. Regarding W5: What efficiency advantage remains under consistent full-cost accounting and matched, optimized baseline implementations? Is there a corresponding end-to-end training-time improvement?
14 comments

We thank Reviewer fKfm\color{green}{\text{fKfm}} for the valuable comments and for checking the appendix equations line by line. We respond to the concerns as follows.

Table, equation and section numbers refer to the submitted PDF, items that exist only in the revised PDF are marked "of the revision", and "answer N" refers to the numbered answers of this response. Answers 1–5 each address one weakness together with the question that refers to it, and answers 6–9 the four corrections.

1. Methodological contribution and the principal result beyond the transport (W1, Q1)

We thank the reviewer for this question. The signed-distance, Busemann and FC constructions of Sections 3.1–3.3 come from prior work, which the paper cites as templates. The methodological result of KNN++ is that these constructions become layers of the Klein model, where none of them had a closed form, and that all of them together form one network. Reviewers w1vu\color{blue}{\text{w1vu}} and ZRE9\color{red}{\text{ZRE9}} raised the same question from two sides. Our response to Reviewer w1vu pairs each prior construction with its Klein closed form, and our response to Reviewer ZRE9 compares KNN++ with the Klein network of Mao et al. [f]. Below we state the result in four parts, the closed forms, the network they allow, the efficiency of that network and why it is more than a combination.

1. Closed forms in the Klein model. A construction in hyperbolic space is defined through distances, and it becomes a layer only once its closed form in some coordinate system is derived. For the Klein model the closed forms of the point-to-hyperplane distance, the MLR, the Busemann function and the FC layers did not exist, and Thms. 3.1–3.4 derive them. Klein geodesic hyperplanes are Euclidean hyperplanes cut by the ball, so the decision boundaries of the Klein MLR and FC layers are linear equations in the coordinates, $ \langle x, \hat z _{k}\rangle = \tanh(\sqrt{-K}, r _{k})/\sqrt{-K}$ (Thm. 3.2), and their direct implementation is numerically stable. The network of Mao et al. [f] builds each layer in the tangent space at the origin instead, so its decision boundaries are not geodesic hyperplanes.

2. A network built only from Klein layers. After submission we built Klein-ResNet, a ResNet-18 in which convolution, batch normalization, residual addition and classifier are all Klein layers, and trained it with the training setup of HCNN [a]. On CIFAR-100 it reaches the accuracy of HCNN with about a third of the GPU memory and a quarter of the time per epoch of the reference HCNN implementation. Against an HCNN that we optimized by hand the speed is the same and the memory saving shrinks to about a fifth (answer 5).

Table A- Fully hyperbolic ResNet-18 on CIFAR-100, 5 seeds

Klein-ResNetHCNN [a]ILNN [b]Poincaré ResNet [c]
Top-1 (%)78.0978.0778.4176.60
GPU memory2.8 GB7.7 GB11.2 GB14.6 GB
Time per epoch15.6 s62.4 s56.5 s195.3 s

Note: Baseline accuracies as reported in their papers. Memory and time measured by us on one A100, using the released code of HCNN and Poincaré ResNet and our implementation of ILNN.

3. Why this network needs less memory and time. Most of the saving is in the convolution. A hyperbolic convolution applies an FC layer to the $ 3 \times 3$ neighbourhood of every pixel. The reference HCNN first copies each neighbourhood into a separate vector, so every input value is stored nine times. The Klein FC needs from a neighbourhood only its inner products with the weights and its squared norm, and both are ordinary convolutions of the input, one with the weights and one with an all-ones kernel (Prop. F.3 of the revision). Klein-ResNet therefore never builds the copies. Its convolution goes from slower than that of HCNN at 16 channels to 4.5 times faster at 512 channels, and its batch normalization is 1.7 times faster at 512 channels (Table G in answer 5).

References

[a] Ahmad Bdeir, Kristian Schwethelm, and Niels Landwehr. Fully Hyperbolic Convolutional Neural Networks for Computer Vision. ICLR, 2024.

[b] Xianglong Shi, Ziheng Chen, Yunhan Jiang, and Nicu Sebe. Intrinsic Lorentz Neural Network. arXiv:2602.23981, 2026.

[c] Max van Spengler, Erwin Berkhout, and Pascal Mettes. Poincaré ResNet. ICCV, 2023.

[f] Yidan Mao, Jing Gu, Marcus C. Werner, and Dongmian Zou. Klein Model for Hyperbolic Neural Networks. NeurIPS Workshop on Symmetry and Geometry in Neural Representations, 2024.

(Continued in 2/10.)

(Continuation of 1/10.)

4. Why the framework is more than a combination. The layers share these primitives rather than sitting side by side. The convolution reuses the FC closed form, the batch normalization takes its centre as the closed-form Einstein midpoint of Gulcehre et al. [e] in the same coordinates, and the classifier is again a Klein closed form, so the network never leaves Klein coordinates. Written as derived, the classifier also avoids the two FP32 steps at which the reference Poincaré and Lorentz heads fail (answer 2).

In short, the methodological result of KNN++ is the Klein closed forms of these constructions (Thms. 3.1–3.4), the convolution they allow as two standard convolutions (Prop. F.3 of the revision), and a network built only from Klein layers, which reaches the accuracy of HCNN with less memory than both its reference and its hand-optimized implementation. The revision states the contribution in these terms in §1.

2. Theoretical support and the result behind the comparison (W2, Q2)

The reviewer is right that our theorems give closed forms rather than comparative guarantees, and that Prop. E.2 is not a stability result. What we can add is a short floating-point account that predicts where each classifier fails, which matches the failures measured in the stability sweeps, and a precise statement of what Prop. E.2 covers. The revision adds the first to §4.1 and the second to App. E.

1. Predicting where each classifier fails. Reviewers ZRE9\color{red}{\text{ZRE9}} and w1vu\color{blue}{\text{w1vu}} asked for the same mechanism. Every non-finite run fails in one step of the baseline head, and the limits of FP32 predict where (Table B).

Table B- Where each classifier fails in FP32 (stability sweeps of Tabs. 2–3)

HeadFailing stepPredictedObserved
Lorentz MLR [a]subtracts two terms of size $ e ^{2a}/4$$ a \approx 8.3$–9.09.0$ a = 8.4$–9.09.0
Poincaré MLR [d]squares a term of size $ e ^{x}/2$$ x \approx 45$$ x = 40$–4545 (5 of 6)
Klein MLRevaluates $ e ^{\rho}$ once$ \rho \approx 87$ ($ 72$ on the boundary)no failure in 30 runs

FP32 stores numbers up to about $ e ^{88.7}$ with 24 bits of precision. Every head contains a term that grows exponentially with the offset of its hyperplane from the origin, written $ a$, $ x$ and $ \rho$ in Table B and measured as geodesic distances in units of $ 1/\sqrt{-K}$. The Klein logit evaluates this term once, without squaring it or dividing by a difference of such terms. The Poincaré head squares it, which halves the usable range. The Lorentz head subtracts two squared terms whose difference is small and then divides by that difference, which rounds to zero once the terms exceed $ 2 ^{24}$ times its size. We located the failing steps by recomputing every non-finite run term by term.

The predicted and observed failure points agree, and the sixth Poincaré failure overflowed in the unsquared term at $ x = 99$, also past the FP32 limit. For Lorentz, rewriting the failing step or evaluating the head in FP64 removes the failures (0 of 3 runs at $ K = -10$), so its row of Table B is a statement about how one formula is evaluated. The Poincaré head still fails in FP64, because training drives its offset $ x$ from $ 0$ to $ 41$ within ten steps until the logits overflow. Swapping the feature radius and initialization between the two heads at $ K = -0.3$ (3 seeds, 400 steps) shows that the Poincaré head is the more sensitive one. With the Klein settings it trains, and the Klein head with the Poincaré settings stays finite with logits below $ 30$. These are measurements rather than guarantees, and §4.1 states them at that scope. When FP32 rounds a point onto the boundary sphere, $ \gamma _{x}$ is clamped at $ 3.2 \times 10 ^{7}$ and the Klein limit drops to $ \rho \approx 72$, which is where the one Klein failure outside these sweeps occurred, at $ K = -1$ and $ r = 10$ with $ \rho = 73$–107107.

References

[a] Ahmad Bdeir, Kristian Schwethelm, and Niels Landwehr. Fully Hyperbolic Convolutional Neural Networks for Computer Vision. ICLR, 2024.

[d] Ryohei Shimizu, Yusuke Mukuta, and Tatsuya Harada. Hyperbolic Neural Networks++. ICLR, 2021.

[e] Caglar Gulcehre, Misha Denil, Mateusz Malinowski, Ali Razavi, Razvan Pascanu, Karl Moritz Hermann, Peter Battaglia, Victor Bapst, David Raposo, Adam Santoro, and Nando de Freitas. Hyperbolic Attention Networks. ICLR, 2019.

(Continued in 3/10.)

(Continuation of 2/10.)

2. What Prop. E.2 covers. Prop. E.2 justifies using the Einstein midpoint as the batch centre in place of the Fréchet mean, which needs an iterative solver. It shows that the two coincide for two points with equal weights and agree to second order when all points lie close to the origin. The running-mean update weights its two points by $ 1-\eta$ and $ \eta$, so part (iii) does not cover it exactly. The weighted midpoint still lies on the geodesic between the two points, which is what a running average needs, but not at the same position as the weighted Fréchet mean. For the batch centre the applicability is tested rather than proved. Swapping the midpoint for the converged Fréchet mean inside the same layer gives $ 80.57$ against $ 80.77$ on Disease and $ 72.50$ against $ 73.08$ on PubMed (Tab. 7), and on Airport the Fréchet mean reaches only $ 85.67 \pm 7.99$ while the midpoint stays above $ 97$.

3. The result that directly supports the comparison. It is the localization of point 1, which places every baseline failure in one step of the head, together with the swap of point 1, which shows that the Klein head trains under the radius and initialization at which the Poincaré head fails. The Klein head had no non-finite value in any of its 30 runs. In accuracy we claim no advantage where all heads train, as our response to Reviewer ZRE9\color{red}{\text{ZRE9}} explains.

4. The printed equations. None of the three equations is implemented as printed, and correcting them changes no reported number. The revision adds the localization to §4.1 and corrects the three equations in App. C and App. E.

Table I- The three printed equations and the code

Printed equationIn the code
Eq. (27), Lorentz to Kleinused only in the centre swap of our response to Reviewer ZRE9, as the corrected inverse $ x _{s}/(\sqrt{-K},x _{t})$
Eq. (30), parallel transportnot used, since no layer calls a parallel transport and Thm. 3.2 uses only that it is a linear isometry
Running-mean update above Alg. 1used in its correct form, the weighted two-point Einstein midpoint below

The code updates the running mean as $ \mu _{r} \leftarrow \frac{(1-\eta)\gamma _{\mu _{r}}\mu _{r} + \eta\gamma _{\mu _{b}}\mu _{b}}{(1-\eta)\gamma _{\mu _{r}} + \eta\gamma _{\mu _{b}}}$.

In short, the theorems give closed forms and no comparative guarantee. The comparative evidence is the floating-point account of Table B and, against Poincaré, its sensitivity to the feature radius and initialization, and Prop. E.2 covers the batch centre only approximately, with its use in GyroBN supported by Tab. 7. None of the three printed equations is implemented as printed, so their correction changes no reported number.

3. Stability diagnostics under matched conditions and effective learning (W3, Q3)

The reviewer is right on both observations. Tab. 4 differentiates the squared norm of each model's coordinates, which is a different function on each chart, so its gradients do not measure instability. Tab. 4 was a supplementary probe, and the sweeps of Tabs. 2–3 and their numbers do not depend on it. The revision replaces it in §4.1 with the localization of the failures on the training loss. On Tiny-ImageNet, the near-chance Klein entries of Tab. 15 come from runs that used only part of the training set, and a five-seed rerun on the full set learns at every radius.

1. A common objective in place of Tab. 4. Following the reviewer's suggestion, the replacement works on the training loss, which all three models share. Table B in answer 2 locates every failure of the sweeps in one step of the baseline head, a cancellation in the Lorentz MLR and an overflow in the Poincaré MLR, and predicts from the limits of FP32 where each occurs. This also corrects the conclusion that the submission drew from Tab. 4. The Lorentz failures occur in the classifier rather than in the coordinate map, and the revision removes that sentence from §4.1. The Klein head had no non-finite value in any of its 30 runs in the sweeps of Tabs. 2–3.

2. Tiny-ImageNet at large clipping radii. Reviewer ZRE9\color{red}{\text{ZRE9}} raised the same point on Tab. 15, and Tables C1 and C2 are the same tables as in our response to them. In checking Tab. 15 we found that its Klein entries at $ r = 5$, $ 10$ and $ 10 ^{9}$ come from runs that iterated over only 31, 47 and 116 of the 782 training batches per epoch, and reached $ 99.9%$ training accuracy on that subset. Rerun on the full training set with five seeds under the protocol of Tab. 15, Klein learns at every one of these radii. Without clipping one of the five seeds becomes non-finite, and projecting the features back into the ball removes it, with $ 63.74 \pm 0.23$ over five finite seeds.

(Continued in 4/10.)

Reviewer_w1vuOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces Klein Neural Networks++, a framework for building hyperbolic neural networks using operations in the Klein model of hyperbolic space. The authors introduce closed-form operators, namely Klein and Busemann MLRs, Klein and Busemann fully connected layers, and Klein GyroBN. Experiments cover numerical stability, image classification, graph link prediction, node classification, and ablations on MLR runtime and GyroBN gyromean.

Critical strengths
  1. The paper has a comprehensive survey of operations in hyperbolic space and summarize them clearly in Table 1. This will be very useful for practical users of hyperbolic neural networks.
  2. The paper also presents comprehensive experiments, including numerical stability, image classification, graph link prediction, node classification, and ablations on MLR runtime and GyroBN gyromean.
  3. The paper is generally well-written. The figures are helpful. The experimental details are reported for reproducibility.
Critical weaknesses
  1. The paper has limited novelty. Klein models have been used in previous papers. Klein MLR follows exactly the same construction as Poincare MLR (Ganea et al., 2018). The use of Busemann function comes from (Chen et al., 2026a). The idea for batch normalization is from (Chen et al., 2025b). Also importantly, Gulcehre et al., (2019) have used the Einstein midpoint in Klein model as a closed-form calculation, which is not correctly mentioned in the paper.
  2. The paper overly claims that KNN++ is an “intrinsic” framework. However, “intrinsic” would mean that the framework only relies on the geometry of hyperbolic space, but not a specific model, and in that sense, there should be no advantage of using Klein model than using Poincare or hyperboloid (PV) model. Indeed, from the explicit use of operations such as Einstein addition and Einstein midpoint, I believe KNN++ is NOT an intrinsic framework, but a framework that depends on extrinsic operations. (I agree that the use of distance, etc., is intrinsic, but it does not mean that with the explicit formulas, the framework itself is intrinsic).
  3. If the framework is motivated by “intrinsic” geometry, then there should be no difference in performance no matter which hyperbolic model is adopted. Indeed, in image classification, the performances from different frameworks are very similar in Table 5, for all three image datasets.
  4. Given the above, it makes it more important to understand why the Klein model enjoys better numerical stability. If the authors can provide more insight, in addition to the seemingly encouraging numerical results, it will be more convincing.
  5. (Minor) KNN++ does not seem to be a good name. KNN (K nearest neighbor) is a very classical method, so using the same name may be confusing.
  6. (Minor) Some tables are squeezed into the text, which makes the paper hard to read. Examples are Tables 5, 6, 7. Please apply some margin around the tables.
  7. The code only includes image classification, but not numerical stability or graph link prediction/node classification.
Questions for authors

See weaknesses.

8 comments

We thank Reviewer w1vu\color{blue}{\text{w1vu}} for the valuable comments and for finding the survey of hyperbolic operations in Table 1 useful to practitioners. We respond to the concerns as follows.

Table, equation and section numbers refer to the submitted PDF, items that exist only in the revised PDF are marked "of the revision", and "answer N" refers to the numbered answers of this response.

1. Relation to prior work on hyperbolic layers and the Klein model

We thank the reviewer for pointing out that Gulcehre et al. [a, Sec. 4.2] already use the Einstein midpoint in the Klein model as a closed-form computation. The submission cites their work in §1, but only as an auxiliary use. The revision credits this use to Gulcehre et al. [a] in §1, §3.4 and App. C, and the abstract states that hyperbolic attention already uses the midpoint. We also agree that KNN++ builds on prior results. The signed-distance MLR [b, c], the Busemann classifier [d] and GyroBN [e] are core constructions of hyperbolic networks, and the aim of KNN++ is to bring them to the Klein model, where none of them had a closed form before. The Klein network of Mao et al. [f] is built differently, through the tangent space at the origin, and the revision describes it accordingly.

1. The prior constructions that KNN++ brings to the Klein model. Table A pairs each construction with its Klein counterpart.

Table A- Prior construction and Klein counterpart of each layer

LayerPrior constructionKlein closed form
MLRsigned distance to a hyperplane [b, c]Thms. 3.1–3.2
BMLRBusemann logit [d]Thm. 3.3
FC, BFCFC as signed distances [c, d]Thm. 3.4
GyroBNFréchet-mean centre [e]Einstein midpoint [a] (§3.4)

2. The Klein network of Mao et al. Mao et al. [f] define the weight transformation and the activation of a Klein layer as Einstein versions of Euclidean maps, $ \operatorname{Exp} _{\mathbf{0}} \circ f \circ \operatorname{Log} _{\mathbf{0}}$ [f, Def. 8 and Thm. 9], and their network comprises one such linear layer followed by a Euclidean classification layer [f, Sec. 5]. In contrast, every classifier and FC layer of KNN++ is a closed form of the signed Klein distance to a geodesic hyperplane or of the Klein Busemann function (Thms. 3.2–3.4), and KNN++ adds a batch normalization that their network does not have. Reviewer ZRE9\color{red}{\text{ZRE9}} raised the same distinction and noted that Tab. 21 compares the two networks in accuracy alone. We therefore reran their network with the released code of Chen et al. [i], and Table B, the same table as in our response to Reviewer ZRE9, compares the training time per epoch of both networks. KNN++ takes 0.80 to 0.83 of the training time of Klein HNN.

Table B- Time per epoch of Klein HNN [f] and KNN++ trained back to back in the pipeline of Tab. 21, green for the shorter.

DatasetKlein HNN [f] time (ms)KNN++ time (ms)Time ratio
Disease50.45 ± 0.6441.90±0.90\color{green}{\mathbf{41.90 \pm 0.90}}0.83
Airport129.89 ± 2.74107.70±2.42\color{green}{\mathbf{107.70 \pm 2.42}}0.83
PubMed6.73 ± 0.185.40±0.18\color{green}{\mathbf{5.40 \pm 0.18}}0.80
Cora14.10 ± 0.3811.61±0.51\color{green}{\mathbf{11.61 \pm 0.51}}0.82

The row labeled KNN in Tab. 21 lists the values reported by Chen et al. [i] rather than a reproduction. The revision states this in the caption of Tab. 21 and relabels the row Klein HNN, the name Mao et al. [f] use.

References

[a] Caglar Gulcehre, Misha Denil, Mateusz Malinowski, Ali Razavi, Razvan Pascanu, Karl Moritz Hermann, Peter Battaglia, Victor Bapst, David Raposo, Adam Santoro, and Nando de Freitas. Hyperbolic Attention Networks. ICLR, 2019.

[b] Octavian-Eugen Ganea, Gary Bécigneul, and Thomas Hofmann. Hyperbolic Neural Networks. NeurIPS, 2018.

[c] Ryohei Shimizu, Yusuke Mukuta, and Tatsuya Harada. Hyperbolic Neural Networks++. ICLR, 2021.

[d] Ziheng Chen, Bernhard Schölkopf, and Nicu Sebe. Hyperbolic Busemann Neural Networks. arXiv:2602.18858, 2026.

[e] Ziheng Chen, Xiao-Jun Wu, Bernhard Schölkopf, and Nicu Sebe. Riemannian Batch Normalization: A Gyro Approach. arXiv:2509.07115, 2025.

[f] Yidan Mao, Jing Gu, Marcus C. Werner, and Dongmian Zou. Klein Model for Hyperbolic Neural Networks. NeurIPS Workshop on Symmetry and Geometry in Neural Representations, 2024.

[i] Ziheng Chen, Zihan Su, Bernhard Schölkopf, and Nicu Sebe. Proper Velocity Neural Networks. ICLR, 2026.

(Continued in 2/6.)

(Continuation of 1/6.)

3. Closed-form centres in the other models. Reviewer ZRE9\color{red}{\text{ZRE9}} raised a related point on GyroBN, and we agree with it. The Lorentz BN of Bdeir et al. [h] and the midpoint BN of Poincaré ResNet [j] also compute their centre in closed form, and all three centres are the same point under the isometries, so closed-form aggregation is not specific to the Klein model. The difference lies in the rest of the layer. Table C, the same table as in our response to Reviewer ZRE9, times one BN layer of each network in its own implementation on the same A100, with ILNN in its official code. Klein GyroBN is the fastest at every width from 16 to 512 channels. In memory it is the lowest at 16 and 64 channels, and at 256 and 512 channels it uses slightly more than the official ILNN layer.

Table C- One BN layer with 512 channels in training mode, forward and backward under torch.compile on one A100. Time is the median of 30 steps averaged over 5 jobs.

BN layerTime (ms)Memory (MB)
Poincaré ResNet [j]1531158
ILNN [g]4.35516
HCNN [h]3.45644
Klein GyroBN2.01578

Note: Data extracted from Tab. 27 of the revision, with ILNN in its official code and HCNN with its shift as a plain parameter, since the original ManifoldParameter breaks the compiled graph 16 times (7.47 ms).

In short, the constructions of KNN++ come from prior work (Table A), the Einstein midpoint from Gulcehre et al. [a], and a Klein network already exists [f]. What KNN++ contributes is their Klein closed forms, which did not exist before (Thms. 3.1–3.4 and §3.4), the network built only from Klein layers that these closed forms make possible (Klein-ResNet, §3.5 and §4.6 of the revision), and the cost and numerical operating range of the resulting layers (Tables B and C, answer 4). The revision states the contribution in these terms in §1.

2. Whether KNN++ is an intrinsic framework

The reviewer is right that the submission never states in which sense it uses the word, and that its formulas, Einstein addition and the Einstein midpoint among them, are specific to the Klein model. The revision adds the definition below, with its sources, at the start of §3. The layers of KNN++ are intrinsic in how they are defined and specific to the Klein model in how they are computed, and these two levels do not conflict.

1. Intrinsic definition, Klein-specific formulas. In the hyperbolic literature a layer is called intrinsic when its output is defined through quantities of the manifold itself, such as the geodesic distance to a hyperplane or the Busemann function, rather than through a Euclidean operation in a tangent space or in the ambient Minkowski space. Chen et al. [d, Sec. 1] contrast the Möbius and Lorentz FC layers, which "rely on flat-space (tangent or ambient) approximations", with the Poincaré FC layer of Shimizu et al. [c], which is "intrinsic but model-specific", and Shi et al. [g, Secs. 1 and 4.1] build a fully intrinsic network whose operations are all defined inside the Lorentz model. The word therefore describes how a layer is defined and allows formulas written in one model. Table D checks each component of KNN++ against this definition, including the two operations the reviewer names.

References

[a] Caglar Gulcehre, Misha Denil, Mateusz Malinowski, Ali Razavi, Razvan Pascanu, Karl Moritz Hermann, Peter Battaglia, Victor Bapst, David Raposo, Adam Santoro, and Nando de Freitas. Hyperbolic Attention Networks. ICLR, 2019.

[c] Ryohei Shimizu, Yusuke Mukuta, and Tatsuya Harada. Hyperbolic Neural Networks++. ICLR, 2021.

[d] Ziheng Chen, Bernhard Schölkopf, and Nicu Sebe. Hyperbolic Busemann Neural Networks. arXiv:2602.18858, 2026.

[f] Yidan Mao, Jing Gu, Marcus C. Werner, and Dongmian Zou. Klein Model for Hyperbolic Neural Networks. NeurIPS Workshop on Symmetry and Geometry in Neural Representations, 2024.

[g] Xianglong Shi, Ziheng Chen, Yunhan Jiang, and Nicu Sebe. Intrinsic Lorentz Neural Network. arXiv:2602.23981, 2026.

[h] Ahmad Bdeir, Kristian Schwethelm, and Niels Landwehr. Fully Hyperbolic Convolutional Neural Networks for Computer Vision. ICLR, 2024.

[j] Max van Spengler, Erwin Berkhout, and Pascal Mettes. Poincaré ResNet. ICCV, 2023.

(Continued in 3/6.)

(Continuation of 2/6.)

Table D- Components of KNN++ and what each is defined through

ComponentDefined throughSource
Klein MLR and FCsigned geodesic distance to a geodesic hyperplaneThms. 3.2 and 3.4
Klein BMLR and BFCBusemann functionThms. 3.3 and 3.4
Einstein additionleft gyrotranslation $ x \mapsto b \oplus _{E} x$, an isometry of the Klein model, in the role of Möbius addition in the Poincaré FC layer[e, Thm. 13 and Sec. 5]
Einstein midpoint in GyroBNclosed-form centre, the same point as the Lorentzian centroid under the isometries, with which GyroBN "remains fully intrinsic"[c, Thm. 1], [g, Sec. 4.2]

None of these components uses a tangent-space linearization or a linear map of the ambient space. Because the definitions are intrinsic, the Klein, Poincaré and Lorentz versions of a layer compute the same function, which is why their accuracies tie in Tab. 5. Because the formulas are written in one model, their cost and floating-point behavior differ, which is where the stability results of Tabs. 2–3 come from.

3. Comparison with the Poincaré and Lorentz realizations in accuracy

We agree with the reviewer. The three realizations are isometric, so well-tuned Klein, Poincaré and Lorentz networks can represent the same functions and should reach the same accuracy, and the opening of App. D reads the experiments in this way. The main text, however, presents accuracy as a gain, in §1 ("matches or improves on its Poincaré, Lorentz, and PV counterparts") and in §4.2 ("Klein MLR attains the highest accuracy"). The revision rewrites both sentences as parity in accuracy.

1. Significance tests behind the ties in Tab. 5. Tab. 27 of the submission compares the better Klein head on each CIFAR dataset with the Euclidean head and with the strongest hyperbolic head, BMLR-P, and Table E copies it.

Table E- CIFAR, 5 seeds per group, copied from Tab. 27 of the submission ($ \Delta$ is Klein minus the comparator, Holm correction over the four tests)

Comparison$ \Delta$95% CIHolm $ p$
Klein MLR − Euclidean, CIFAR-100+0.53[+0.32, +0.74]0.0014
Klein BMLR − Euclidean, CIFAR-10+0.19[+0.04, +0.34]0.065
Klein MLR − BMLR-P, CIFAR-100+0.15[−0.28, +0.58]0.82
Klein BMLR − BMLR-P, CIFAR-10+0.01[−0.16, +0.18]0.89

The two lower rows are the comparison the reviewer has in mind. Against the strongest head of the Poincaré model the differences are $ +0.15$ and $ +0.01$ points, with $ p = 0.82$ and $ 0.89$. The two upper rows change the geometry instead, and against the Euclidean head the same test finds a gain on CIFAR-100 ($ p = 0.0014$). This is the pattern that isometry predicts. Changing the geometry can change accuracy, while changing the model within one geometry does not. The realizations differ in cost and in how their formulas behave in floating point, which answer 4 addresses.

References

[c] Ryohei Shimizu, Yusuke Mukuta, and Tatsuya Harada. Hyperbolic Neural Networks++. ICLR, 2021.

[e] Ziheng Chen, Xiao-Jun Wu, Bernhard Schölkopf, and Nicu Sebe. Riemannian Batch Normalization: A Gyro Approach. arXiv:2509.07115, 2025.

[g] Xianglong Shi, Ziheng Chen, Yunhan Jiang, and Nicu Sebe. Intrinsic Lorentz Neural Network. arXiv:2602.23981, 2026.

(Continued in 4/6.)

Reviewer_ZRE9OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces Klein Neural Networks++ (KNN++), a framework comprising geodesic and Busemann classifiers, corresponding fully connected layers, and Klein batch normalization using the Einstein midpoint. It evaluates these components on image and graph tasks, reporting competitive predictive performance, faster classifier execution, and improved numerical robustness under selected settings. The work offers a useful implementation framework, but its incremental contribution and the attribution of its efficiency and stability gains require clearer positioning and more controlled comparisons.

Critical strengths
  1. The paper presents a unified toolkit of Klein-based neural network components, covering geodesic and Busemann classifiers, fully connected layers, and batch normalization. Explicit formulations and derivations make the framework useful for further research.

  2. The proposed classifiers use compact parameterizations and batch-efficient computations. Runtime measurements complement the theoretical complexity analysis and demonstrate efficiency gains over the evaluated hyperbolic heads.

  3. The evaluation covers image classification, link prediction, and node classification, alongside numerical stability and component ablations. Multiple seeds and statistical tests strengthen the main empirical comparisons.

Critical weaknesses
  1. The distinction from prior Klein networks needs clarification. Mao et al. (2024) already developed Klein linear layers and a neural network framework, rather than only an MLR layer as characterized in the introduction. Although this baseline appears in Table 21, direct comparisons addressing the claimed stability and efficiency advantages are missing.

  2. The batch normalization comparisons do not adequately establish a Klein-specific advantage. The Einstein midpoint is primarily compared against iterative Fréchet means, while existing Poincaré midpoint BN and Lorentz centroid BN also provide closed-form alternatives. Comparisons against these methods are necessary to distinguish the benefits of closed-form aggregation from those of Klein coordinates.

  3. The experiments do not fully isolate the contributions of the proposed components. Graph experiments differ in normalization and activation configurations, while image experiments primarily replace the final classifier. Despite the existing depth ablations, a representative end-to-end experiment integrating the proposed components under controlled settings would better establish the framework’s practical benefits.

Questions for authors
  1. The theoretical results primarily establish closed-form Klein formulations, rather than explain their advantages over existing hyperbolic models. Since Klein, Poincaré, and Lorentz are isometric representations of the same geometry, the claimed benefits must arise from parameterization, computational structure, or numerical implementation. Although the paper provides FLOP counts and empirical stability diagnostics, it does not sufficiently establish when and why Klein coordinates should offer better numerical conditioning or optimization behavior. It would be better if the authors could provide corresponding analysis.

  2. Could the authors clarify whether the claimed robustness under extreme settings reflects effective learning or merely the avoidance of non-finite values? Table 15 shows that Klein accuracy on Tiny-ImageNet decreases to 22.60%, 2.34%, and 0.99% at clipping radii of 5, 10, and 10910^9, respectively, against a chance level of 0.5%. Although these runs avoid NaNs, the latter two results indicate severely degraded predictive performance. Please explain whether this degradation is associated with boundary saturation or vanishing gradients, and provide training-loss and gradient diagnostics for these configurations.

9 comments

We thank Reviewer ZRE9\color{red}{\text{ZRE9}} for the constructive suggestions and the positive feedback. We respond to the concerns as follows.

Table, equation and section numbers refer to the submitted PDF, items that exist only in the revised PDF are marked "of the revision", and "answer N" refers to the numbered answers of this response.

1. Distinguishing KNN++ from the Klein network of Mao et al. [a]

The reviewer is right that Mao et al. [a] contribute a Klein linear layer and a network built from it rather than only an MLR layer, and that Tab. 21 compares the two networks in accuracy alone. The revision corrects our description of their work in §1 and App. D.3.3. Below we compare the two networks in construction, stability and efficiency.

1. Construction. Mao et al. [a] build each Klein layer in the tangent space at the origin, $ f ^{\otimes _{E}} = \operatorname{Exp} _{\mathbf{0}} \circ f \circ \operatorname{Log} _{\mathbf{0}}$ [a, Def. 8], and classify with a Euclidean MLR [a, Sec. 5]. KNN++ instead writes the Euclidean MLR and FC layer through signed distances to hyperplanes [c, b] and measures these distances with the Klein metric, which gives closed-form layers (Thms. 3.2–3.4), and adds a batch normalization that their network does not have. The two layers also differ geometrically. As Shimizu et al. [b, Sec. 3.2] observed for the Poincaré version, the zero set of a tangent-space layer is not a hyperplane, and in a two-dimensional example with $ b = (0.3, 0.2)$ the zero set of the layer of Mao et al. deviates from a straight line by up to $ 0.12$, whereas each output of the Klein FC layer vanishes exactly on a geodesic hyperplane. The construction also determines what can be built. KNN++ provides every layer that a hyperbolic ResNet needs, including a Klein convolution computed as two standard convolutions (Prop. F.3 of the revision), which makes the network of answer 3 possible, in which every layer is a Klein layer. The network of Mao et al. consists of one linear layer and a Euclidean classifier.

2. Stability. The stability claims of the submission concern the geodesic Poincaré and Lorentz classifiers and batch normalization with iterative Fréchet means, which the network of Mao et al. does not contain. To compare the two networks directly, we ran both through the curvature and radius sweeps of Tabs. 2–3 in the pipeline of Tab. 21, with $ K \in {-0.01, -0.1, -0.3, -1, -10}$ at input radius 1 and $ r \in {1, 2, 5, 10, 10 ^{9}}$ at $ K = -0.3$, every input row scaled to norm $ r$ (the input map of both networks caps it at 20), the learning rate and dropout selected for each network, and five seeds (Table A2). Neither network has a non-finite run in any of its 180 runs, and both keep learning at every setting. Each loses most in one place, KNN++ on Airport at $ K = -0.01$ and Klein HNN on Airport at $ K = -10$, and neither is consistently more accurate.

Table A2- Test accuracy (%) at $ K = -0.3$, $ r = 1$, with the lowest over the 9 settings of the sweeps in parentheses, 5 seeds. Non-finite runs are counted over all 180 runs of each network.

Non-finite runsDiseaseAirportPubMedCora
KNN++0 / 18078.8 (78.6)94.0 (71.3)72.9 (72.4)57.1 (52.0)
Klein HNN [a]0 / 18079.9 (79.3)95.5 (87.5)73.3 (70.2)56.9 (55.7)

The two networks therefore do not differ in stability, and we make no stability claim against Mao et al.

3. Efficiency. We reran their network with the released code of Chen et al. [d] in the pipeline of Tab. 21 and trained both networks back to back on one node. KNN++ takes 0.80 to 0.83 of the training time per epoch of Klein HNN on all four datasets (Table A), and the revision adds this comparison to App. D.3.3.

Table A- Time per epoch of Klein HNN [a] and KNN++ trained back to back in the pipeline of Tab. 21, green for the shorter.

DatasetKlein HNN [a] time (ms)KNN++ time (ms)Time ratio
Disease50.45 ± 0.6441.90±0.90\color{green}{\mathbf{41.90 \pm 0.90}}0.83
Airport129.89 ± 2.74107.70±2.42\color{green}{\mathbf{107.70 \pm 2.42}}0.83
PubMed6.73 ± 0.185.40±0.18\color{green}{\mathbf{5.40 \pm 0.18}}0.80
Cora14.10 ± 0.3811.61±0.51\color{green}{\mathbf{11.61 \pm 0.51}}0.82

References

[a] Yidan Mao, Jing Gu, Marcus C. Werner, and Dongmian Zou. Klein Model for Hyperbolic Neural Networks. NeurIPS Workshop on Symmetry and Geometry in Neural Representations, 2024.

[b] Ryohei Shimizu, Yusuke Mukuta, and Tatsuya Harada. Hyperbolic Neural Networks++. ICLR, 2021.

[c] Octavian-Eugen Ganea, Gary Bécigneul, and Thomas Hofmann. Hyperbolic Neural Networks. NeurIPS, 2018.

[d] Ziheng Chen, Zihan Su, Bernhard Schölkopf, and Nicu Sebe. Proper Velocity Neural Networks. ICLR, 2026.

(Continued in 2/7.)

(Continuation of 1/7.)

The row labeled KNN in Tab. 21 lists the values reported by Chen et al. [d] rather than a reproduction, and the revision relabels it Klein HNN, the name Mao et al. [a] use.

In short, the two networks differ in how every layer is constructed and in what can be built from it, do not differ in stability under the sweeps of Tabs. 2–3, and KNN++ trains 17 to 20% faster per epoch.

2. Comparison with closed-form Poincaré and Lorentz batch normalization

We agree with the reviewer. The Lorentz BN of Bdeir et al. [e] and the midpoint BN of Poincaré ResNet [g] also compute their centre in closed form, and all three centres are the same point under the isometries, so closed-form aggregation is not specific to the Klein model. The abstract, §1 and App. C of the submission overstated this, and the revision corrects these sentences. The difference lies in the rest of the layer. Computing the Einstein midpoint alone takes longer than computing the Lorentzian centroid, yet the complete Klein GyroBN layer is the fastest of the hyperbolic BN layers we timed.

1. The complete Klein BN layer is the fastest. Table B1 times one BN layer of each network in its own implementation on the same A100, with ILNN in its official code. Klein GyroBN is the fastest at every width from 16 to 512 channels. In memory it is the lowest at 16 and 64 channels, and at 256 and 512 channels it uses slightly more than the official ILNN layer.

Table B1- One BN layer with 512 channels in training mode, forward and backward under torch.compile on one A100. Time is the median of 30 steps averaged over 5 jobs.

BN layerTime (ms)Memory (MB)
Poincaré ResNet [g]1531158
ILNN [h]4.35516
HCNN [e]3.45644
Klein GyroBN2.01578

Note: Data extracted from Tab. 27 of the revision, with ILNN in its official code and HCNN with its shift as a plain parameter, since the original ManifoldParameter breaks the compiled graph 16 times (7.47 ms).

2. Inside a Klein network, the native centre is the cheapest at comparable accuracy. Tables B2 and B3 swap only the centre of Klein GyroBN under the protocol of Tab. 7. The three centres reach comparable accuracy, and the native Einstein midpoint is the fastest because the other two convert the batch to another model and back.

Table B2- Accuracy (%) with only the centre of Klein GyroBN swapped, 5 seeds

CentreDiseaseAirportPubMedCora
Lorentzian centroid via Eq. (27)80.4898.5072.5256.18
Poincaré midpoint via Eq. (26)81.0598.6873.0255.94
Einstein midpoint (native)80.5798.3772.6056.48

Table B3- Fit time (ms per epoch) of the same runs

CentreDiseaseAirportPubMedCora
Lorentzian centroid via Eq. (27)81.5222.313.827.3
Poincaré midpoint via Eq. (26)87.7248.314.629.5
Einstein midpoint (native)78.2213.813.326.3

Note: All three centres ran back to back in the same jobs. These jobs are separate from those of Tab. 7, and the largest difference from its Einstein row is $ 1.56$ points on Cora.

In short, closed-form aggregation is not specific to the Klein model, the native Einstein midpoint is the cheapest centre inside a Klein network at comparable accuracy, and the complete Klein GyroBN layer is the fastest hyperbolic BN layer we timed.

References

[a] Yidan Mao, Jing Gu, Marcus C. Werner, and Dongmian Zou. Klein Model for Hyperbolic Neural Networks. NeurIPS Workshop on Symmetry and Geometry in Neural Representations, 2024.

[d] Ziheng Chen, Zihan Su, Bernhard Schölkopf, and Nicu Sebe. Proper Velocity Neural Networks. ICLR, 2026.

[e] Ahmad Bdeir, Kristian Schwethelm, and Niels Landwehr. Fully Hyperbolic Convolutional Neural Networks for Computer Vision. ICLR, 2024.

[g] Max van Spengler, Erwin Berkhout, and Pascal Mettes. Poincaré ResNet. ICCV, 2023.

[h] Xianglong Shi, Ziheng Chen, Yunhan Jiang, and Nicu Sebe. Intrinsic Lorentz Neural Network. arXiv:2602.23981, 2026.

(Continued in 3/7.)

(Continuation of 2/7.)

3. A representative end-to-end experiment integrating the proposed components under controlled settings

We thank the reviewer for this suggestion. We agree that the image experiments of the submission mainly replace the classifier and that the graph experiments differ in normalization and activation. Below we answer with an end-to-end network built only from Klein layers, a replacement of its components one at a time, and a rerun of the graph experiments in one configuration.

1. End-to-end experiment under controlled settings. We built Klein-ResNet, a ResNet-18 in which every layer is a Klein layer, and trained it with the protocol of HCNN [e] (RiemannianSGD, learning rate 0.1 multiplied by 0.2 at epochs 60, 120 and 160, weight decay 5e-4, batch size 128, 200 epochs). Table C compares it with the Euclidean, Poincaré and Lorentz ResNets. On CIFAR-100 it matches HCNN in accuracy and trains 4.0 times faster than the reference HCNN implementation with about a third of its GPU memory. Most of this gap comes from two implementation steps of that code, and with both removed by hand HCNN runs at the speed of Klein-ResNet, which still uses about a fifth less memory, as we show in our response to Reviewer fKfm\color{green}{\text{fKfm}}. Klein-ResNet is added to §3.5 and §4.6 of the revision.

Table C- ResNet-18 on CIFAR-100 and Tiny-ImageNet, 5 seeds. Baseline accuracies as reported in the original papers; time and memory measured by us on one A100.

MethodCIFAR-100 Acc.VRAM$ t _{\text{epoch}}$Tiny-ImageNet Acc.VRAM$ t _{\text{epoch}}$
Euclidean ResNet [f]77.72 ± 0.150.8 GB4.9 s65.19 ± 0.122.5 GB29.9 s
Poincaré ResNet [g]76.60 ± 0.3214.6 GB195.3 s62.01 ± 0.5657.6 GB949.7 s
ILNN [h]78.41 ± 0.2311.2 GB56.5 sNaN (5/5), our run44.2 GB372.4 s
HCNN [e]78.07 ± 0.177.7 GB62.4 s65.71 ± 0.1330.3 GB270.3 s
Klein-ResNet78.09 ± 0.202.8 GB15.6 s63.03 ± 0.2510.5 GB110.7 s

Note: Timed under torch.compile with the released code of Poincaré ResNet and HCNN and our implementation of ILNN. The Tiny-ImageNet entry of ILNN is our run of the official ILNN code under the same protocol, since ILNN reports no Tiny-ImageNet accuracy.

The baseline accuracies of Table C are those reported in the original papers. Under our pipeline and the same protocol, the official HCNN code reaches 77.76 ± 0.19 on CIFAR-100 over eight seeds and 62.11 ± 0.23 on Tiny-ImageNet over five, against 78.09 ± 0.20 and 63.03 ± 0.25 for Klein-ResNet. On Tiny-ImageNet Klein-ResNet stays below the accuracies reported by HCNN and by the Euclidean ResNet, and the official ILNN code stops with a non-finite GyroBN output in all five seeds after completing between 64 and 84 of the 200 epochs.

2. Contributions of the components. To isolate what each Klein component adds, we replaced one component of Klein-ResNet at a time by its tangent-space version, which applies the Euclidean operation between the logarithmic and exponential maps at the origin as in Mao et al. [a], and the Klein classifier by a Euclidean MLR (Table C2).

Table C2- Klein-ResNet on CIFAR-100 with one component replaced at a time, same protocol and seeds as Table C. Δ is the change from the full network. This is a separate run from the operator ablation of Tab. 52 of the revision, which uses seeds 30 to 34 and does not replace the batch normalization or the classifier.

Replaced componentAccuracy (%)Δ
None (full Klein-ResNet)78.09 ± 0.20—
Convolution77.93 ± 0.31−0.16
Activation77.41 ± 0.28−0.68
Residual addition77.84 ± 0.30−0.25
Batch normalization76.55 ± 0.21−1.54
Classifier77.67 ± 0.17−0.42

References

[a] Yidan Mao, Jing Gu, Marcus C. Werner, and Dongmian Zou. Klein Model for Hyperbolic Neural Networks. NeurIPS Workshop on Symmetry and Geometry in Neural Representations, 2024.

[e] Ahmad Bdeir, Kristian Schwethelm, and Niels Landwehr. Fully Hyperbolic Convolutional Neural Networks for Computer Vision. ICLR, 2024.

[f] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep Residual Learning for Image Recognition. CVPR, 2016.

[g] Max van Spengler, Erwin Berkhout, and Pascal Mettes. Poincaré ResNet. ICCV, 2023.

[h] Xianglong Shi, Ziheng Chen, Yunhan Jiang, and Nicu Sebe. Intrinsic Lorentz Neural Network. arXiv:2602.23981, 2026.

(Continued in 4/7.)

Comments on the paper

3 comments

We thank the three reviewers for their careful reading. Reviewer ZRE9\textcolor{red}{ZRE9} found the toolkit useful for further research, Reviewer w1vu\textcolor{blue}{w1vu} found the survey of hyperbolic operations useful to practitioners, and Reviewer fKfm\textcolor{green}{fKfm} checked the appendix equations line by line. This comment gives the AC and all reviewers one place to see what changed in the revised PDF and why. The individual answers are posted under each review.

Table, equation and section numbers refer to the submitted PDF, and items that exist only in the revised PDF are marked "of the revision".

Three points raised by more than one reviewer

Novelty (w1vu W1, fKfm W1). The signed-distance MLR, the Busemann logit and GyroBN are templates from prior work, and the paper cites them as such. None of their Klein closed forms existed, and deriving them is the contribution, namely the Klein point-to-hyperplane distance with a parameterization in which all class logits are one matrix product (Thm. 3.2), the Klein Busemann function (Thm. 3.3), the closed-form FC layers (Thm. 3.4) and a closed-form centre inside GyroBN (Sec. 3.4). The Einstein midpoint is credited to Gulcehre et al. (ICLR 2019), as §1 already does, and we claim that it is native to the Klein model rather than new.

Accuracy parity (ZRE9 W3, w1vu W3, fKfm W4). The three realizations are isometric and share one function class, so equal accuracy at a well-tuned operating point is what the paper predicts, and the CIFAR significance tests (Tab. 27) and the CKA measurement (Tab. 34) confirm it. The revised abstract and conclusion state the claim as parity in accuracy at lower cost over a wider range of settings. On the graphs, the link-prediction comparison of Tab. 6 mixed normalization, activation and tuning settings across methods. We re-ran it in one configuration for every method, each with the batch normalization of its own model and one tuning grid. Klein FC is the most accurate layer on Airport, matches the best layer on Disease and Cora and is $ 0.38$ points below it on PubMed, and the Klein layers train fastest on every dataset, $ 1.6$ to $ 2.1$ times faster than the most accurate layer. The revision reports this in §4.3.

Why Klein is more stable (ZRE9 Q1, w1vu W4, fKfm W2). We located every non-finite run of the stability sweeps at the level of a single expression. The Lorentz MLR of Bdeir et al. (ICLR 2024) fails where an intermediate that equals $ \lVert z _{k}\rVert$ is computed as a difference of two large squares, and the Poincaré MLR of Shimizu et al. (ICLR 2021) fails where $ \operatorname{arsinh}(u)$ is evaluated as $ \log(u + \sqrt{1+u ^{2}})$ and $ u ^{2}$ overflows. Rewriting the Lorentz step or evaluating that head in FP64 removes its failures, so against Lorentz the difference is one of numerical implementation. The Poincaré head still fails in FP64, because under its reference radius and initialization training drives its offset up until the logits overflow. With the Klein radius and initialization it trains, and the Klein head trains under either, since it contains neither failing expression. The revised §4.1 says so.

Corrections we found while preparing this response

The Lorentz column of the stability sweeps was run at the wrong curvature. The Lorentz implementation reads the curvature argument as $ k$ with $ K = -1/k$, while the Poincaré and Klein implementations read it as $ c$ with $ K = -c$, and our sweep script passed the same number to all three. The Lorentz column of Tabs. 2–3 was therefore at $ K = -1/0.3 \approx -3.33$ where $ K = -0.3$ is printed, and at $ K = -100$ where $ K = -0.01$ is printed. We re-ran all 90 runs with the curvature checked in every log.

(Continued in 2/3.)

(Continuation of 1/3.)

Table G1: CIFAR-100 top-1 (%), 3 seeds, corrected curvature, at $ r = 1$ in the curvature sweep and $ K = -0.3$ in the clipping sweep. NaN $ k$/3 means $ k$ of the 3 seeds became non-finite.

SettingKleinPoincaréLorentz
$ K = -0.01$$ 72.29 \pm 0.21$NaN 3/3$ 72.37 \pm 0.14$
$ K = -0.1$$ 72.90 \pm 0.15$$ 73.91 \pm 0.12$$ 72.58 \pm 0.29$
$ K = -0.3$$ 73.32 \pm 0.19$$ 73.54 \pm 0.21$$ 73.10 \pm 0.20$
$ K = -1$$ 73.11 \pm 0.15$$ 73.17 \pm 0.24$$ 73.17 \pm 0.12$
$ K = -10$$ 70.88 \pm 0.16$NaN 3/3NaN 3/3
$ r = 1$$ 73.41 \pm 0.33$$ 73.47 \pm 0.19$$ 72.84 \pm 0.13$
$ r = 2$$ 74.54 \pm 0.13$NaN 2/3$ 74.68 \pm 0.56$
$ r = 5$$ 74.16 \pm 0.03$NaN 3/3$ 73.88 \pm 0.13$
$ r = 10$$ 73.51 \pm 0.12$NaN 3/3$ 72.44 \pm 0.21$
$ r = 10 ^{9}$$ 71.69 \pm 0.15$NaN 3/3NaN 2/3

The Klein and Poincaré columns agree with the submission within seed variation, and every Poincaré failure recurs. The Lorentz column changes in both directions. It now trains at $ K = -0.01$, $ r = 5$ and $ r = 10$, where the submission marks it NaN, and at $ K = -0.1$, where two seeds of the submission failed during training, and it fails at $ K = -10$, where the submission reports $ 72.71$. Lorentz therefore fails in two settings instead of four, and Klein remains the only realization that is finite in all ten. We also report the $ r = 2$ Poincaré run as non-finite in 2 of 3 seeds rather than as NaN, in §4.1 and Table G1.

The single-seed sweeps in the appendix had the same error. The revision reruns the Lorentz column of Tabs. 13, 14 and 19 at the labeled curvature, corrects the two rows of Tab. 30 derived from them, and marks the Lorentz column of the Tiny-ImageNet sweep in Tab. 15 as run at $ K = -1/0.3$.

The sentence attributing the failures to the Möbius square root is removed. The Poincaré head in the sweep contains no Möbius addition, and the localization above shows the actual failing expression.

Three Klein entries of the Tiny-ImageNet clipping sweep came from incomplete runs. In checking Tab. 15 we found that the Klein runs at $ r = 5$, $ 10$ and $ 10 ^{9}$ and the Poincaré runs at $ r = 0.1$ and $ 0.5$ iterated over only 31 to 242 of the 782 training batches per epoch. Rerun on the full training set with five seeds, Klein reaches $ 65.50$, $ 64.09$ and $ 63.32$ at these radii instead of $ 22.60$, $ 2.34$ and $ 0.99$, the last over the four of five seeds that stay finite, and the revision replaces these entries. The two Poincaré entries are not rerun, and the revision marks them as unavailable.

The node-classification runs selected their checkpoint and settings on the test split. The pipeline we reused kept the checkpoint and the hyperparameters with the best test accuracy, while the appendix stated selection on a validation split. The revision reruns PVNN, LNN++, Klein-HNN++ and Klein-HBNN++ with a validation split held out from the training nodes, one grid selected on validation accuracy and five test seeds, and keeps the HNN++, LNN and Klein HNN rows of Tab. 21 as reported by Chen et al. Under this protocol Klein-HNN++ is statistically indistinguishable from PVNN on Disease, Airport and PubMed and 6.94 points higher on Cora.

Four errors in the printed equations, none of which reaches the code. Reviewer fKfm found errors in the inverse of Eq. (27), the parallel transport of Eq. (30), the running-mean update above Alg. 1 and a sign in App. E.1. Where the code maps between the Lorentz and Klein models it uses the corrected inverse of Eq. (27), it never evaluates Eq. (30), and it computes the running mean as the correct weighted Einstein midpoint, so no reported number changes.

Two attributions are made precise. Following Reviewer ZRE9, §1 now describes Mao et al. (2024) as Klein linear layers and a network built in the tangent space at the origin with a Euclidean classifier, which is the construction that the distance-based layers of KNN++ replace, and App. C no longer says that the Einstein midpoint has no Möbius analogue.

(Continued in 3/3.)

(Continuation of 2/3.)

Corrections we found while preparing this response (cont.)

The framework is renamed. Following Reviewer w1vu, the revision renames KNN++ to Klein-HNN++ and KBNN++ to Klein-HBNN++, so that the acronym no longer reads as $ k$-nearest neighbours, and keeps the title. It also labels the network of Mao et al. (2024) Klein HNN, the name they use. The individual answers keep the name KNN++ of the submission.

Summary of revisions

  • Contribution statement in §1. The contribution is the Klein closed forms of layers that have counterparts in the isometric models, the network built only from Klein layers that they make possible, and the cost and numerical operating range of the resulting toolkit.
  • Credit and attribution in the abstract, §1, §3.4 and App. C. The closed-form Einstein midpoint is credited to Gulcehre et al., the closed-form centres of the Lorentz and Poincaré batch normalizations are stated, App. C names the prior construction behind each Klein layer and no longer says the midpoint has no Möbius analogue, and Mao et al. are described as a Klein linear layer and a network built in the tangent space, with a timing comparison in App. D.3.3.
  • Definition of intrinsic at the start of §3. The word is defined following Chen et al. and Shi et al., and the closed forms are stated as specific to the Klein model.
  • Name Klein-HNN++ in §1 and every table. KNN++ and KBNN++ become Klein-HNN++ and Klein-HBNN++, and the network of Mao et al. is labelled Klein HNN.
  • Stability sweeps in §4.1 and App. D.1. The Lorentz column is rerun at the labelled curvature, Tab. 3 is indexed by geodesic radius, Tab. 4 and the sentence on the Möbius square root are replaced by the failing expression of every non-finite run and its FP32 prediction, and the limit of the Klein coordinates is stated.
  • Tiny-ImageNet clipping sweep in App. D.1. The Klein entries of Tab. 15 at $ r \ge 5$ are replaced by five-seed runs on the full training set, and the two incomplete Poincaré entries are marked unavailable.
  • Accuracy claims in §1 and §4.2. Image classification is stated as parity with the Poincaré and Lorentz realizations.
  • Graph experiments in §4.3, §4.4, App. D.3 and App. D.4.3. Tab. 6 is replaced by link prediction in one configuration for every method, each with the batch normalization of its own model, the graph rows of Tab. 25 compare Klein FC and Klein BFC with only the boundary type differing, node classification is rerun with selection on a validation split, and the claims of significant gains on graphs are removed, apart from node classification on Cora ($ 6.94$ points, $ p = 0.001$).
  • Klein-ResNet in §3.5 and §4.6 of the revision. A ResNet-18 built only from Klein layers matches HCNN on CIFAR-100 and trains $ 4.0$ times faster than the reference HCNN with a third of its memory. An HCNN optimized by hand matches its speed but needs an added radius bound to train and uses more memory.
  • Efficiency statements in §1, §3, §4.5 and App. E. The Klein head is the fastest by a small margin over BMLR-L, and every FLOP comparison states its condition.
  • Scope of Prop. E.2 and printed equations in App. C and App. E. Prop. E.2 is stated as justifying the midpoint as the GyroBN centre, without a comparative stability claim, and Eqs. (27), (30), the running-mean update above Alg. 1 and the sign in App. E.1 are corrected.
  • Layout of Tabs. 5–7. The three tables are set as floats.
  • Code. The stability sweeps and the link-prediction and node-classification pipelines are at https://anonymous.4open.science/r/klein-hnn-anon-057F.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.