acceptodds
Under review as a conference paper at ICLR 2027

HexFormer: Hyperbolic Vision Transformer with Exponential Map Aggregation

Abstract

Data across modalities such as images, text, and graphs often contains hierarchical and relational structures, which are challenging to model within Euclidean geometry. Hyperbolic geometry provides a natural framework for representing such structures. Building on this property, this work introduces HexFormer, a hyperbolic vision transformer for image classification that incorporates exponential map aggregation within its attention mechanism. Two designs are explored: a hyperbolic ViT (HexFormer) and a hybrid variant (HexFormer-Hybrid) that combines a hyperbolic encoder with an Euclidean linear classification head. HexFormer incorporates a novel attention mechanism based on exponential map aggregation, which yields more accurate and stable aggregated representations than standard centroid based averaging, showing that simpler approaches retain competitive merit. Experiments across multiple datasets demonstrate consistent performance improvements over Euclidean baselines and prior hyperbolic ViTs, with the hybrid variant achieving the strongest overall results. Additionally, this study provides an analysis of gradient stability in hyperbolic transformers. The results reveal that hyperbolic models exhibit more stable gradients and reduced sensitivity to warmup strategies compared to Euclidean architectures, highlighting their robustness and efficiency in training. Overall, these findings indicate that hyperbolic geometry can enhance vision transformer architectures by improving gradient stability and accuracy. In addition, relatively simple mechanisms such as exponential map aggregation can provide strong practical benefits.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Computer Vision, Image Classification, Vision Transformer, Hyperbolic Geometry

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_DxqgOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes HexFormer, a vision Transformer built in the Lorentz model of hyperbolic space. It computes attention scores from the squared Lorentzian distance and aggregates value vectors in the tangent space via the exponential map (ExpAgg), instead of the commonly used centroid aggregation. Two variants are presented: a fully hyperbolic HexFormer and HexFormer-Hybrid, which pairs a hyperbolic encoder with a Euclidean classification head. The experiments cover CIFAR-10, CIFAR-100, and Tiny-ImageNet and examine performance across different model sizes, warmup settings, optimizers, aggregation methods, and image corruptions. The hybrid model achieves the best classification accuracy, but at a substantially higher computational cost than the Euclidean ViT, more than doubling both training and inference time.

Critical strengths
  1. The exponential-map aggregation is simple, but it fixes the NaN problem that centroid aggregation runs into.
  2. The experiments are thorough, covering different scales, optimizers, gradient behavior and corruption robustness.
  3. The authors are honest about the limitations, such as the higher cost and no gain on non-hierarchical data.
Critical weaknesses
  1. The novelty is limited. Most components are borrowed from prior work, such as HyboNet [1], HCNN [2] and LResNet [3], and the authors admit that ExpAgg itself barely improves accuracy, so the main contribution comes down to a stability fix. [1] Fully Hyperbolic Neural Networks. ACL, 2022. [2] Fully Hyperbolic Convolutional Neural Networks for Computer Vision. ICLR, 2024. [3] Lorentzian Residual Neural Networks. KDD, 2025.

  2. The accuracy gains are small. HexFormer improves over the Euclidean ViT by only 0.04 points on CIFAR-10 (93.42 vs. 93.38) and 0.13 points on Tiny-ImageNet (60.87 vs. 60.74), both well within the standard deviation (±0.18 and ±0.64). There are also no results on larger benchmarks like ImageNet-1k. While gains are clearer on CIFAR-100 (+1.01) and for the hybrid variant on Tiny-ImageNet (+2.19), they are not consistent across datasets and variants.

  3. The Euclidean baseline is weak. The authors deliberately avoid a competitive training recipe (Section 5.1), and the baseline even gets worse as the model grows, dropping from 74.64 to 73.60 to 70.54 from Tiny to Base on CIFAR-100 with warmup (Table 8). This raises the question of whether it was properly tuned at larger scales (Table 7).

  4. The efficiency cost is insufficiently emphasized in the main paper. On CIFAR-100 with ViT-Tiny, HexFormer and HexFormer-Hybrid take 117.42 and 121.26 ms per training iteration versus 43.77 ms for the Euclidean ViT, roughly 2.7–2.8× slower (Table 19). Inference takes about 0.31 ms per image versus 0.13 ms, roughly 2.3× slower (Table 20), and reserved GPU memory rises from 1.90 GB to 3.33 GB (Table 21). These costs are only reported in the appendix and briefly mentioned in Section 5.5. Given the small accuracy gains, the accuracy–efficiency trade-off should be presented alongside the main results.

  5. It’s not clear the gains really come from hyperbolic geometry. The paper says the two models “differ only in geometry” (Section 5.1), but HexFormer also uses Lorentz batch normalization instead of standard pre-layer normalization, plus different projections and scale constraints, and any of these could affect accuracy and warmup sensitivity on their own. In fact, with AdamW and warmup, the two models perform almost identically on CIFAR-100 (74.63 vs. 74.64, Table 10). The gap only shows up without warmup, which is exactly where normalization matters most. Ablations like a Euclidean ViT with batch norm would help separate the effect of curvature from everything else.

Questions for authors
  1. The paper says the converse hybrid (Euclidean encoder with a hyperbolic head) is not competitive, but no numbers are given. Could you report these results?
  2. How does HexFormer do on ImageNet-1k?
  3. Could the authors report FLOPs including the hyperbolic operations omitted by THOP?
2 comments

We thank the reviewer for the detailed review. Table, section and figure numbers refer to the revised PDF.

W1: Novelty

Sec. 4.1 credits the building blocks from HyboNet, HCNN and LResNet. Our contributions are:

  • ExpAgg, which removes the numerical failure mode of centroid attention (A.4).
  • A combination of hyperbolic components that clearly outperforms the existing hyperbolic ViT (LViT) in the same pipeline, on all four datasets.
  • To our knowledge, the first study of the gradient stability and warmup sensitivity of hyperbolic transformers compared with Euclidean ones (Sec. 5.3).

W2 / Q2: Gains and ImageNet-1k

New Table 2 (224×224, ViT-Tiny; mean ± std over 3 seeds):

ModelImageNet-1k
Euclidean ViT67.00 ± 0.08
LViT67.06 ± 0.23
HexFormer67.79 ± 0.13
HexFormer-Hybrid66.55 ± 0.20

HexFormer outperforms the Euclidean ViT by 0.79 points, a statistically significant difference (Welch's t-test, p = 0.002), while LViT is on par with the Euclidean ViT. One of our two models is the most accurate on every dataset: the hybrid on CIFAR-10, CIFAR-100 and Tiny-ImageNet, and HexFormer on ImageNet-1k. Over LViT, HexFormer gains 1.45, 4.42, 6.51 and 0.72 points on the four datasets (statistically significant on ImageNet-1k, p = 0.015). Removing warmup costs the Euclidean ViT 2.78 points on CIFAR-100, but HexFormer only 0.59 (Table 9).

W3: Euclidean baseline

To compare the geometries, we used the same simple ViT and training recipe for both models, so the benefit of the geometry is clear before further improvements are added (Sec. 5.1). The Euclidean ViT was tuned at every scale (Table 8), and all models use drop path 0.1 and the same augmentation. Its decline with scale comes from overfitting: the training–validation gap of Euclidean ViT-Base ends near +8 points, while HexFormer-Base stays below zero (A.3.1, Fig. 4). On ImageNet-1k, it is still outperformed.

W4: Efficiency

The cost is reported in A.5, mentioned in Sec. 5.5, and stated as a limitation. Inference remains fast (about 0.31 ms per image). Training time and memory are higher, which is a known challenge for hyperbolic networks. The corrected FLOP count (Q3) shows only about 4% arithmetic overhead; the time overhead comes mainly from memory-bound element-wise operations.

W5: Do the gains come from the geometry?

Lorentz linear layers and normalization on the manifold are the hyperbolic counterparts of the Euclidean layers, needed to keep representations on the hyperboloid and used in the same places; they are not additional design choices. HexFormer's CLS token and normalization offsets are points on the manifold, which Riemannian AdamW keeps there; for all other parameters it performs the AdamW update. With AdamW for both models, the warmup-10 comparison is a tie, but without warmup HexFormer leads by 2.8 points (Table 11). Matched-optimizer ablation (Table 12; CIFAR-100, ViT-Tiny, warmup 10, same seed):

ModelAdamWRiemannian AdamW
Euclidean ViT74.5774.86
LViT71.4270.79
HexFormer75.2275.68
HexFormer-Hybrid74.9575.95

We also ran the suggested Euclidean ViT with batch normalization (CIFAR-100, ViT-Tiny, AdamW; Table 13):

ModelNo warmupWarmup 10
Euclidean ViT, layer norm (10 seeds)71.86 ± 0.2674.64 ± 0.41
Euclidean ViT, batch norm (3 seeds)70.47 ± 2.2672.85 ± 0.13

Batch normalization lowers accuracy in both settings and still loses 2.4 points without warmup, so normalization alone does not explain HexFormer's behavior.

Q1: Converse hybrid

A Euclidean encoder with a Lorentz MLR head reaches 73.63 ± 0.28 (CIFAR-100, ViT-Tiny, warmup 10, 3 seeds), below the Euclidean ViT (74.64). The benefit therefore comes from hyperbolic feature extraction. Added to A.3.

Q3: FLOPs

We recomputed the FLOPs with an operator-level counter (at the PyTorch dispatcher). It includes all hyperbolic operations, as well as the attention products, which THOP misses for every model. Forward FLOPs per image (ViT-Tiny, CIFAR-100):

ModelParamsTHOP (previous)TotalOverhead
Euclidean ViT2.71 M0.351 G0.380 G–
HexFormer (ExpAgg)2.72 M0.352 G0.396 G+4.1%
HexFormer-Hybrid (ExpAgg)2.72 M0.352 G0.396 G+4.1%
HexFormer (Centroid)2.72 M0.352 G0.394 G+3.7%

The hyperbolic overhead is about 4% (not 0.4%), and ExpAgg costs about the same as the centroid. Element-wise operations are memory-bound, so the measured runtime (Tables 22–24) remains the practical measure. Table 25 is updated.

I thank the authors for the detailed response and the additional experiments, in particular the ImageNet-1k results, the converse hybrid numbers, and the corrected FLOP count. However, some of my concerns remain unresolved:

  1. The gains on CIFAR-10 and Tiny-ImageNet are still within noise, and the gain on ImageNet-1k is only +0.79.
  2. The authors do not explain why the Hybrid variant helps on small datasets but hurts on ImageNet-1k, so it remains unclear which variant should be used when.
  3. The accuracy–efficiency trade-off is still not shown with the main results. That higher cost is a known challenge for hyperbolic networks does not tell us whether these small gains are worth a ~2.7× training slowdown (Appendix A.5).
Reviewer_3UesOpenReview ↗
Rating
2
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes HexFormer, a hyperbolic vision transformer for image classification. By incorporating exponential map aggregation into its attention mechanism and making some other adaptations, HexFormer and its hybrid variant show competitive performance on CIFAR-10, CIFAR-100, and Tiny-ImageNet. The paper further explores the model's warmup dependency and gradient stability, as well as the effects of different aggregation strategies.

Critical strengths

As I understand it, the main contribution of this paper is the adoption of the exponential map aggregation strategy, and the authors demonstrate its effectiveness through experiments.

The paper further discusses HexFormer's sensitivity to training strategies and its gradient behavior, comparing them with those of a standard ViT, and presents some interesting results.

Critical weaknesses

The analysis of the advantages of the exponential map aggregation strategy is limited to experiments and empirical explanations, without theoretical results. Meanwhile, HexFormer's improvements over the baselines are marginal, which limits the impact of this work.

The writing of the paper needs improvement. The methodology and preceding sections occupy nearly six pages, but many modules in Section 4.1 are borrowed from previous work, such as Patch Embeddings, Lorentz Batch Normalization and the Classification Head. Furthermore, the authors spend nearly a full page introducing the background in the Preliminaries section, yet Equation (1), which is also a core computation of the proposed method, contains an error: −2K-2K should be 2/K2/K. This makes me feel that the authors are merely stacking formulas.

Questions for authors

Could the authors provide experimental results on ImageNet, as LViT did?

Could the authors explain what lines 260–262 mean: "The pairwise compatibility is computed from ... With curvature parameter K < 0 the (squared) geodesic distance may be expressed via the Lorentzian product." The geodesic distance does not seem to appear in the equations before or after this statement. As I understand it, the geodesic distance can indeed be expressed in terms of the Lorentzian inner product, but this relationship is nonlinear, so they cannot be directly substituted for each other within the Softmax operation.

Is the proposed method itself limited to image classification or the field of computer vision?

In addition to the formula issue mentioned in the Weaknesses section, I think the definition of ϵ\epsilon in line 221 may also be incorrect, since KK is defined as negative earlier in the paper.

2 comments

We thank the reviewer for the review and the useful questions. Table, section and figure numbers refer to the revised PDF.

Q1: ImageNet-1k

New Table 2 (224×224, ViT-Tiny; mean ± std over 3 seeds):

ModelImageNet-1k
Euclidean ViT67.00 ± 0.08
LViT67.06 ± 0.23
HexFormer67.79 ± 0.13
HexFormer-Hybrid66.55 ± 0.20

HexFormer achieves the highest accuracy, 0.79 points above the Euclidean ViT and 0.72 above LViT; both differences are statistically significant (Welch's t-test, p = 0.002 and p = 0.015).

W1: Theory and size of the gains

The centroid failure is a floating-point effect, which is why the paper shows a concrete float32 example (Appendix A.4):

  • In exact arithmetic, the centroid's denominator is at least 1/|K|. The failure is therefore not a mathematical problem of the centroid, but a computational one caused by floating-point precision.
  • In float32, it is computed as the difference of two nearly equal squares. The rounding error (about 10⁻⁷ · x_t²) exceeds that bound once the time component x_t reaches 10³–10⁴, and the denominator can become zero.
  • ExpAgg never divides by a Lorentzian norm. The log map grows only logarithmically, and a weighted average is no larger than its largest input, so its output is never larger than its inputs.

Over the hyperbolic baseline LViT, trained in the same pipeline, HexFormer improves by 1.45, 4.42 and 6.51 points on CIFAR-10, CIFAR-100 and Tiny-ImageNet, and by 0.72 on ImageNet-1k, which is statistically significant (p = 0.015). It also outperforms the Euclidean ViT on ImageNet-1k. Beyond accuracy, hyperbolic ViTs are much more robust to removing warmup: on CIFAR-100 it costs the Euclidean ViT 2.78 points, but HexFormer only 0.59 (Table 9). To our knowledge, this had not been studied for hyperbolic transformers before.

W2, Q2, Q4: Presentation and formulas

As is common in hyperbolic deep learning papers [1][2][3][4], the Preliminaries keep the paper self-contained, and the modules in Sec. 4.1 that come from prior work are cited and described for reproducibility. Within the architecture, our contributions are the attention and aggregation (Sec. 4.2) and the LorentzFC refinements (Sec. 4.1).

We thank the reviewer for checking the formulas so carefully and for pointing out these errors. We want to clarify that they are errors in the paper's text only; the implementation used for all experiments is correct, as explained below:

  • Eq. (1): The reviewer is right; it should read d² = 2/K − 2⟨x,y⟩_L. The "−2K" came from the convention of our implementation (k = −1/K > 0), in which the code is correct, and with K = −1 (all experiments) both forms coincide. We corrected it.
  • ε (Q4): Also right; it should read ε > √(−1/K). With K = −1, as in all our experiments, this requires ε > 1, and our implementation uses ε = 1.5. We corrected it.
  • Geodesic distance (Q2): This was a wording error: "squared geodesic distance" should read "squared Lorentzian distance" (Eq. 1), which is linear in the Lorentzian inner product, so no nonlinear substitution is involved in the softmax. Corrected in Sec. 4.2.

Q3: Beyond image classification

Only the patch embedding is image-specific; the attention operates on any sequence of tokens. HexFormer produces an embedding for every patch token, so it can serve as a backbone for detection and segmentation. Evaluating it on these tasks is future work.

References

[1] O. Ganea, G. Bécigneul, T. Hofmann. Hyperbolic Neural Networks. NeurIPS 2018.

[2] I. Chami, R. Ying, C. Ré, J. Leskovec. Hyperbolic Graph Convolutional Neural Networks. NeurIPS 2019.

[3] W. Chen et al. Fully Hyperbolic Neural Networks. ACL 2022.

[4] A. Bdeir, K. Schwethelm, N. Landwehr. Fully Hyperbolic Convolutional Neural Networks for Computer Vision. ICLR 2024.

Thank you for the detailed response. The additional ImageNet-1k results are a necessary and valuable supplement to the experimental evaluation. I have also checked the revised manuscript and confirmed that the previously identified issues with the formulas and descriptions have been corrected.

However, as also noted by reviewer Dxqg, I remain concerned that a substantial portion of both the model architecture and the manuscript is devoted to components adopted from prior work. Although the authors have reiterated their core contributions in the response to Dxqg, these contributions do not appear to receive substantially more attention in the manuscript than the descriptions of existing components. I believe the paper would benefit from a clearer separation between established techniques and the proposed contributions, with greater emphasis on the latter.

For these reasons, while I appreciate the authors' efforts and the improvements made during the rebuttal, I am inclined to maintain my current score.

Reviewer_485bOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

HexFormer is a ViT built in the Lorentz model of hyperbolic space. Attention uses Lorentzian distance. Values are aggregated in the tangent space at the origin and mapped back (ExpAgg). A hybrid variant swaps in a Euclidean linear head. Models are mainly trained on CIFAR-10/100 and Tiny-ImageNet and compared with a Euclidean ViT and LViT.

Critical strengths
  1. The experiments are reported fairly. LViT is retrained in the same pipeline and exceeds its published accuracy. Negative findings are also reported openly.
  2. At Tiny and Small scale, removing warmup costs hyperbolic models under 0.8 points, versus 1.3-3.1 for the Euclidean ViT. This is a useful practical observation, backed by a quantitative gradient analysis.
Critical weaknesses
  1. The gain is not clearly due to the geometry. On CIFAR-10 and Tiny-ImageNet, HexFormer is within noise of the Euclidean ViT. On CIFAR-100, the +1.0 gain disappears when both models use AdamW (74.63 vs 74.64). So the gain in the standard setting appears tied to the optimizer choice. Why the Euclidean head of the hybrid variant helps is also not analyzed.
  2. The scaling claim relies on a questionable baseline. The Euclidean ViT gets worse as it grows (74.6 → 73.6 → 70.5). It uses the same ~3e-3 learning rate for Small and Base, and the recipe lists no stochastic depth. In addition, all experiments use small images of at most 64 pixels. Given the 2.7x training and 2.3x inference cost, ImageNet-scale evidence is needed.
  3. Tangent-space aggregation is standard in hyperbolic GNNs such as HGCN, which is not cited. ExpAgg's gain over the centroid is negligible. The centroid fails only under plain AdamW, and only a tiny epsilon guard is discussed.
Questions for authors
  1. What are the results on all datasets and scales when all models use the same optimizer? Why does Riemannian AdamW hurt the Euclidean ViT, which has no manifold parameters?
  2. Does HexFormer still win against a properly regularized Euclidean ViT? Does the gain hold on ImageNet-1k?
1 comment

We thank the reviewer for the constructive review. Table, section and figure numbers refer to the revised PDF.

W1 / Q1: Geometry or optimizer?

Optimizer and geometry are coupled. HexFormer has parameters that are points on the hyperboloid (the CLS token and the offsets of the Lorentz normalization layers). Riemannian AdamW keeps them on the manifold, while plain AdamW moves them off it; for all other parameters, the two optimizers perform the same update. The optimizer alone does not produce the gain: LViT also uses Riemannian AdamW (Table 7), but it stays below the Euclidean ViT on CIFAR-10, CIFAR-100 and Tiny-ImageNet and is only on par with it on ImageNet-1k, while HexFormer outperforms LViT on all four datasets (ImageNet-1k results under W2/Q2).

New matched-optimizer ablation (Table 12; CIFAR-100, ViT-Tiny, warmup 10, same seed for all runs):

ModelAdamWRiemannian AdamW
Euclidean ViT74.5774.86
LViT71.4270.79
HexFormer75.2275.68
HexFormer-Hybrid74.9575.95

With the same optimizer, HexFormer is ahead of the Euclidean ViT under both. These are single runs. With warmup, the 10-seed AdamW comparison is a tie (Table 11), but without warmup HexFormer leads by 2.8 points with AdamW for both models (74.68 vs 71.86). Repeating this on all datasets and scales requires a large number of additional runs, which was not feasible within the rebuttal period.

Does Riemannian AdamW hurt the Euclidean ViT? No. Without manifold parameters, the two optimizers are the same algorithm, and with a shared seed they give 74.86 vs 74.57 (Table 12), whereas Table 11 averages runs with different seeds. Even same-seed GPU runs are not bit-identical (kernels with a non-fixed summation order, including those from torch.compile, and TF32), so differences of a few tenths are expected with the same seed, and larger ones with different seeds.

Hybrid head. One of our two models is the most accurate on every dataset: the hybrid on CIFAR-10, CIFAR-100 and Tiny-ImageNet, and HexFormer on ImageNet-1k. The benefit of the Euclidean head therefore depends on the dataset, and we revised the paper's claims accordingly.

W2 / Q2: Baseline, scaling and ImageNet-1k

ImageNet-1k (new Table 2; 224×224, ViT-Tiny; mean ± std over 3 seeds):

ModelImageNet-1k
Euclidean ViT67.00 ± 0.08
LViT67.06 ± 0.23
HexFormer67.79 ± 0.13
HexFormer-Hybrid66.55 ± 0.20

At full scale and resolution, HexFormer outperforms the Euclidean ViT by 0.79 points, a statistically significant difference (Welch's t-test, p = 0.002), while LViT is on par with the Euclidean ViT.

The learning rate and weight decay of the Euclidean ViT were tuned at every scale (Table 8). All models use drop path 0.1 at all scales, plus label smoothing, CutMix/MixUp, AutoAugment and random erasing. We apologize for not listing the drop path rate and have added it to Table 7. Under identical regularization, the Euclidean ViT overfits much more as it grows: the training–validation gap of Euclidean ViT-Base ends near +8 points, while HexFormer-Base stays below zero (A.3.1, Fig. 4). This explains its decline with scale. We agree that the cost (about 2.7× training and 2.3× inference time) is a real limitation, and we report it in A.5.

W3: HGCN and ExpAgg

We added HGCN to Sec. 2.3. Tangent-space aggregation itself is not new; the difference is how it is combined with the scores. HGCN computes its attention weights with an MLP on tangent-space features, HVT computes the whole attention in the tangent space, and LViT scores on the manifold but aggregates with the centroid. HexFormer scores with the squared Lorentzian distance on the manifold and aggregates only the values in the tangent space. To our knowledge, it is the first hyperbolic ViT to combine the two, and the first to compare this aggregation with the centroid in terms of numerical stability.

ExpAgg's role is stability rather than accuracy (Sec. 5.4). HexFormer with the centroid (75.38) is still far above LViT (71.23), which also uses it. The centroid failure is not specific to AdamW: it happens in the forward pass, when the Lorentzian norm in the denominator, a difference of two nearly equal squares, rounds to zero in float32 (A.4). The optimizer only changes how often this happens. An ε guard does not fix it, since a small ε produces gradients of order 10^15. ExpAgg never divides by a Lorentzian norm, so it avoids the problem by construction.

Comments on the paper

1 comment

We thank all reviewers for their careful reviews and helpful suggestions. We have uploaded a revised PDF. Because of the new Table 2, the later tables are renumbered, and our replies use the revised numbering. The main changes are:

New results

  • ImageNet-1k (Table 2, Sec. 5.2): all four models with ViT-Tiny at 224×224. HexFormer achieves the highest accuracy (67.79, vs. 67.00 for the Euclidean ViT and 67.06 for LViT), averaged over 3 seeds; both differences are statistically significant (Welch's t-test, p = 0.002 and p = 0.015).
  • Matched-optimizer ablation (Table 12, A.3.2): all four models trained with both AdamW and Riemannian AdamW, with the same seed.
  • Euclidean ViT with batch normalization (Table 13, A.3.2): it is worse than the Euclidean ViT with layer normalization reported in the main tables, and it does not reproduce HexFormer's accuracy or its robustness to warmup.
  • FLOPs (Table 25, A.5): recomputed including all hyperbolic operations. The overhead is about 4%.

Corrections and clarifications

  • Eq. (1) and the ε condition in Sec. 4.1 are corrected. These were errors in the text only; the implementation is correct.
  • Sec. 4.2: a wording error in the description of the score computation is corrected; "squared geodesic distance" is replaced by "squared Lorentzian distance".
  • Sec. 2.3: HGCN is now cited.
  • The claims that the hybrid is the strongest variant overall are revised, since HexFormer outperforms HexFormer-Hybrid on ImageNet-1k. Both are our proposed models, and one of the two is the most accurate on every dataset: the hybrid on CIFAR-10, CIFAR-100 and Tiny-ImageNet, and HexFormer on ImageNet-1k.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.