- Rating
- 3(+2)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The authors introduce HBFormer, a hyperbolic transformer-based model designed to operate across multiple domains, including graph learning, computer vision, and natural language processing. They reformulate the fundamental components of the Transformer, including linear transformations, similarity computation, and feature aggregation, such that HBFormer operates intrinsically within hyperbolic space under the Lorentz model. In particular, the authors employ the Busemann function to formulate a hyperbolic counterpart of the dot product, enabling attention scores to be computed directly in hyperbolic space. Nevertheless, the paper raises several significant concerns regarding the architectural design choices, and insufficient justification to explain the effectiveness of the proposed components.
Critical strengths
- Applying Busemann function as similarity function for hyperbolic graph transformer, instead of geodesic distance, is both well motivated and novel, with a clear interpretation provided in the Introduction (lines 45-53)
Critical weaknesses
-
The authors do not provide any supplementary materials, such as source code or a link to a GitHub repository, making it difficult to assess the reproducibility of the proposed method.
-
Some existing hyperbolic Transformer methods, such as Hypformer, still employ the standard Euclidean dot product rather than geodesic distance or other intrinsic hyperbolic similarities. Therefore, the paper would benefit from a clearer explanation of why the intrinsic properties of the Busemann function are advantageous over the Euclidean dot product in the Lorentz model. Reporting higher accuracy alone is insufficient to explain the source of this advantage. Additional theoretical analysis or experiments are needed to identify which geometric properties are preserved by the Busemann function and under what conditions it is expected to outperform the Euclidean dot product.
-
In Eq. (5), the Lorentz inner-product term should be written as to ensure dimensional and notational consistency. This issue may also affect to Eq. (13), which defines the similarity function in the Lorentz model and serves as a core component of the proposed Busemann attention mechanism.
-
Eq. (7) lacks a clear definition of . To the best of my knowledge, denotes the special orthogonal group, i.e., the set of orthogonal matrices with determinant one. If this is indeed the intended definition, then the proposed isometry map appears closely related to the formulation introduced in H2H-GCN [1]. This may weaken the novelty claim regarding the proposed orthogonal-weight construction, and the authors should clarify the distinction between their approach and prior work.
-
The authors construct an intrinsic hyperbolic fully connected layer by decomposing the standard FC layer into multiple regression hyperplanes and then applying Lorentz multinomial logistic regression. However, compared with a conventional FC layer followed by logarithmic mapping, the Lorentz multinomial logistic regression formulation is considerably more complex and may introduce substantial computational overhead. Moreover, the empirical benefit appears marginal: Appendix G.2 reports an accuracy improvement of less than . The authors should therefore provide a direct comparison of the computational efficiency of L2EFC against the simpler FC + logarithmic mapping alternative. If L2EFC is not more efficient, the paper should further justify whether such a complex replacement is needed, given its limited performance gain.
[1] Jindou Dai, Yuwei Wu, Zhi Gao, and Yunde Jia. A hyperbolic-to-hyperbolic graph convolutional network. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 154–163, 2021.
Questions for authors
-
How are the weight matrices in the Lorentz homomorphism transformation and point-to-hyperplane transformation optimized in practice? In particular, the weight matrices in the Lorentz homomorphism transformation must satisfy the special orthogonal group constraints, which likely require a specialized optimization procedure to preserve these constraints during training. Such a procedure may introduce additional computational overhead compared with simpler alternatives such as HypFormer. A more detailed discussion of the optimization strategy, together with an empirical comparison of computational cost, would therefore strengthen the paper.
-
Which version of HBFormer is reported in the main experimental results: the full-attention variant or the linear-attention variant? It would be helpful if the authors could report the performance of both versions across all main experiments for a more complete comparison.
Hi Reviewers, We are now near the end of the rebuttal period. If you haven't done so yet, please review the authors' responses and post a reply addressing whether their clarifications or new results resolve your concerns. Even if your score or recommendation remains unchanged, a brief note confirming what did or didn't shift your assessment—and why—is essential for the discussion and final decision. Thank you for your time and engagement. Best regards, Area Chair