acceptodds
Under review as a conference paper at ICLR 2027

Pseudo-Hyperbolic Neural Networks

Abstract

Standard hyperbolic neural networks typically rely on a single timelike dimension, providing a strong inductive bias for hierarchical data but limiting their flexibility in modeling heterogeneous data with mixed structural patterns. In this work, we investigate representation learning on pseudo-hyperbolic manifolds with multiple timelike dimensions. Our main contribution is the proposal of Pseudo-Hyperbolic Multinomial Logistic Regression (QMLR), which, to our knowledge, is the first MLR classifier formulated directly on pseudo-hyperbolic manifolds. Specifically, QMLR derives an exact spacelike point-to-hyperplane decision function and constructs a timelike surrogate for cases where the corresponding global minimum-distance formulation is not well-defined. Class-dependent offsets are encoded through generalized Lorentz boosts, while a learnable gate adaptively combines the two causal branches. To enhance the representations supplied to QMLR, we further develop a pseudo-hyperbolic attention module that integrates geometry-aware aggregation, radial projection, and learnable residual interpolation while preserving the manifold constraint. Extensive experiments on graph node classification, electroencephalography (EEG) decoding, image classification, and genomic sequence learning demonstrate that our proposed framework outperforms the compared baselines in most settings, supporting pseudo-hyperbolic geometry as a flexible alternative for heterogeneous representation learning.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
hyperbolic representation learning, pseudo-Riemannian manifolds, multinomial logistic regression, pattern recognition, lorentz model, mixed topologies

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_uZdYOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes QMLR, a new classifier for pseudo-hyperbolic manifolds with multiple timelike dimensions. It combines spacelike and timelike decision scores using a learnable gate. This study also introduces a pseudo-hyperbolic attention module to improve feature representations while keeping them on the manifold.

Critical strengths
  1. This study roposes a new classifier based on pseudo-hyperbolic geometry, which is an interesting and less explored direction in geometric deep learning.
  2. The proposed method is evaluated on several different tasks, including graph classification, EEG decoding, image classification, and genomic sequence learning, showing its potential for use in different application areas.
  3. This paper also includes ablation studies and comparisons with existing methods.
Critical weaknesses
  1. The timelike branch uses a surrogate score instead of a well-defined minimum-distance formulation, which makes its geometric meaning less clear.
  2. The performance improvements are not consistent across all datasets, and QMLR does not always outperform existing methods.
  3. Some design choices, such as the number of timelike and spatial dimensions, may strongly affect performance, but their sensitivity and general impact need further investigation.
Questions for authors
  1. How sensitive is QMLR to the choice of timelike and spatial dimensions, and how should these values be selected in practice?
  2. Can the author/authors provide a stronger theoretical explanation for why the proposed timelike surrogate score is suitable for classification?
  3. Why does QMLR perform worse than some baseline methods on certain genomic sequence tasks?
  4. Can you provide more analysis to explain why the proposed method gives clear improvements on some datasets but only small or no improvements on others?
  5. How well does QMLR scale to larger datasets and deeper models, and what additional computational cost does it introduce compared with standard multinomial logistic regression?
3 comments
AuthorsOpenReview ↗

Response to Reviewer uZdY - Part 1

We thank Reviewer uZdy for the valuable comments.

Q1. Sensitivity to timelike and spatial dimensions

The signature affects performance substantially, and the preferred allocation differs between datasets. We therefore treat (t,s)(t,s) as a validation-selected hyperparameter rather than prescribe a single signature.

  • Geometric motivation: At fixed total dimension d=t+sd=t+s, changing (t,s)(t,s) changes the balance between temporal angular directions and spatial hyperbolic directions. The constructions in [a,b] motivate considering more angular temporal directions for cyclic relations and more spatial directions for hierarchical relations.

  • Sensitivity comparison: Figure 3 uses dataset-specific signatures motivated by [a] and keeps them fixed across spacelike, timelike, and fused heads. The sensitivity analysis is shown below.

    ttssCora F1 (%)Airport F1 (%)
    11556.8093.75
    21471.1092.36
    31376.3091.99
    41278.5092.37
    51179.2091.99
    61080.2093.13
    7977.2092.37
    8881.4089.50
    9780.2091.79
    10681.0086.26
    11580.2091.60
    12480.6091.22
    13381.5092.13
    14281.7091.83
    15180.0092.70

    The reported results favor different signatures on the two datasets: (14,2)(14,2) on Cora and (1,15)(1,15) on Airport. Performance is not totally monotonic in the number of timelike directions, so validation-based selection is needed. The procedure in [b, Section 5] compares estimated graph sectional-curvature distributions with reference distributions and offers a geometry-based starting point. Appendix D.3.3 shows the sensitivity analysis and selection guidelines, referenced in Section 5.1.

Q2. Suitability of the timelike surrogate

  • Timelike distance. We appreciate the reviewer’s request for a clearer justification of the timelike branch. The spacelike branch derives its score from a minimum point-to-section distance within the admissible domain. A direct extension of this construction to timelike normals encounters two difficulties: the normal foot need not exist for every input, and the indefinite metric does not provide the same minimum-distance interpretation. Specifically, let R=−βR=\sqrt{-\beta}, let nkn_k be a unit timelike normal satisfying ⟨nk,nk⟩Q=−1\langle n_k,n_k\rangle_{\mathcal Q}=-1, and define z=−⟨p,nk⟩Qz=-\langle p,n_k\rangle_{\mathcal Q}. The component orthogonal to the normal is
\qquad \langle p_\perp,p_\perp\rangle_{\mathcal Q}=-R^2+z^2.$$ When $|z|<R$, this component is timelike and can be rescaled to obtain a normal foot $q^\star$ on the decision section. When $|z|\geq R$, it is null or spacelike, so this normalization cannot produce a point on the pseudo-hyperboloid. Moreover, even where the normal foot exists, a local timelike normal parameter does not establish a global minimum over the decision section. We therefore retain the timelike branch through an explicitly defined surrogate score. - **Classification rationale** The surrogate uses the signed normal coordinate $z$, whose zero defines the class decision section and whose sign distinguishes its two sides: $$u_k^t(p)=\|a_k\|_2F_R(z), \qquad F_R(z)=R\operatorname{asinh}(z/R).$$ This transformation preserves the zero and sign of $z$. It is also smooth and strictly increasing for every real $z$, since $$F_R'(z)=\frac{1}{\sqrt{1+(z/R)^2}}>0.$$ Thus, it preserves the ordering of each class’s signed normal coordinate and provides a finite score for every finite $z$. The surrogate also has a local geometric interpretation. Where $|z|<R$, the principal timelike normal geodesic can be written as $$p=q^\star\cos(r/R)+Rn_k\sin(r/R),$$ giving $z=R\sin(r/R)$ and $r=R\arcsin(z/R)$. Near the decision section, where $z/R$ is small, the local expansions are $$R\arcsin(z/R)=z+O(z^3/R^2), \qquad F_R(z)=z+O(z^3/R^2).$$ The surrogate therefore agrees with the local signed normal parameter to first order, while extending smoothly beyond the domain where that parameter is available. - **Shared flat limit** The timelike surrogate and the spacelike score use the same scalar transformation: $$u_k^t(p)=\|a_k\|_2F_R(z_k^t(p)), \qquad u_k^s(p)=\|a_k\|_2F_R(z_k^s(p)).$$ For a fixed, bounded normal coordinate $z$, we obtain the following expansion as $\beta\to-\infty$, or equivalently $R\to\infty$: $$F_R(z)=z-\frac{z^3}{6R^2}+O(R^{-4}) \longrightarrow z.$$ Both branches therefore approach the same linear transformation of their respective normal coordinates, which gives them a consistent limiting score scale for adaptive fusion.
AuthorsOpenReview ↗

Response to Reviewer uZdY - Part 2

Q3–4. Dataset-dependent improvements

  • Dataset dependence. We agree that QMLR’s benefits vary across datasets. In these experiments, LMLR, BMLR-L, and QMLR share the CNN architecture and experimental settings, with only the classification head changed. The results show that pseudo-hyperboloid and QMLR is useful on most tasks. For the EEG tasks, we further examine input conditioning and geometric structure to explore possible reasons for the performance differences in that domain.

  • Comparison with matrix-manifold methods. We have examined input correlation conditioning to investigate the performance differences between Q-Net and CorAtt. Experiment shows that ERN has the largest condition number among the three datasets, suggesting greater sensitivity to matrix operations involving small eigenvalues. Q-Net learns pseudo-hyperbolic embeddings from encoded EEG features, providing an alternative that may be useful in this setting.

    DatasetCorrelation condition numberQ-Net Vs CorAtt (%)
    MI1.61×1041.61\times10^40.70↓
    SSVEP5.38×1025.38\times10^20.36↓
    ERN4.38×1044.38\times10^43.58↑

    Larger condition numbers indicate poorer numerical conditioning. Performance differences use accuracy for MI/SSVEP and ROC-AUC for ERN.

  • Comparison with hyperbolic models. We also use sampled four-point hyperbolicity to examine waveform geometry. The analysis shows that ERN has the largest relative hyperbolicity value in waveform space, indicating less tree-like metric structure under this measure. Pseudo-hyperbolic geometry provides angular directions alongside hyperbolic slices, offering additional flexibility when relationships extend beyond hierarchy.

    MI δrel\delta_{\mathrm{rel}}SSVEP δrel\delta_{\mathrm{rel}}ERN δrel\delta_{\mathrm{rel}}
    0.1802±0.02820.1802\pm0.02820.2162±0.08190.2162\pm0.08190.2607±0.06750.2607\pm0.0675

    Here δrel=2δ/D\delta_{\mathrm{rel}}=2\delta/D, where δ\delta is estimated from sampled four-point maxima and DD is the sampled point cloud’s diameter. The reported dispersion is the between-subject standard deviation after averaging sampling repeats. This observation motivates considering representations beyond purely hierarchical geometry. Pseudo-hyperbolic space offers an option by combining hyperbolic slices with angular directions.

  • Manifold selection. These comparisons illustrate why no single geometry should be expected to perform best across all datasets, the following table gives practical starting points by our suggestion

    Data structureGeometryMotivation
    HierarchicalHyperbolicCaptures hierarchy.
    Periodic or cyclicCircle / sphereEncodes angular variation.
    Mixed hierarchy and cyclesPseudo-hyperbolicCombines hyperbolic and angular structure.
    Informative covarianceSPDPreserves variances and dependencies.
    Correlation with nuisance scalingCorrelation manifoldRemoves positive scaling effects.
    Subspace-basedGrassmannRepresents basis-invariant subspaces.
AuthorsOpenReview ↗

Response to Reviewer uZdY - Part 3

Q5. Scalability and computational cost

  • Asymptotic cost: For CC classes and representation dimension dd, QMLR evaluates class projections and scalar score transformations in O(Cd)O(Cd) time per example, the same asymptotic order as Euclidean MLR. Norms, hyperbolic functions, and elementwise operations add a constant computational overhead.

  • Training cost: Table 5 reports training time and classifier parameter counts under the shared ResNet-18 setup.

    MethodCIFAR-10 TimeCIFAR-10 ParamsCIFAR-100 TimeCIFAR-100 Params
    MLR14.25.13K14.151.30K
    PMLR16.35.14K33.851.40K
    PBMLR17.110.24K37.9102.40K
    BMLR-P14.55.14K14.551.40K
    LMLR14.55.13K14.551.30K
    BMLR-L14.45.14K14.351.40K
    QMLR15.05.15K14.951.50K

    Time: seconds per epoch. Params: classifier parameter count.

    We additionally evaluate training cost in node classification datasets.

    DatasetReadoutParamsInference (ms)Fwd+bwd (ms)Inference peak memory (MiB)Fwd+bwd peak memory (MiB)
    Coralog+Linear1334.0749.87110.59419.460
    CoraQMLR1340.7982.9018.99617.416
    Airportlog+Linear724.1959.10910.90919.829
    AirportQMLR730.8102.8828.82917.220
    CiteSeerlog+Linear1084.1469.20710.99919.978
    CiteSeerQMLR1090.7842.8959.06017.494
    Arxivlog+Linear7206.68220.814158.540233.576
    ArxivQMLR7213.96414.456255.104367.233
  • Pretrained backbones: To evaluate Q-Net on larger datasets and with deeper networks, we replace its Euclidean feature extractor with pretrained LaBraM or EEGPT and evaluate it on BCIC2a, TUAB, and TUEV. The encoder’s output tokens are mapped to the pseudo-hyperbolic manifold via the exponential map, followed by Q-Net’s attention, residual refinement, and QMLR classifier. We fine-tune the encoder jointly with these downstream components. Results are summarized in Section 5.2 and Table 4, with experimental details and additional metrics provided in Appendix D.1.7.

    BCIC2a

    MethodBalanced accuracyF1Cohen's κ\kappa
    LaBraM32.37±0.2132.37\pm0.2130.73±0.3930.73\pm0.399.83±0.309.83\pm0.30
    LaBraM + Q-Net34.78±0.66\mathbf{34.78\pm0.66}30.99±2.22\mathbf{30.99\pm2.22}13.04±0.88\mathbf{13.04\pm0.88}
    EEGPT44.07±3.2744.07\pm3.2738.61±4.2338.61\pm4.2325.42±4.3625.42\pm4.36
    EEGPT + Q-Net44.16±1.68\mathbf{44.16\pm1.68}41.74±1.07\mathbf{41.74\pm1.07}25.54±2.24\mathbf{25.54\pm2.24}

    TUAB

    MethodBalanced accuracyAUROCPRAUC
    LaBraM79.50±0.8179.50\pm0.8186.91±0.5186.91\pm0.5183.86±1.9683.86\pm1.96
    LaBraM + Q-Net81.35±1.50\mathbf{81.35\pm1.50}90.51±1.44\mathbf{90.51\pm1.44}90.82±1.23\mathbf{90.82\pm1.23}
    EEGPT78.66±0.2478.66\pm0.2487.52±1.0187.52\pm1.0186.94±2.1586.94\pm2.15
    EEGPT + Q-Net91.55±0.33\mathbf{91.55\pm0.33}91.11±1.23\mathbf{91.11\pm1.23}91.55±1.89\mathbf{91.55\pm1.89}

    TUEV

    MethodBalanced accuracyAUROCPRAUC
    LaBraM59.58±4.3159.58\pm4.3177.72±3.2577.72\pm3.2564.06±5.0364.06\pm5.03
    LaBraM + Q-Net64.01±3.77\mathbf{64.01\pm3.77}80.35±2.96\mathbf{80.35\pm2.96}67.74±6.10\mathbf{67.74\pm6.10}
    EEGPT66.86±2.5466.86\pm2.5483.95±0.1483.95\pm0.1474.45±0.09\mathbf{74.45\pm0.09}
    EEGPT + Q-Net67.19±2.27\mathbf{67.19\pm2.27}85.25±0.66\mathbf{85.25\pm0.66}70.32±0.1270.32\pm0.12

References

Reviewer_hS95OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Pseudo Hyperbolic Multinomial Logistic Regression (QMLR), a classifier for pseudo hyperbolic representations with multiple timelike dimensions. This method combines a spacelike score with a timelike surrogate through a learnable scalar gate. The authors also introduces an attention module based on indefinite inner products and radial projection.

The proposed classifier is a potentially useful contribution, and the experimental coverage is broad.

Critical strengths
  1. The paper focuses on the complementary problem of classification directly on the resulting representations. This is a meaningful direction beyond introducing another embedding model.
  2. The authors distinguish the restricted distance interpretation of the spacelike branch from the surrogate nature of the timelike branch. The generalized Lorentz boost provides and explicit parametrization that preserves the causal norm of the decision normal.
  3. The authors present evidence of complementary components. Specifically, Figure 3 shows different branch preferences on Cora and Airport. Also Table 3 reports that attention can substantially improve the full model SSVEP and ERN.
Critical weaknesses
  1. The attention guarantee relies on an assumption that the architecture does not explicitly enforce.
  2. If I understand correctly, the minimum distance interpretation is narrower than some of the wording suggest. Theorem 1 minimizes over a point dependent admissible cluster that must contain the prescribed normal foot and satisfy a pairwise spacelike condition, it doesn't establish a global minimum over the whole decision section.
  3. QMLR combines two nonlinear scores with a learned gate. Comparisons against a linear head don't determine whether improvements require the geometric construction or could also be obtained with a matched nonlinear head.
Questions for authors

Is there any application of your results to any hyperbolic isometric model?

3 comments
AuthorsOpenReview ↗

Response to Reviewer hS95 - Part 1

We thank Reviewer hS95 for the valuable comments.

W1. Pairwise condition for attention

  • Conditional guarantee and empirical diagnostics. Thank you for raising this point. The exponential map maps individual tokens onto the pseudo-hyperboloid, but does not guarantee the all-pairs condition. This condition is sufficient for both the attention aggregate and the residual interpolation to remain timelike. Both intermediate vectors can still be timelike when some token pairs violate the condition. Table 9 in Appendix B.6 reports the following diagnostics on held-out MI and ERN trials, counting each unordered pair of distinct tokens once.

    DatasetAdmissible token pairsFlagged aggregatesFlagged residuals
    MI93.34%0.29%1.27%
    ERN96.25%0.01%0.26%

    A token pair is admissible if ⟨hj,hk⟩Q≤β\langle h_j,h_k\rangle_{\mathcal Q}\leq\beta. An aggregate or residual is flagged if its squared pseudo-norm is nonnegative or nonfinite before stabilization.

  • Stabilization: As shown in Algorithm 1, the forward pass applies the reconstruction in Eq. (24) after attention aggregation and residual interpolation. For a finite intermediate vector v=(vt,vs)v=(v_t,v_s) with vt≠0v_t\neq 0, this step preserves the direction of vtv_t and sets its magnitude using the spatial component s′s'.

\left( \sqrt{-\beta+\|s'\|_2^2}\, \frac{v_t}{\|v_t\|_2}, \;s' \right).$$ The reconstructed vector has squared pseudo-norm $\beta$ in exact arithmetic. Because this step uses $\|v_t\|_2$ for normalization, it also applies to finite vectors with zero or positive squared pseudo-norm, provided $v_t\neq 0$. All inspected outputs were finite after reconstruction. ### W2. Admissible domain of Theorem 1 Thank you for pointing out this ambiguity. Theorem 1 establishes a minimum of the spacelike geodesic length over the point-dependent admissible set $\mathcal A_k^s(p)$ in Eq. (11). We have revised Section 4.3.2 and Appendix C.4 to make this restriction explicit. The set $\mathcal A_k^s(p)$ contains the prescribed normal foot $q^\star$ and satisfies the stated pairwise spacelike condition. The same $q^\star$ attains the minimum over every such set, so the minimum value is independent of the choice of admissible set. The classifier can therefore evaluate the closed-form score directly. To see this, let $a=\tilde a_k^s$, $z=\langle p,a\rangle_{\mathcal Q}$, and $\lambda=\sqrt{1+z^2/(-\beta)}$, so that $p=\lambda q^\star+za$. For every $q\in\mathcal A_k^s(p)$, we have $\langle a,q\rangle_{\mathcal Q}=0$ because $q$ lies in the decision section. The pairwise condition applied to $q^\star$ and $q$ then gives $$\frac{\langle p,q\rangle_{\mathcal Q}}{\beta}=\lambda\frac{\langle q^\star,q\rangle_{\mathcal Q}}{\beta} \geq\lambda.$$ Equality holds at $q=q^\star$. By monotonicity of $\operatorname{arcosh}$, the infimum in Eq. (11) is attained at $q^\star$ over the stated admissible domain.
AuthorsOpenReview ↗

Response to Reviewer hS95 - Part 2

W3. Geometric scoring, classifier size, and efficiency

Thank you for raising this point. We agree that comparison with a linear head alone cannot disentangle the effect of QMLR’s geometric construction from that of generic nonlinearity.

  • Geometric score: The spacelike score is derived from a signed point-to-section distance, with the asinh⁡\operatorname{asinh} term arising from the pseudo-hyperbolic distance expression. The timelike branch applies the same scalar transformation to a timelike normal coordinate. Both branches share ak,bka_k,b_k, and their fusion introduces a single learned scalar gate. Appendix C.5 explains the zero set, sign, and local first-order behavior of the timelike surrogate.

  • Classifier size: Table 5 shows that QMLR has a parameter count close to those of MLR and several nonlinear manifold classifiers. These comparisons put QMLR’s parameter count in context.

    MethodCIFAR-10 TimeCIFAR-10 ParamsCIFAR-100 TimeCIFAR-100 Params
    MLR14.25.13K14.151.30K
    PMLR16.35.14K33.851.40K
    PBMLR17.110.24K37.9102.40K
    BMLR-P14.55.14K14.551.40K
    LMLR14.55.13K14.551.30K
    BMLR-L14.45.14K14.351.40K
    QMLR15.05.15K14.951.50K

    Time: seconds per epoch. Params: classifier parameter count.

  • Measured efficiency: We profiled the QGCN2 classification paths for QMLR and the logmap+linear baseline [c], excluding the shared encoder. Compared with the baseline, QMLR uses one additional parameter and has lower inference and forward–backward latency on all four tasks. Peak memory is lower on Cora, Airport, and CiteSeer, whereas it is higher on Arxiv. Appendix D.3.4 provides the profiling details.

    DatasetReadoutParamsInference (ms)Fwd+bwd (ms)Inference peak memory (MiB)Fwd+bwd peak memory (MiB)
    Coralog+Linear1334.0749.87110.59419.460
    CoraQMLR1340.7982.9018.99617.416
    Airportlog+Linear724.1959.10910.90919.829
    AirportQMLR730.8102.8828.82917.220
    CiteSeerlog+Linear1084.1469.20710.99919.978
    CiteSeerQMLR1090.7842.8959.06017.494
    Arxivlog+Linear7206.68220.814158.540233.576
    ArxivQMLR7213.96414.456255.104367.233
AuthorsOpenReview ↗

Response to Reviewer hS95 - Part 3

Q1. Applicability to isometric hyperbolic models

The spacelike branch applies directly to Lorentz embeddings as a special case. For multiple timelike directions, the full pseudo-hyperbolic construction is not isometric to standard hyperbolic space. We explain it from theory and experiment.

  • Metric signature: With multiple ambient timelike coordinates, the pseudo-hyperboloid has an indefinite induced metric, whereas standard hyperbolic space has a positive-definite Riemannian metric. Isometries preserve metric signature, so the general construction cannot be transformed by an isometric change of coordinates.

    For a direct comparison at the same intrinsic dimension d=t+sd=t+s and scale β<0\beta<0, the two spaces are

Lβd={x∈R1,d:−x02+∑i=1dxi2=β,x0>0}.\mathcal L_\beta^{d}=\left\lbrace x\in\mathbb R^{1,d}:-x_0^2+\sum_{i=1}^{d}x_i^2=\beta,\quad x_0>0\right\rbrace.

Qβt,s={x∈Rt+1,s:−∑i=0txi2+∑j=t+1t+sxj2=β}.\mathcal Q_\beta^{t,s}=\left\lbrace x\in\mathbb R^{t+1,s}:-\sum_{i=0}^{t}x_i^2+\sum_{j=t+1}^{t+s}x_j^2=\beta\right\rbrace.

  • Lorentz specialization: With one ambient timelike coordinate (t=0t=0), the positive-time sheet of Qβ0,s\mathcal Q_\beta^{0,s} is the Lorentz hyperboloid. On this sheet, the spacelike branch reduces to Lorentz MLR and applies directly to Lorentz embeddings [a, Section 4.3].

  • Experimental comparison: In Appendix B.5.1, we freeze all parameters and running statistics of an ILNN [b] pretrained on CIFAR-10 and train classification heads on STL-10. For QMLR, we first map the frozen Lorentz embeddings to tangent-space representations and then use the exponential map to project them onto the pseudo-hyperboloid.

    HeadTest accuracy (%)NLLBrier score
    Lorentz MLR78.41±0.0978.41\pm0.090.63730.3000
    Tangent+QMLR77.87±0.1577.87\pm0.150.65090.3079
    Tangent+Linear77.83±0.0377.83\pm0.03——

    In this table, Negative Log-Likelihood (NLL) measures the negative log-probability assigned to the true class, while the Brier score measures the squared difference between predicted class probabilities and one-hot labels. Lower values indicate better probabilistic predictions for both metrics. This experiment compares two ways of classifying the same frozen Lorentz features: applying Lorentz MLR directly or mapping the features to the pseudo-hyperboloid and applying QMLR. On STL-10, the transfer pipeline with QMLR achieved 77.87% accuracy, 0.54% below the native Lorentz MLR pipeline, with slightly higher NLL and Brier score.

  • Transfer interpretation and future work: When multiple timelike directions induce an indefinite metric, the pseudo-hyperboloid and standard hyperbolic space have different metric signatures, which precludes an isometry between them. Mapping features through tangent-space provides a practical way to transform pretrained Lorentz embeddings to the pseudo-hyperboloid. In our frozen-feature experiment, this indirect pipeline with QMLR achieved 77.87% accuracy, 0.54% below the native Lorentz MLR pipeline. A direction for future work is to develop a direct embedding mapping that could be capable of preserving specific geometric relationships.

References

Reviewer_8Sw7OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes QMLR for classification on pseudo-hyperbolic manifolds with multiple timelike directions. It combines a spacelike decision score, with a minimum-distance interpretation on a restricted admissible set, and a timelike surrogate through a learned gate. The paper also introduces an attention module for EEG decoding and evaluates the approach on graph, EEG, image and genomic tasks.

Critical strengths

The proposed classifier works directly on pseudo-hyperbolic representations, rather than mapping them to the tangent space and applying a Euclidean linear classifier.

The authors derive a closed-form score for the spacelike branch using the metric-preserving generalised Lorentz boost. They state the assumptions for its distance interpretation and use a surrogate for the timelike branch.

The graph, image and genomic experiments use the same backbone architecture within each task to compare the classifiers. Figure 3 shows higher mean F1 when the two branches are combined, and Table 3 shows what the attention module adds.

Critical weaknesses

Please check the questions.

Questions for authors
  1. Section 4.2 assumes ⟨h_j,h_k⟩_Q ≤ β for every pair of attention inputs, but I could not find how this condition is enforced. The exponential map does not guarantee it when there are multiple timelike directions. Valid manifold inputs can lead to an aggregate or residual with zero or positive squared pseudo-norm. In these cases, Eq. 6 is undefined or returns a point outside the manifold. The proof in Appendix B.6 is correct under the assumption. Please explain how the implementation prevents or handles these cases.

  2. I am not sure which metric is used for SSVEP. Table 2 and Appendix D state AUC, but several baseline values exactly match the accuracies reported in MAtt and GDLNet. Is Q-Net’s 67.91 an accuracy or an AUC? Please clarify this and confirm that all methods are compared using the same metric.

  3. I could not recover the Euclidean MLR score from the general parameterisation. For example, take t = s = 1, a_t = (0,1), a_s = 1 and a nonzero bias b. At the canonical origin, Eq. 14 gives a spacelike score of zero for every β, while the Euclidean score at x = 0 is b. Does Eq. 39 require an additional assumption? If so, please state it in Appendix C.7.

  4. I find Figure 4 difficult to interpret because the saliency method is not explained. The paper only says “gradient-based”. Please specify the attribution method and target output, how the gradients were treated, which trials and subjects were selected, whether individual maps or averages were plotted, and how they were normalised. It is also unclear whether the patterns are consistent across trials or subjects. Without this information, I cannot assess how well the figure supports the interpretation of what the model has learned.

  5. I found it difficult to build an intuition for pseudo-hyperbolic geometry and why it is used here. Section 3 gives the definitions, but I still do not see clearly how additional timelike dimensions help model mixed hierarchical and cyclic structures, as suggested in the introduction and Figure 1. A concrete comparison with the standard Lorentz model would help explain what these dimensions allow and when they are useful.

  6. CorAtt (Hu et al., IJCAI 2025) is a relevant baseline for Table 2, as it uses manifold attention on the same three benchmarks with a closely matching protocol. Its reported MI accuracy is 75.56, compared with 74.86 for Q-Net. Please include this baseline and use a common metric for the SSVEP comparison. The ERN training, validation and test split also needs to be specified.

  7. On ERN in Table 3, the linear head with attention reaches 82.46, compared with 82.62 for QMLR. Please explain what QMLR adds beyond the attention module in this setting. It is also unclear how the tokens are pooled into the 65-dimensional representation in Table 7 and whether this representation is projected back onto the manifold before classification. Please report the attention temperature and explain how β and the signature were selected.

  8. The LTR Copia results for LMLR and BMLR-L are much lower than those in Chen et al.’s Table 5: 58.26 and 55.73 here versus 73.01 and 75.86, respectively. What explains this difference?

3 comments
AuthorsOpenReview ↗

Response to Reviewer 8Sw7 - Part 1

We thank Reviewer 8Sw7 for the valuable comments.

Q1. Pairwise admissibility and projection in Q-Net

Thank you for raising this point. The exponential map places each token on the pseudo-hyperboloid. For a general set of inputs with multiple timelike dimensions, manifold membership alone does not guarantee the pairwise inequality ⟨hj(0),hk(0)⟩Q≤β\langle h_j^{(0)},h_k^{(0)}\rangle_{\mathcal Q}\leq\beta.

  • Conditional guarantee and empirical diagnostics. The all-pairs condition is sufficient for attention aggregation and residual interpolation to remain timelike, so the radial projection in Eq. (6) well-defined. An aggregate or residual can still be timelike when some token pairs violate the inequality. Table 9 in Appendix B.6 reports the following results on held-out trials.

    DatasetsAdmissible token pairsFlagged aggregatesFlagged residuals
    MI93.34%0.29%1.27%
    ERN96.25%0.01%0.26%

    Admissible pairs satisfy ⟨hj,hk⟩Q≤β\langle h_j,h_k\rangle_{\mathcal Q}\leq\beta. Flagged aggregates and residuals have nonnegative or nonfinite squared pseudo-norms before reconstruction.

  • Numerical stabilization. The diagnostics show that some intermediate vectors fall outside the domain of radial normalization. To handle these cases, the forward pass applies the numerical reconstruction in Eq. (24) for intermediate vectors after both attention aggregation and residual interpolation, as shown in Algorithm 1. For a finite intermediate vector v=(vt,vs)v=(v_t,v_s) with vt≠0v_t\neq 0, this step preserves the direction of vtv_t and determines its magnitude from the spatial component s′s' used in the reconstruction.

\left( \sqrt{-\beta+\|s'\|_2^2}\, \frac{v_t}{\|v_t\|_2}, \;s' \right).$$ The reconstructed vector has squared pseudo-norm $\beta$ in exact arithmetic. Because the normalization uses $\|v_t\|_2$, the same step applies to finite vectors with zero or positive squared pseudo-norm whenever $\|v_t\|\neq 0$. All inspected outputs were finite after reconstruction, as a numerical stability strategy for handling outliers. ### Q2. SSVEP evaluation metric Thank you for pointing out this labeling error. The reported result of Q-Net on SSVEP $(67.91\pm0.95)$ is **accuracy (%)**. All methods in Table 2 are compared using the same metric within each dataset. We report accuracy (%) for MI and SSVEP and ROC-AUC for ERN. We have corrected this typo and clarified the evaluation metrics in the caption of Table 2, Section 5.2, and Appendix D.1 of the revised manuscript. ### Q3. Euclidean MLR recovery in the flat limit Thank you for pointing out this missing assumption. The reviewer’s example shows that Euclidean recovery does not hold for arbitrary temporal orientations. Our previous statement omits the required canonical temporal alignment. Appendix C.7 now states this condition and gives the corresponding local coordinates. - **Reference alignment:** Let $R=\sqrt{-\beta}$ and let $e_0$ be the unit temporal direction defining the canonical origin $o_R=(Re_0,0)$. Write $v_k=\hat a_{k,t}$ and $w_k=\hat a_{k,s}$ for the normalized temporal and spatial directions of class $K$. To recover the original Euclidean bias, we set $v_k=e_0$ for every class. - **Local coordinates:** For fixed $(x_t,x_s)$ with $x_t\perp e_0$, and consider the manifold point $p_R=(P_t(R),p_s(R))$ given by $$p_t(R)=\sqrt{R^2+\|x_s\|_2^2-\|x_t\|_2^2}\,e_0+x_t, \qquad p_s(R)=x_s.$$ As $R\to \infty$, we have $p_t(R)=Re_0+x_t+O(R^{-1})$. - **Flat-limit expansion:** With $\phi_k=-\bar b_k/R$ and and the classifier parameters fixed, the inner projection in Eq. (14) satisfies $$\begin{aligned} z_k^s(p_R) &=\cosh(\phi_k)\langle p_s(R),w_k\rangle -\sinh(\phi_k)\langle p_t(R),v_k\rangle \\ &=\langle x_s,w_k\rangle +\bar b_k\langle e_0,v_k\rangle +O(R^{-1}). \end{aligned}$$ Since $R\,\operatorname{asinh}(z/R)=z+O(R^{-2})$ for bounded $z$, the spacelike logit converges to $$\|a_k\|_2\langle x_s,w_k\rangle +\|a_k\|_2\bar b_k\langle e_0,v_k\rangle.$$ - **Euclidean recovery and the example:** Under $v_k=e_0$ for every class, the limit is the Euclidean MLR logit $W_k^\top x_s+b_k$, with $W_k=\|a_k\|_2w_k$ and $b_k=\|a_k\|_2\bar b_k$. For a general temporal direction, the limiting bias is $b_k<e_0,v_k>$. In the reviewer's example, $v_k=(0,1)$ is orthogonal to $e_0=(1,0)$, so both terms at the origin vanish and Eq. (14) gives a zero spacelike logit for every $\beta<0$.
AuthorsOpenReview ↗

Response to Reviewer 8Sw7 - Part 2

Q4. Gradient-based interpretation in Figure 4

Thank you for pointing out that the attribution procedure was insufficiently described. We clarify below how the maps were computed and what conclusions they support.

  • Method. Figure 4 uses input-gradient saliency [b,c], differentiating the pre-softmax true-class logit with respect to the preprocessed EEG input. For electrode cc and time point tt, the absolute gradient is Si(c,t)=∣∂fyi(xi)∂xi(c,t)∣.S_i(c,t)=\left|\frac{\partial f_{y_i}(x_i)}{\partial x_i(c,t)}\right|. We use the trained Subject 3 checkpoint and correctly classified trials from that subject's MI validation set. Panel (a) averages the maps within each MI class, and panel (b) sums each class-average map over time to obtain one value per electrode.

  • Display and scope. Each class's heatmap and electrode values are independently min-max normalized to [0,1][0,1], indicating relative saliency within each panel. Figure 4 provides a qualitative interpretation of one subject’s class-average responses. It does not establish causal relevance or consistency across trials or subjects. Appendix D.1.5 specifies the attribution and rendering procedure. Additional ERN visualizations are provided in Appendix D.1.6.

Q5. Geometric intuition for multiple timelike directions

  • Coordinate interpretation: The diffeomorphism in [d] parameterizes the pseudo-hyperboloid through sphere and Euclidean factors, while [e] gives a related parameterization with spherical and Lorentz factors. Appendix B.5 explains their angular and hierarchical interpretations, with the Lorentz comparison in Figure 5. To see the geometric intuition, write a point on the pseudo-hyperboloid as
\qquad \omega\in\mathbb S^t,\quad v\in\mathbb R^s.$$ - **Lorentz comparison:** When $t=0$, $\omega\in\mathbb S^0=\{-1,+1\}$, and selecting the positive-time sheet fixes $\omega=+1$. The representation then reduces to the standard Lorentz hyperboloid, with no continuous temporal angular variation. When $t\geq1$, $\omega$ can vary continuously on $\mathbb S^t$, adding angular degrees of freedom alongside the Lorentz hyperbolic slices described above. These directions allow the model to represent cyclic variation together with hierarchical relationships. We expand this explanation in Appendix B.5 with a visual comparison in Figure 5. ### Q6. CorAtt comparison, metrics, and ERN protocol - **CorAtt comparison:** We have added all three CorAtt variants [a] to Table 2. All methods are compared using the same metric within each dataset: accuracy (%) for MI and SSVEP, and ROC-AUC (%) for ERN. It can be seen that Q-Net is below CorAtt-MIX by 0.70% and 0.36% on MI and SSVEP and exceeds it by **3.58%** on ERN. | Method | MI | SSVEP | ERN | | :--- | ---: | ---: | ---: | | CorAtt-OLM | $75.01\pm2.78$ | $67.39\pm3.22$ | $78.78\pm3.40$ | | CorAtt-LSM | $74.47\pm2.43$ | $67.74\pm2.44$ | $78.63\pm3.31$ | | CorAtt-MIX | $\mathbf{75.56\pm1.58}$ | $\mathbf{68.27\pm2.50}$ | $79.04\pm2.91$ | | **Q-Net** | $74.86\pm1.58$ | $67.91\pm0.95$ | $\mathbf{82.62\pm0.54}$ | - **Additional benchmarks:** Appendix D.1.4 and Table 14 report balanced accuracy (%) on BNCI2015-001, BCICIV2b, and Zhou2016 under a subject-specific leave-one-session-out protocol. Q-Net achieves the highest mean balanced accuracy on all three datasets, exceeding the best CorAtt variant on each by **2.43%**, **1.62%**, and **6.01%**, respectively. | Method | BNCI2015-001 | BCICIV2b | Zhou2016 | | :--- | ---: | ---: | ---: | | SPDNet | $68.52\pm11.49$ | $68.24\pm8.98$ | $72.15\pm4.77$ | | SPDNetBN | $75.15\pm4.17$ | $67.95\pm7.91$ | $76.15\pm8.55$ | | MAtt | $80.80\pm14.80$ | $72.96\pm9.61$ | $82.37\pm4.48$ | | GDLNet | $76.90\pm13.60$ | $73.24\pm8.20$ | $86.51\pm3.27$ | | HEEGNet | $76.19\pm7.84$ | $75.61\pm7.87$ | $82.18\pm11.08$ | | CorAtt-OLM | $79.55\pm4.76$ | $73.90\pm8.69$ | $81.38\pm5.55$ | | CorAtt-LSM | $80.12\pm4.91$ | $74.00\pm9.17$ | $81.11\pm4.13$ | | CorAtt-MIX | $78.87\pm5.12$ | $73.92\pm8.81$ | $81.04\pm5.39$ | | **Q-Net** | $\mathbf{82.55\pm7.00}$ | $\mathbf{75.62\pm8.65}$ | $\mathbf{87.39\pm3.97}$ | - **ERN allocation:** Our implementation follows the trial allocation described alongside CorAtt: trials 1-240 for training, 181-240 for validation, and 241-340 for testing for each of the 16 subjects. Appendix D.1.1 and Table 11 specify the allocations for all three benchmarks.
AuthorsOpenReview ↗

Response to Reviewer 8Sw7 - Part 3

Q7. ERN contribution, pooling, and hyperparameters

  • Contribution on ERN: With attention fixed, QMLR increases mean AUC from 82.46 to 82.62, while attention accounts for the main improvement. The larger head differences on MI and SSVEP indicate that QMLR's contribution depends on the task and learned representation.

  • Pooling: The encoder produces 64-dimensional features for each token. Before the exponential map, we prepend a zero-valued coordinate to form a tangent vector at the canonical origin, yielding a 65-dimensional ambient representation after mapping. Attention and residual refinement preserve this coordinate dimension. We then average the refined tokens in the ambient space. When the pooled vector is timelike, the radial projection in Eq. (6) maps it back onto the pseudo-hyperboloid before QMLR classification:

\qquad p=\Pi_{\mathcal Q}(\bar h).$$ Pooling reduces the token sequence to a single 65-dimensional representation. - **Hyperparameters.** The learned multiplier is $\tau=\operatorname{softplus}(\eta)+10^{-6}$, initialized with $\eta=1$, giving $\tau\approx1.3133$. The curvature scale $\beta$ and signature $(t,s)$ are selected by validation-set grid search. Appendix D.1.3 and Table 12 report the selected settings. ### Q8. LTR Copia results under two experimental protocols - **Experimental setup:** Thank you for pointing out the discrepancy with Chen et al. [g, Table 5]. The LMLR and BMLR-L scores in our original Table 6 were obtained using the Euclidean CNN setup of [f]. Chen et al. used a different experimental protocol. Under the [f] protocol, we take the Euclidean MLR result from [f] and evaluate PMLR, LMLR, BMLR-L, and QMLR using the same feature extractor architecture and training-evaluation settings, varying only the classification head. - **Results under both protocols.** For clarity, we have additionally conducted classification experiments using the experiemntal setup reported by Chen et al. The table below presents the results under both protocols and identifies the values taken from prior work. | Method | Protocol in [f], MCC (%)| Protocol in [g], MCC (%)| | :--- | ---: | ---: | | MLR | $54.73\pm1.45$ [f] | $73.92\pm0.72$ | | PMLR | $55.84\pm3.14$ | $75.34\pm1.02$ [g] | | LMLR | $58.26\pm1.80$ | $73.01\pm1.07$ [g]| | BMLR-L | $55.73\pm2.79$ | $75.86\pm1.52$ [g]| | QMLR | $\mathbf{58.28\pm3.10}$ | $\mathbf{77.09\pm0.98}$ | The results of Protocol-[f] column reproduces our original Table 6. In the Protocol-[g] column, the PMLR, LMLR, and BMLR-L results are from [g, Table 5], and the MLR and QMLR results are from our new runs. It can be found that QMLR achieves the highest mean MCC among the classifiers shown under both protocols. Furthermore, we would like to emphasize that the experimental comparisons conducted under each protocol is fair. ### References - [a] Hu, C., et al. (2025). [A Correlation Manifold Self-Attention Network for EEG Decoding](https://www.ijcai.org/proceedings/2025/0598.pdf). IJCAI. - [b] Simonyan, K., Vedaldi, A., and Zisserman, A. (2014). [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://www.robots.ox.ac.uk/~vgg/publications/2014/Simonyan14a/). ICLR Workshop. - [c] Farahat, A., et al. (2019). [Convolutional neural networks for decoding of covert attention focus and saliency maps for EEG feature visualization](https://doi.org/10.1088/1741-2552/ab3bb4). Journal of Neural Engineering, 16(6), 066010. - [d] Xiong, B., et al. (2022). [Pseudo-Riemannian Graph Convolutional Networks](https://proceedings.neurips.cc/paper_files/paper/2022/hash/16c628ab12dc4caca8e7712affa6c767-Abstract-Conference.html). NeurIPS. - [e] Le, V. Q., and Ta, V. C. (2025). [Pseudo-Riemannian Graph Transformer](https://proceedings.neurips.cc/paper_files/paper/2025/hash/17a1a1439421f1837e10cd612bf92861-Abstract-Conference.html). NeurIPS. - [f] Khan, R. R., Chlenski, P., and Pe'er, I. (2025). [Hyperbolic Genome Embeddings](https://proceedings.iclr.cc/paper_files/paper/2025/hash/b63ad8c24354b0e5bcb7aea16490beab-Abstract-Conference.html). ICLR. - [g] Chen, Z., Schölkopf, B., and Sebe, N. (2026). [Hyperbolic Busemann Neural Networks](https://openaccess.thecvf.com/content/CVPR2026/papers/Chen_Hyperbolic_Busemann_Neural_Networks_CVPR_2026_paper.pdf). CVPR.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.