- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The submission investigates spatial inductive biases in EEG Transformers. The authors argue that because EEG sensors lie on an approximately spherical scalp surface, Euclidean chord distance and Cartesian coordinate encodings fail to capture the true intrinsic scalp metric. The paper makes two core contributions:
- Theoretical analysis of three existing spatial encoding families: proving an metric distortion bound for Euclidean chord surrogates ("Theorem 1 (O(ψ 3) metric distortion)"), an irreducible discontinuity at the poles for azimuthal spherical encodings, and frame-dependence under rotations for per-axis Cartesian encodings such as NeurIPT ("Proposition 4 (Frame dependence of per-axis encodings)").
- GeoRoPE & GDA: A geometric positional encoding framework comprising GeoRoPE (2D rotary encodings parameterized by geodesic distance) and GDA (an additive linear geodesic distance attention bias , whose induced exponential/Laplacian geodesic kernel is proven strictly positive definite on for via "Theorem 6 (Strict positive definiteness of the GDA kernel)"). The authors evaluate this framework on four downstream EEG benchmarks (Mental Arithmetic, PhysioP300, Sleep-EDFx, BCIC-IV-2A), reporting improvements over six baselines on 11 of 12 metrics.
Critical strengths
-
Rigorous Differential Geometry Foundation (Section 4, Theorems 1 & 6): The paper provides a principled mathematical characterization of spatial encodings on spherical manifolds. In "Theorem 1 (O(ψ 3) metric distortion)", the authors prove that Euclidean chord distance incurs an metric distortion that compresses long-range scalp separations. In Section 4.3 ("Theorem 6 (Strict positive definiteness of the GDA kernel)"), the authors prove that linear geodesic distance bias induces a strictly positive-definite exponential kernel () on , using a random-hyperplane embedding with , which reduces to a Gaussian kernel matrix on distinct points (Wendland, 2005). Closely related positive-definiteness results on are classical (e.g., Schoenberg, 1942; Gneiting, 2013), and the paper would benefit from positioning Theorem 6 against them; the link to the attention bias is nonetheless clearly made and, on my reading of Appendix A, correct.
-
Frame-Invariance and Topological Well-Posedness (Section 3.2 & Section 4.2): By constructing rotary positional encodings directly from scalar geodesic distance, GeoRoPE eliminates coordinate-frame sensitivity. In "Proposition 4 (Frame dependence of per-axis encodings)", the authors prove that per-axis Cartesian encodings are frame-dependent: for a fixed electrode pair and a rotation about the -axis, the rotary per-axis analogue changes the logit by for some admissible representations, and the similarity induced by NeurIPT's absolute per-axis embedding changes by at least . GeoRoPE avoids this frame dependence and eliminates polar singularities.
Critical weaknesses
- Missing Component and Metric Ablations and Non-Causal Single-Task Visualizations (Sections 5.1–5.3, Table 2, Figures 2–3): The empirical validation does not isolate the causal factors behind the headline claim that "downstream tasks, the resulting model improves over existing baselines on 11 of 12 metrics.":
- GeoRoPE Parameterization and Computational Cost: Standard RoPE (Su et al., 2024) factorizes into per-token rotations with cost. GeoRoPE parameterizes rotation via intrinsic pairwise distance , which, as the authors note (Section 3.2), cannot be factorized into per-token rotations; the logits instead require two block Gram matrices per frequency pair, weighted entrywise by and . The paper describes this as "a constant-factor overhead over standard attention" but reports no runtime or memory measurements, and it is unclear whether the computation can be fused into FlashAttention-style kernels. Furthermore, because is invariant under all of , reflections included, the positional encoding by itself does not distinguish mirror-symmetric electrodes (e.g., C3 vs. C4 relative to Cz). The authors acknowledge this and state that lateralized information reaches the network "through channel identity and signal content rather than through the spatial encoding" (Section 4), but it is unclear which model component carries channel identity (see Question 4).
- GeoRoPE vs. GDA: The method combines rotary modulation (Eq. 3) and additive distance bias (Eq. 4). The paper does not report an ablation isolating GeoRoPE alone versus GDA alone versus the combined model. It remains unknown whether rotary encoding contributes beyond simple distance decay.
- Geodesic vs. Euclidean Chord: Despite Theorem 1's chord distortion bound, there is no experiment comparing geodesic distance against Euclidean chord distance on the exact same architecture. Without this control, the central thesis—that geodesic metric fidelity drives downstream gains—is not directly tested.
- Rotational Stress Test & Non-Causal Single-Task Visualizations: Despite Proposition 4 proving NeurIPT's frame dependence under rotation, the paper does not report an evaluation of GeoRoPE and NeurIPT under rotated frames at test time. Furthermore, visualizations in Section 5.3 (Figures 2–3) are restricted to Mental Arithmetic, and the authors explicitly concede that Figure 2's gradient attribution "does not by itself establish a causal role for the geodesic bias." and Figure 3's UMAP separation "is not evidence for a specific geometric mechanism."
- Methodological Deficiencies in Benchmark Comparisons (Section 5.1, Table 2):
- Borrowed Baseline Results & Unspecified Pretraining Regime: Table 2 caption explicitly states "Baseline results are taken from Fang et al. (2026);" rather than evaluating baselines locally. All six compared baselines (
BENDR, BIOT, LaBraM, CBraMod, EEGPT, NeurIPT) are described in Section 2.1 as self-supervised or pretrained EEG models, yet Section 5.1 does not specify whether GeoRoPE was pretrained on an upstream corpus or trained purely from scratch for 50 epochs on each downstream benchmark on a single GPU ("All experiments are conducted on a single NVIDIA RTX 4090 GPU."), nor whether the Fang et al. (2026) numbers reflect pretrained checkpoints or from-scratch training. - Omission of Variance: Although the Table 2 caption states "our results are averaged over five seeds under the same data splits and preprocessing.", Table 2 reports only mean point estimates without standard deviations across seeds. Cross-subject EEG results can vary noticeably across seeds and test subjects, so margins such as 79.09% vs. 77.57% Cohen's kappa on Sleep-EDFx are difficult to interpret without uncertainty reporting or paired significance tests.
- Missing Architecture Specifications: Transformer layer count, attention heads, hidden dimensions, and parameter counts are omitted from Section 5.1.
Score rationale
Presentation (3). The paper is well-written, logically structured, and features clear mathematical exposition. Proof sketches in Section 4 are readable, and Appendix A provides complete, unabridged proofs for all theorems and propositions. Minor naming inconsistencies exist between the title header (A Geodesic Perspective) and metadata (A Geodesic Approach), but they do not hinder comprehension.
Informativeness (2). The paper provides valuable theoretical insights for geometric deep learning in neural interfaces. In particular, the proof in Section 4.3 (specifically "Theorem 6 (Strict positive definiteness of the GDA kernel)") that the exponential (Laplacian) geodesic kernel () induced under the softmax exponential by the linear geodesic distance bias is strictly positive definite on —shown via a random-hyperplane embedding into under which becomes a squared Hilbert-space distance, so that is a Gaussian kernel matrix on distinct points—is an elegant result, contrasted with the failure of positive definiteness of the squared form for (Remark 7(i)). However, I believe Theorem 6 follows from classical results on positive definite functions on spheres (e.g., Schoenberg, 1942; Gneiting, 2013) and on geodesic exponential kernels (Feragen et al., 2015, which the paper cites only in Remark 7), and its sign-function embedding mirrors the random-hyperplane argument of Goemans & Williamson (1995). The paper should position Theorem 6 against these results; I read it as a clean application of known tools rather than a new theorem, supporting informativeness: 2.
Soundness (2). While the theoretical derivations appear sound on my reading of the proof sketches and Appendix A, the empirical methodology has significant gaps:
- Borrowed baseline numbers: Baseline numbers in Table 2 are directly copied from Fang et al. (2026) rather than evaluated in a unified local codebase.
- Missing component ablations: There are no ablation experiments separating GeoRoPE from GDA, nor comparing geodesic distance against Euclidean chord distance on the same backbone.
- Missing variance reporting: Table 2 provides only point estimates without standard deviations across seeds.
- Missing architecture specifications: Layer counts, attention heads, and parameter counts are omitted.
Questions for authors
-
[clarification] Component & Metric Ablations: Could you provide an ablation table on the four evaluated benchmarks reporting: (a) Backbone baseline without spatial encoding, (b) Backbone with Euclidean chord rotary encoding, (c) Backbone with GeoRoPE only, (d) Backbone with GDA bias only, and (e) Full model? Specifically, what proportion of the gain is attributable to GDA versus GeoRoPE?
-
[clarification] Standard Deviations & Statistical Significance: Could you provide standard deviations () across the 5 random seeds for all 12 metrics in Table 2, and report whether improvements over NeurIPT are statistically significant under a paired t-test or Wilcoxon signed-rank test?
-
[clarification] Pretraining vs. From-Scratch Regime & Architecture Parity: Was GeoRoPE trained purely from scratch for 50 epochs on each downstream dataset, or pretrained on an upstream EEG corpus? Do the borrowed baseline numbers from Fang et al. (2026) in Table 2 come from pretrained foundation-model checkpoints or from-scratch training, and what are the exact architectural hyperparameters (layers, heads, hidden dimension , parameter count) compared to NeurIPT?
-
[future work] Rotational Invariance vs. Anatomical Lateralization: Since is invariant under all transformations (including reflections), the spatial encoding by itself is unchanged when the montage is mirrored left–right or anterior–posterior, and Section 4 states that lateralized information reaches the network "through channel identity and signal content rather than through the spatial encoding." Which component supplies channel identity (e.g., a learned channel embedding), and, in motor imagery tasks (BCIC-IV-2A) that depend heavily on contralateral sensorimotor rhythms (C3 vs. C4), how does relying on channel identity rather than the spatial encoding affect cross-subject generalization?
-
[future work] Computational Overhead & FlashAttention: Because pairwise geodesic rotations cannot be factorized into per-token transformations, GeoRoPE requires computing two full Gram matrices across frequency bands (Section 3.2 describes this as "a constant-factor overhead over standard attention"). Could you report wall-clock training throughput and peak GPU memory compared to standard attention, and state whether the implementation can use fused kernels such as FlashAttention?