- Rating
- 3(+1)
- Confidence
- 2(−1)
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces SPD-MetaFormer, an attention-free architecture built on uniformly weighted Fréchet aggregation under log-Euclidean geometry. Its backbone design combines geodesic residual updates and spectral feedforward transformations while retaining memory tokens for a learned weighted readout. The motivation, based on the observation that replacing these weights with uniform weights during both training and evaluation has little effect on mean predictive performance, is clear. The model design is incremental; the proposed model largely combines MetaFormer and SPD neural networks. Results on three EEG benchmarks: SPD-MetaFormer matches or improves upon the performance of strong Euclidean and manifold baselines.
Critical strengths
S1: The paper asks a motivated question about whether adaptive weighting is actually responsible for the gains of existing SPD manifold-attention models
S2: The paper is well written and easy to follow; the experimental analysis part is complex.
Critical weaknesses
W1: The motivation is that replacing these weights with uniform weights during both training and evaluation has little effect on mean predictive performance, but SPDMeta has many architectural designs that are different from Matt and GBWAtt. Given the performance gain is marginal, it is hard to interpret whether it is the uniform mixing that provides gains.
W2: The model novelty is moderate, as the proposed model largely combines MetaFormer and SPD neural networks. The main EEG comparison against prior methods relies on published numbers rather than matched reproductions, making the marginal improvements difficult to interpret.
W3: The experiment's SOTA comparison is not solid. In the manifold literature, it does not compare recent advances in hyperbolic batch normalization for EEG decoding [1] and recent high-impact EEG foundation models [2][3].
[1] Li et al. "HEEGNet: Hyperbolic embeddings for EEG." International Conference on Learning Representations. Vol. 2026. 2026.
[2] Jiang, Wei-Bang, Liming Zhao, and Bao-Liang Lu. "Large brain model for learning generic representations with tremendous EEG data in BCI." International Conference on Learning Representations. Vol. 2024. 2024.
[3] El Ouahidi, Yassine, et al. "REVE: A foundation model for EEG-adapting to any setup with large-scale pretraining on 25,000 subjects." Advances in Neural Information Processing Systems 38 (2026): 22541-22577.
Questions for authors
Can you justify whether the gain is from uniform mixing, given that there are differences between architectures?
What are SPDMetaFormer's results compared with SOTA models in W3?
Why do you investigate adaptive weighting only for SPD-manifold models? Would we observe a similar phenomenon for any attention-based EEG decoding models?
4 comments
Response to Reviewer CnkA (2/2): W2; W3&Q2; Q3
W2.
Table 1 follows prior work on these benchmarks (e.g., Jin et al., 2026) in comparing with published scores on the same splits. To verify this comparison, App. H (Table 15) adds a matched comparison with MAtt and GBWAtt, the original SPD-attention model and the strongest baseline of Table 1. All three models share the same data splits, the same tuning budget of 10 configurations, and validation-only selection. The matched results lead to the same conclusion as Table 1, and the gains are not marginal where they matter: SPD-MetaFormer is significantly ahead of all baselines on SSVEP, by 3.8–5.7 points, and of MAtt on ERN, by 3.1–3.3 AUC points ( < 0.001). It has the highest mean on MI and is on par with GBWAtt on ERN, and no baseline is significantly better on any dataset. Regarding novelty, please see point 2 under W1.
W3&Q2.
SPD-MetaFormer obtains the highest mean on all three datasets against HEEGNet [1], LaBraM [2] and REVE [3], and also against CBraMod [4], another widely used EEG foundation model.
Table R1. SPD-MetaFormer against HEEGNet and EEG foundation models (10 seeds; App. J, Table 17 in the paper). Accuracy (%) for MI/SSVEP, AUC × 100 for ERN, as mean ± SD across runs (averaged over subjects). *: significantly below SPD-MetaFormer (paired t-test and Wilcoxon test, < 0.05).
| Model | MI (acc.) | SSVEP (acc.) | ERN (AUC) |
|---|---|---|---|
| HEEGNet [1], session-adaptive | 65.66 ± 6.98* | 55.15 ± 6.98* | 82.02 ± 2.34 |
| HEEGNet [1], inductive | 61.96 ± 5.14* | 52.15 ± 6.73* | 81.73 ± 2.40 |
| LaBraM-Base [2] | 49.17 ± 3.51* | 41.09 ± 4.77* | 66.62 ± 5.56* |
| REVE-Base [3] | 66.68 ± 2.58* | 34.09 ± 4.59* | 73.77 ± 2.69* |
| CBraMod [4] | 51.63 ± 2.52* | 39.19 ± 4.25* | 73.09 ± 2.65* |
| SPD-MetaFormer | 75.44 ± 1.69 | 70.79 ± 2.17 | 82.53 ± 1.85 |
All baselines are trained under the protocol of Table 1 (one model per subject), using their official code and, for the foundation models, the released pretrained weights. Each is tuned on validation data, starting from its published recipe. The low SSVEP scores of the foundation models are consistent with a linear probe of their frozen features, which reaches only 21.7–25.5% on this five-class task (chance: 20%; App. J). Kontras et al.[5] similarly report that frozen REVE and NeuroLM "remain near chance level" on other SSVEP datasets. This broader comparison further supports the paper's finding.
Q3.
Why SPD-manifold models. Recent work has proposed more sophisticated attention on the SPD manifold for EEG decoding (Pan et al., 2022; Hu et al., 2025; Jin et al., 2026) and has credited the improvements to it, whereas the contribution of the learned weights had not been examined. We therefore focus on this line of work, where the question is most directly tied to published claims.
A similar finding outside the manifold. A concurrent ICLR submission, Does Attention Matter in Motor Imagery EEG Decoders? Diagnosing and Repairing Attention Failure [6], examines eight Euclidean convolution–attention decoders on four motor-imagery datasets and finds that their predictions do not depend consistently on the attention weights. Our MI results agree with it, and our results on two further paradigms, SSVEP and ERN, give additional support (Tables 4–5).
Future direction. Whether this holds for attention-based EEG decoders in general is beyond the scope of this paper, but it is a question we would very much like to investigate in future research. Our two controls, the entropy of the weights and the locked-uniform comparison, do not depend on the geometry, so they carry over directly to other decoders, and we plan to take this up as our next step.
Thank you again for the careful and constructive review, which has made the paper clearer and its evidence much broader. We hope that these responses address your concerns, and we would be grateful if you would consider raising your score. We are glad to clarify any remaining point.
References
[4] Wang, Jiquan, et al. "CBraMod: A criss-cross brain foundation model for EEG decoding." International Conference on Learning Representations, 2025.
[5] Kontras, Konstantinos, et al. "NeuroAtlas: Benchmarking foundation models for clinical EEG and brain-computer interfaces." arXiv:2605.14698, 2026.
[6] Anonymous. "Does Attention Matter in Motor Imagery EEG Decoders? Diagnosing and Repairing Attention Failure." Under review at ICLR 2027. https://openreview.net/forum?id=VcgYbGuRnf
I thank the author for the detailed rebuttal, which has addressed most of my concerns. Looking at the results, I am wondering why recent high-impact foundation models have such a large performance drop, especially on SSVEP tasks, and cbramod is usually the state-of-the-art on motor imagery datasets, but here we see a huge drop.
So far, the authors have not submitted the revised manuscript; I would also like to see how the authors have revised the manuscript based on all reviewers comments.
Response to Reviewer CnkA: W3&Q2
Thank you for the quick follow-up. We are glad most of your concerns are addressed.
The gap comes from the evaluation regime, and our results agree with published ones. Foundation models are usually fine-tuned on pooled data from many subjects, whereas Table 1, like prior manifold-attention work, trains one model per subject on a few hundred trials. App. J of the revised manuscript details the settings, and we summarize the key points here because the numbers can be confusing without them.
Independent work agrees. Bdeir et al. [7], accepted at NeurIPS 2026, fine-tune CBraMod under the same per-subject protocol and report 45.22% / 40.45% / 68.49 on MI / SSVEP / ERN (their Table 9), against our 51.63% / 39.19% / 73.09. They likewise find that foundation models "underperform on these smaller EEG datasets".
MI. On BCIC-IV-2a, the CBraMod [4] (their Table 15) reports 51.38% for CBraMod and 48.69% for LaBraM across subjects, and we obtain 51.63% and 49.17% per subject, the same level despite the different protocols.
SSVEP. These models read each channel as 1-s patches and a trial here lasts 1 s, so the pretrained temporal modelling goes unused, which Bdeir et al. [7] also note for CBraMod. A linear probe on the frozen features is accordingly near chance (21.7–25.5% vs. 20%), as in Kontras et al. [5] (NeurIPS 2026).
We have double-checked that the revised PDF is uploaded, with the fine-tuning and tuning settings of each foundation model in App. J. Could you kindly check, by clicking the pdf file sign? If it is still not visible, we will ask the PCs to resolve it.
References.
[7] Bdeir, Ahmad, et al. "LAtte: Hyperbolic Lorentz attention for joint-subject EEG classification." Advances in Neural Information Processing Systems, 2026.
Response to Reviewer CnkA (1/2): W1&Q1; W2
Thank you for your careful reading and constructive comments, and for recognizing the motivation of our study. Your comments prompted several new experiments, including a matched-tuning comparison and a comparison with HEEGNet and recent EEG foundation models. All results are in the revised paper, and we summarize the key findings here.
W1&Q1.
Thank you for the opportunity to clarify our claim: uniform mixing is not the primary source of the observed performance gains; rather, it serves as a motivating principle for our architectural design. We isolate uniform mixing from the architecture by testing it inside MAtt and GBWAtt, where only the weights change. SPD-MetaFormer then tests whether a model built on uniform mixing can be competitive. Uniform mixing is performance-neutral: it matches learned attention with a simpler and cheaper model, and the margin comes from the SPD path built around it. We examine the two parts separately:
Uniform mixing also makes the model interpretable, attention-free, and linear in the number of tokens (App. K). In the revision, we therefore present the attention diagnosis as the key finding and SPD-MetaFormer as the design that follows from it.
W2.
Table 1 follows prior work on these benchmarks (e.g., Jin et al., 2026) in comparing with published scores on the same splits. To verify this comparison, App. H (Table 15) adds a matched comparison with MAtt and GBWAtt, the original SPD-attention model and the strongest baseline of Table 1. All three models share the same data splits, the same tuning budget of 10 configurations, and validation-only selection. The matched results lead to the same conclusion as Table 1, and the gains are not marginal where they matter: SPD-MetaFormer is significantly ahead of all baselines on SSVEP, by 3.8–5.7 points, and of MAtt on ERN, by 3.1–3.3 AUC points (p < 0.001). It has the highest mean on MI and is on par with GBWAtt on ERN, and no baseline is significantly better on any dataset. Regarding novelty, please see point 2 under W1.