- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
In this paper, the authors propose TAHLP, a self-supervised pretrained Transformer for cross-montage motor-imagery EEG. Two components are claimed as contributions. (1) Hierarchical latent-prediction self-distillation: an EMA teacher encodes the unmasked trial and provides, per token, the instance-normalized average of its last K block outputs as a continuous target; the student sees the trial under two complementary token masks and predicts the target at masked positions through n=3 cumulative residual levels with a weighted cosine loss. The objective is single-stage and uses no codebook, negative pairs, or labels. (2) I3D-ERoPE: a parameter-free rotary transform on query/key pairs whose angles come from each electrode’s 3D scalp coordinate, with frequency planes assigned round-robin to the x/y/z axes, so attention depends on relative electrode displacement rather than channel index. Heterogeneous montages are handled by a shared per-patch temporal conv stem, a learned electrode-name embedding, and dynamic padding with validity masks.
Validation: pretraining on 22 public MI datasets (559 subjects, 221 h, 6.2M-parameter encoder), then subject-independent fine-tuning on four held-out MI benchmarks (PhysioNet-MI, SHU-MI, BCICIV-2a, BNCI2015001) against seven specialist decoders and six generalist EEG foundation models. Supporting experiments: a same-corpus control re-pretraining all six generalists on the 221 h corpus (Appendix G), a component ablation (Table 2), a level-weight ablation, a pretraining-budget curve on BCICIV-2a, and an attention-geometry probe (Appendix C). The paper claims state-of-the-art on all four benchmarks with under 8.8% of the smallest baseline’s pretraining data.
Critical strengths
-
I agree that the pretraining objective is the right kind of contribution for this problem, and the authors test it properly. Predicting continuous full-context latents instead of waveforms, codebook indices, or contrastive pairs addresses a real mismatch between existing EEG pretraining targets and the narrowband, nuisance-dominated structure of MI. The ablation attributes the largest single gain to this change (3.0 to 4.3 balanced-accuracy points over masked reconstruction), and the same-corpus control in Appendix G is the a very good experiment for separating objective from data budget.
-
I very much like the proposed I3D-ERoPE, as it is a parameter-free way to make attention a function of relative 3D electrode displacement, which is the property cross-montage transfer actually needs. The ablation reports a strictly larger gain than a learned coordinate embedding across all benchmarks, and the pretraining-budget curve shows that the advantage is largest in the low-data regime. The mechanism is a direct adaptation of interleaved multi-axis RoPE, so the novelty lies in the application rather than the idea, but it is well executed and, by construction, generalizes to unseen electrodes (which is a crucial thing to have).
Critical weaknesses
-
The component ablation (Table 2) is very strange and not credible as reported. The PhysioNet-MI and SHU-MI columns differ by exactly 1.27 points in all six rows (59.63/58.36, 62.63/61.36, 63.13/61.86, 64.33/63.06, 65.53/64.26, 66.30/65.03). Consequently every per-component delta is identical on the two benchmarks (3.00/3.00, 0.50/0.50, 1.70/1.70, 1.20/1.20, 0.77/0.77), and Section 3.5 reports these as independent findings. Five-seed means on two unrelated datasets with different class counts, subject pools, and montages do not produce a constant offset, or at least the chances of that are miniscule. Since the ablation is the primary evidence for both claimed contributions, this is consequential to the whole assessment. Per-seed raw results are required before the ablation can be evaluated.
-
The state-of-the-art claim does not survive a baseline the authors themselves cite, but don't compare to. REVE (El Ouahidi et al., NeurIPS 2025) reports results on PhysioNet-MI and BCICIV-2a under the same CBraMod-derived subject splits used here (the CBraMod rows match in both papers), and its BCICIV-2a balanced accuracy is roughly 7 points above TAHLP’s 57.15. REVE is cited, its positional encoding is used as an ablation arm, yet it is absent from Table 1. This falsifies “state of the art on all four benchmarks” and weakens the paradigm-specialization thesis, since a generalist beats the MI specialist on an MI benchmark. The remaining margins are also small relative to the evaluation noise: BCICIV-2a and BNCI2015001 test on 2 and 3 subjects respectively, the 0.58-point lead over CSBrain on BCICIV-2a is within one reported std, and no significance test is given.
Questions for authors
-
Please provide per-seed results for every row of Table 2 and explain the constant 1.27-point offset between the PhysioNet-MI and SHU-MI columns. I find it hard to believe that such runs would produce such a perfect offset.
-
Why is REVE omitted from Table 1 given that it is cited, used in the ablation, and reports numbers under the same splits? Please add it, or state what in the protocol makes its published numbers incomparable. You could also compare to topology-agnostic baselines such as LUNA (also cited in the appendix).
-
The paper never reports the 6.2M-parameter architecture trained from scratch on each downstream set (no pretraining). Table 2 starts from a reconstruction-pretrained baseline. What does pretraining contribute over no pretraining at all? This is a super important baseline to compare to and cleanly separates the effect of the pretraining from the model construction itself.
-
The anti-collapse mechanism (EMA teacher plus BYOL-style predictor) is never ablated. Did you try a single-encoder variant with stop-gradient only, or a regularized JEPA formulation without a teacher? If latent prediction is the contribution, the reader needs to know which part of the machinery is load-bearing; also, thoughts from the authors on using a method that maybe doesn't require stop-gradients and teacher-student networks (such as LeJEPA) would be interesting to hear.
-
Common average reference behaves very differently on a 13-channel montage than on 64 channels, so the reference is not actually common across the corpus. Was this considered, and how are datasets with bipolar or linked-ear references handled?
-
The 50/60 Hz notch is applied together with a 0.5 to 40 Hz bandpass. What does the notch contribute in this setting, I would assume that doing bandpass alone would be sufficient?