- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates whether EEG foundation models pretrained with masked reconstruction rely primarily on spatial interpolation between electrodes rather than learning useful EEG representations. The authors propose the Spatial Short-cut hypothesis and analyze reconstruction behavior, spatial attention, and learned representations. Their experiments reveal highly trial-invariant spatial attention across several models and examine its implications through downstream interventions. Overall, the paper addresses a valuable research question and provides useful empirical observations. However, the interpretation of trial-invariant spatial attention and its relationship with representation quality remain insufficiently supported.
Critical strengths
-
Addresses an important question about the limitations of masked reconstruction in EEG foundation models.
-
Provides a systematic analysis of reconstruction behavior, spatial attention, and learned representations.
-
Evaluates multiple EEG foundation models and datasets, with additional controls involving initialization, supervised training, and masking strategies.
-
Connects the findings to downstream performance through attention substitution, feature augmentation, and fine-tuning experiments.
Critical weaknesses
- The presentation lacks clarity, particularly regarding the objectives, interpretations, and connections between experiments.
- The interpretation of trial-invariant spatial attention is insufficient. Such patterns may reflect meaningful EEG spatial structure, and their causal relationship with representation limitations remains unclear.
- Some diagnostic experiments have limited coverage across models and datasets, with insufficient analysis of subject-level and trial-level variability.
- The representation analysis relies mainly on DE, covariance, and linear regression, which may not fully characterize the learned EEG information.
- The paper identifies potential limitations of masked reconstruction but provides limited discussion of concrete and feasible directions for addressing them.
Questions for authors
- Given the inherent spatial structure of EEG, how do the authors distinguish meaningful spatial priors from potentially restrictive trial-invariant spatial maps? Do spatial attention patterns differ across downstream tasks?
- In Figure 2, both Static Share and downstream performance increase during early training. Does this suggest that higher Static Share does not necessarily indicate limited representation quality?
- In Section 5.1, Subject Identity is more decodable than Task Class, but this may reflect inherent individual differences in EEG. Is this sufficient evidence of limited task-relevant representations? Additionally, does the comparison in Figure 3(b) account for correlations among trials from the same subject?
- Why are only DE and covariance used to evaluate neurophysiological information? Would additional EEG features or nonlinear regression models lead to different conclusions?
- Could the authors compare the final spatial attention maps of pretrained-and-fine-tuned models and models trained directly under supervision, particularly their Static Share and spatial distributions?
6 comments
Question 2 response
Question 2: In Figure 2, both Static Share and downstream performance increase during early training. Does this suggest that higher Static Share does not necessarily indicate limited representation quality?
Response: Thank you for highlighting this. A higher Static Share means that the spatial attention maps are more similar across trials. In Figure 2, Static Share and downstream linear-probe accuracy both increase during early pretraining. As you point out, their joint rise leaves open whether learning the spatial attention weights was necessary for the performance gains.
To test this, we repeated pretraining with the spatial-attention parameters frozen, either from initialisation or after epoch 1, while the rest of the encoder continued to train. We evaluated both variants at epoch 9 with the same linear-probe protocol on the frozen encoder. The results are in the new Appendix L.3 (Freezing spatial attention in training), Table 16.
Table 16. (in the paper, Appendix L.3) Linear-probe balanced accuracy (%, mean ± standard deviation over three seeds) with spatial attention trained or frozen during pretraining.
| Encoder | ISRUC | FACED | MA |
|---|---|---|---|
| Original run, epoch 1 | 61.5 ± 3.5 | 18.9 ± 1.2 | 69.3 ± 8.5 |
| Original run, epoch 9 | 66.9 ± 1.7 | 22.2 ± 0.5 | 73.7 ± 7.5 |
| Spatial attention frozen from initialisation, epoch 9 | 64.7 ± 3.2 | 22.7 ± 0.8 | 67.6 ± 5.0 |
| Spatial attention frozen from epoch 1, epoch 9 | 66.8 ± 0.9 | 21.8 ± 1.3 | 73.1 ± 8.5 |
Freezing spatial attention after epoch 1 gives scores close to those of the original run at epoch 9. Freezing it from initialisation, where attention is close to uniform and Static Share is near zero, gives comparable scores, within the spread across seeds. Downstream gains therefore occur while the spatial-attention parameters are held fixed, and with either a high or a near-zero Static Share: the two are not coupled.
A likely explanation for the gains is that pretraining makes per-electrode spectral information more linearly accessible. The decodability of differential entropy (DE) from the pooled representation is higher after pretraining than at initialisation for our encoder, ST-EEGFormer and REVE on all five datasets tested (Appendix I.2).
Table 12. (in the paper, Appendix I.2) Decodability of per-electrode DE (R² on held-out subjects) from the pooled last-layer representation, at initialisation → after pretraining.
| Model | FACED | BCI IV-2a | Mumtaz | MA | ISRUC |
|---|---|---|---|---|---|
| Ours | 0.42 → 0.68 | 0.64 → 0.86 | 0.44 → 0.73 | 0.22 → 0.52 | 0.56 → 0.85 |
| ST-EEGFormer | 0.45 → 0.64 | 0.65 → 0.78 | 0.49 → 0.67 | 0.29 → 0.42 | 0.61 → 0.81 |
| REVE | 0.23 → 0.58 | 0.00 → 0.74 | 0.00 → 0.60 | 0.00 → 0.29 | 0.09 → 0.77 |
We agree that a higher Static Share does not by itself indicate poorer representation quality. It characterises the spatial solution that masked reconstruction settles on, and we now describe it as a by-product of solving the objective rather than a driver of transfer. This is stated in Section 4.2, after the discussion of the training trajectory. These values are reported in Appendix I.2, where the initialisation columns for ST-EEGFormer and REVE have been added alongside those of our encoder.
Question 3 response
Question 3: In Section 5.1, Subject Identity is more decodable than Task Class, but this may reflect inherent individual differences in EEG. Is this sufficient evidence of limited task-relevant representations? Additionally, does the comparison in Figure 3(b) account for correlations among trials from the same subject?
Response: Thank you for raising both points. We agree that high subject decodability alone is insufficient evidence of limited task-relevant information. Hand-computed differential entropy and covariance features alone identify participants with balanced accuracy of 0.78–0.99 across MA, BCI IV-2a and ISRUC, which illustrates how strongly EEG reflects individual differences.
Does subject-specific variation make task decoding harder? To test this directly, we asked whether subtracting each participant's average representation makes task labels easier for a linear classifier to decode. For each test participant, we estimated the mean representation from half of their windows, without labels, and subtracted it from the representations of the remaining windows. Centred and uncentred performance are compared on exactly the same evaluation windows, over five subject-disjoint folds repeated three times.
Table 21 (in the paper, Appendix O.2). Balanced accuracy of a linear readout shared across subjects, before and after subtracting each test subject's mean representation, estimated on separate unlabelled calibration windows of that subject.
| Model | Readout | MA | BCI IV-2a | ISRUC |
|---|---|---|---|---|
| Ours | Shared linear readout | 0.69 | 0.39 | 0.70 |
| After label-free centring | 0.80 | 0.46 | 0.72 | |
| ST-EEGFormer | Shared linear readout | 0.69 | 0.36 | 0.70 |
| After label-free centring | 0.76 | 0.43 | 0.69 | |
| LaBraM | Shared linear readout | 0.69 | 0.29 | 0.70 |
| After label-free centring | 0.77 | 0.30 | 0.69 |
Centring improves balanced accuracy in seven of nine comparisons, by 1.5 to 11 percentage points. All three encoders improve on MA and BCI IV-2a, while ISRUC shows mixed results. This suggests that subject-specific shifts in the frozen representations can make task decoding harder for a linear classifier in some settings. We state this in Section 5.1, under "Subject identity is more decodable than task content", and give the full analysis in Appendix O.2 (Table 21).
Correlations among trials from the same subject. The original Static Share estimate pooled all windows. To check whether repeated windows from the same participants drove the result, we recomputed Static Share with one randomly selected map per participant, repeating the draw 20 times. The resulting estimates (0.86–0.88) are close to the pooled estimate on the same sample (0.86–0.87). Static Share is also high within participants (0.90–0.92) and across participant-average maps (0.94–0.95), and each participant's average map correlates strongly with the average map of the remaining participants (Pearson r ≥ 0.85, 0.96–0.97 on average). Together, these checks show that the spatial pattern remains the same when each participant contributes equally.
We added this result to Section 4.1 and describe the participant-level checks in Appendix H.5 (Participant Identity and Static Share, Table 9). The figure itself (Figure 3b of the original submission) is now Figure 9b in Appendix N.
Question 1 response
Question 1: Given the inherent spatial structure of EEG, how do the authors distinguish meaningful spatial priors from potentially restrictive trial-invariant spatial maps? Do spatial attention patterns differ across downstream tasks?
Response: Thank you for your time reviewing our paper and for raising this point. A spatial attention map that stays similar across trials can still reflect genuine EEG structure, and we do not regard it as a flaw in itself. We take a spatial prior to be useful for a downstream task if it is specific to that task and if the transferred representation depends on it. The pretrained map meets neither condition.
Same map across datasets. The pretrained encoder uses a similar pattern for combining electrode signals on every dataset. On their shared electrodes, its average maps correlate at Pearson r = 0.91–0.96 across datasets (permutation null ≤ 0.23), despite different subjects, montages and tasks. The same holds for LaBraM (r = 0.91–0.99) and ST-EEGFormer (r = 0.89–0.97; permutation null ≤ 0.27). The maps of the same architecture trained with supervision differ from one dataset to another (r = 0.16–0.24, no higher than the permutation null). This is added to Section 4.1, and the calculation is detailed in the new Appendix H.4 (Cross-Dataset Comparison).
Performance does not depend on the learned map. We ran two controls. First, we left spatial attention untrained during pretraining and compared the resulting encoder with the original run at the same epoch. Second, we shuffled which electrode signal went into each input position of the frozen encoder and trained a new linear classifier on its outputs.
Table X1. Frozen linear-probe balanced accuracy (%, mean ± standard deviation over three seeds) for the spatial attention controls. (i) The two runs are compared at the same pretraining epoch (epoch 9); from Table 16, Appendix L.3. (ii) The encoder is held fixed and a linear classifier is fitted separately for each input ordering; from Table 23, Appendix Q.3.
The trial-invariant map is thus a prior about electrode proximity, identical across tasks, that the transferred representation does not rely on. It can also be improved on where a task needs other cross-channel relationships: on FACED, replacing the frozen encoder's maps by those of a model trained on the task, with all pretrained weights unchanged, raises balanced accuracy from 21.8 to 30.5, and to 40.2 with maps taken from a fine-tuned model. Together, this indicates that the learned spatial map is not what transfers, and that task-specific routing can add to it.
We added these configurations (untrained map, permuted electrodes, supervised map swap) to Section 5.2 and Figure 4, and moved the previous downstream interventions to Appendix Q.1. The runs with untrained spatial attention are reported in Appendix L.3 (Table 16), the map swap in Appendix Q.2 (Table 22), and the electrode-permutation results for the released models in Appendix Q.3 (Table 23).