- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
EEG-GPRL combines absolute electrode-coordinate features, three-dimensional rotary attention, spatially and temporally correlated masking, and EMA-teacher latent prediction regularized with VISReg. The model is pretrained on approximately 18,546 retained hours and evaluated on nine EEG datasets using frozen representations with k-nearest-neighbor and MLP readouts. It achieves the highest reported balanced accuracy on eight of nine datasets under each protocol, although the exceptions differ between protocols. Ablations vary the prediction target, geometric encoding, VISReg, and masking strategy. The principal unresolved issues are how much the ranking depends on readout conventions and whether the reported dataset partitions establish the claimed unit of generalization.
Critical strengths
-
Frozen-representation evaluation across nine datasets with two readouts. The nine datasets cover six task families, and both kNN and trained MLP readouts are considered. Agreement across these different readouts is more informative than relying exclusively on full fine-tuning. The model does not dominate every dataset, and the paper reports those exceptions.
-
Useful ablations of the main design choices. Table 3 compares latent and waveform targets, removes geometric encoding and VISReg, and evaluates random, block, channel, and temporal masking alternatives. For the five-task aggregate, the full model reaches 52.9 kNN balanced accuracy versus 42.9 with random masking and 49.7 without geometric encoding; the MLP comparisons also favor the full recipe. These are meaningful controls, and the proposed mask is computationally well specified through its spatial and temporal covariance construction.
Critical weaknesses
-
The readout protocol leaves a consequential ambiguity in the representation-quality comparison. Appendix C applies Euclidean kNN without feature standardization or L2 normalization, so distances reflect each model's coordinate scaling and anisotropy as well as label information. The MLP protocol flattens model-specific features and sets hidden dimensions relative to their size, which does not hold classifier capacity constant across encoders. This does not make the reported pipeline scores incorrect, but weakens an interpretation as a comparison of intrinsic representation quality. Table 2 illustrates why a sensitivity check matters: on Mumtaz, CBraMod has 53.34% balanced accuracy despite 97.37% AUROC and 97.01% AUPR. That discrepancy calls for checking thresholding, calibration, and optimization before interpreting the balanced-accuracy gap as poor features. Please provide a compact, validation-selected normalization/readout sensitivity analysis, including a genuine linear probe or a matched-capacity classifier. The aim is to establish that the main ranking survives reasonable readout choices, not to optimize every baseline exhaustively.
-
The dataset partitions and uncertainty reporting do not yet make the generalization claims auditable. Table 5 lists Mumtaz training/validation/test counts of 42 MDD + 40 controls, 10 + 8, and 10 + 10, totaling 120 units, without identifying whether these are people or recordings. The linked public dataset includes multiple recording conditions, and its BIDS conversion lists 64 participants. These facts can be consistent if the table counts recordings, but then person-level grouping must be made explicit. Please provide a file-to-participant-to-split manifest establishing that recordings from one person remain together. Separately, SEED-V and SEED-VII use trial splits within sessions, whereas other tasks use subject splits; the results should not be summarized as uniformly cross-subject. Finally, kNN standard deviations over K choices describe hyperparameter sensitivity, not uncertainty over subjects or training runs. These distinctions matter especially for datasets with only two held-out test participants. A participant-level analysis and correctly labeled variability would make the evidence much more interpretable.
Questions for authors
-
What exactly do the Mumtaz split counts in Table 5 count? Can a manifest demonstrate participant-disjoint assignment across eyes-open, eyes-closed, and any other included recordings? If a different dataset version or subset is used, please identify it precisely.
-
Does the ranking persist with train-fitted feature normalization and a common linear or capacity-controlled readout? What explains the near-chance Mumtaz balanced accuracy but high AUROC/AUPR of CBraMod?
-
What evidence shows that ST-GFM reduces the local predictive shortcuts motivating it, beyond producing spatially correlated masks? A matched local-predictability diagnostic or shuffled-coordinate mask control would be informative; alternatively, the explanation can be stated as a design hypothesis supported by downstream ablations.
9 comments
Response to W2 & Q1
W2.1 & Q1. The dataset split for Mumtaz2016
Thank you for pointing out this ambiguity. The counts in Table 5 refer to EEG recordings, not unique participants. We use 120 recordings from 63 participants (30 HC and 33 MDD), excluding MDD S8 whose recordings are unavailable. Among these participants, seven recordings are missing (HC: S12 EC, S18 EC, S25 EO. MDD: S4 EC, S7 EO, S12 EC, S16 EC), while HC S15 has two EO recordings, resulting in 120 recordings in total.
We verified the participant-disjoint splits:
- Train (21 HC + 22 MDD): HC S1–S2, S10–S28; MDD S1–S2, S10–S29.
- Val (4 HC + 5 MDD): HC S3–S4, S29–S30; MDD S3, S30–S33.
- Test (5 HC + 6 MDD): HC S5–S9; MDD S4–S7, S9, S34.
HC and MDD IDs refer to distinct participants. All available EO/EC recordings and their derived segments from each participant are assigned to the same split, with no participant overlap. We have added a detailed description of the Mumtaz2016 splitting protocol in the appendix.
File-to-participant-to-split manifest:
| Split | HC subjects | MDD subjects |
|---|---|---|
| Train | S1–2,S10–28 | S1–2,S10–29 |
| Val | S3–4,S29–30 | S3,S30–33 |
| Test | S5–9 | S4–7,S9,S34 |
Each listed subject has EC.edf and EO.edf, except:
- Missing: HC (S12-EC,S18-EC,S25-EO) and MDD (S4-EC,S7-EO,S12-EC,S16-EC).
- For
HC S15has6921143_H S15 EO.edf,6921959_H S15 EO.edfandH S15 EC.edf.
W2.2. Trial Splits within Sessions for SEED-V / SEED-VII
We agree with the reviewer that SEED-V and SEED-VII use trial-level splits within sessions, whereas the other datasets primarily use subject-disjoint splits. These protocols evaluate different generalization settings and should not be interpreted uniformly as cross-subject generalization.
Our primary goal is to assess representation transferability on downstream tasks, rather than exclusively cross-subject generalization. All pretrained encoders remain frozen during downstream evaluation, and identical splits are used for all compared models within each dataset.
To avoid potential ambiguity, we will provide a more detailed description of the dataset-specific splitting protocols in the revised paper and explicitly distinguish trial-level and subject-level evaluation settings when interpreting generalization results.
W2.3. KNN Variability and Participant-Level Uncertainty
We appreciate the reviewer's suggestion regarding uncertainty reporting. The original KNN standard deviations quantify sensitivity across K choices, rather than inter-participant or run-to-run uncertainty. We conducted a participant-level KNN analysis on six subject-disjoint datasets using L2-normalized features and cosine similarity. We report the mean and standard deviation of balanced accuracy across test participants.
Participant-level KNN balanced accuracy (%).
| Dataset | Participants | Bal. Acc. | Individual Scores / Range |
|---|---|---|---|
| BCICIV-2a | 2 | 33.91±2.73 | 31.98, 35.83 |
| CHB-MIT | 2 | 79.37±12.77 | 88.40, 70.34 |
| MentalArithmetic | 4 | 65.21±14.75 | 48.06, 60.00, 70.00, 82.78 |
| ISRUC | 10 | 60.00±8.92 | 41.96–70.24 |
| PhysioNet-MI | 20 | 30.53±5.34 | 21.63–40.98 |
| HMC | 26 | 47.69±10.36 | 28.60–70.67 |
These results reveal varying degrees of inter-participant variability. We acknowledge that datasets with few test participants cannot provide reliable estimates of population-level uncertainty. Nevertheless, this analysis makes participant-level performance differences explicit rather than relying solely on aggregate metrics.
We will include these results, along with participant-level analyses for the remaining K values, in the revised appendix and clarify the interpretation of the reported standard deviations in the Experimental Setup.
Additional Experimental Results
Table 1. KNN-probing with feature standardization and Euclidean distance (%).
| BCICIV-2a | SEED-V | MentalArithmetic | |||||||
|---|---|---|---|---|---|---|---|---|---|
| Method | Bal. Acc. | kappa | W-F1 | Bal. Acc. | kappa | W-F1 | Bal. Acc. | AUC-PR | AUROC |
| LaBraM | 30.07±0.76 | 6.76±1.01 | 29.70±0.70 | 22.60±0.07 | 3.41±0.08 | 22.57±0.43 | 55.71±2.92 | 44.25±5.47 | 57.34±4.26 |
| CBraMod | 32.12±1.61 | 9.49±2.14 | 31.34±1.37 | 21.52±0.14 | 2.12±0.21 | 21.69±0.17 | 54.10±1.24 | 35.45±5.29 | 56.52±2.42 |
| CSBrain | 27.67±1.43 | 3.56±1.91 | 26.76±1.21 | 20.75±0.16 | 1.03±0.24 | 19.72±0.44 | 52.36±1.46 | 32.70±1.43 | 53.79±1.79 |
| REVE | 33.65±0.59 | 11.53±0.79 | 26.68±1.34 | 21.15±0.09 | 1.67±0.13 | 20.60±0.54 | 50.62±1.81 | 25.93±1.47 | 53.06±2.80 |
| CodeBrain | 28.94±0.73 | 5.25±0.97 | 22.06±1.31 | 26.02±0.50 | 7.75±0.67 | 26.65±0.41 | 54.38±1.73 | 39.67±1.10 | 58.39±3.23 |
| EEG-GPRL (ours) | 35.75±2.33 | 14.33±3.10 | 35.06±2.24 | 29.31±0.63 | 11.66±0.85 | 29.75±0.64 | 59.24±0.63 | 47.71±4.74 | 61.86±3.22 |
| Physio-MI | Mumtaz2016 | SEED-VII | |||||||
|---|---|---|---|---|---|---|---|---|---|
| Method | Bal. Acc. | kappa | W-F1 | Bal. Acc. | AUC-PR | AUROC | Bal. Acc. | kappa | W-F1 |
| LaBraM | 28.77±1.22 | 5.02±1.62 | 28.73±1.24 | 80.36±0.26 | 88.47±2.08 | 89.33±2.46 | 20.27±0.32 | 7.20±0.41 | 19.50±0.11 |
| CBraMod | 32.74±1.45 | 10.32±1.93 | 31.10±1.43 | 63.27±0.54 | 65.30±3.41 | 65.32±0.82 | 17.55±0.24 | 3.94±0.30 | 16.63±0.18 |
| CSBrain | 28.58±0.45 | 4.77±0.60 | 27.97±0.53 | 76.50±0.45 | 74.56±1.88 | 80.24±1.05 | 17.05±0.08 | 3.37±0.10 | 15.88±0.38 |
| REVE | 29.59±0.62 | 6.13±0.83 | 29.20±0.60 | 66.32±0.45 | 78.64±0.95 | 72.84±2.65 | 23.50±0.48 | 11.17±0.31 | 23.77±0.28 |
| CodeBrain | 30.63±1.43 | 7.51±1.91 | 30.39±1.43 | 80.79±1.06 | 84.98±1.52 | 86.10±1.33 | 24.19±0.67 | 12.31±0.54 | 24.02±0.53 |
| EEG-GPRL (ours) | 30.75±1.12 | 7.67±1.50 | 30.72±1.23 | 84.36±0.25 | 89.68±0.38 | 90.69±0.98 | 25.57±0.27 | 13.41±0.36 | 25.91±0.29 |
| ISRUC | CHB-MIT | HMC | |||||||
|---|---|---|---|---|---|---|---|---|---|
| Method | Bal. Acc. | kappa | W-F1 | Bal. Acc. | AUC-PR | AUROC | Bal. Acc. | kappa | W-F1 |
| LaBraM | 26.20±0.97 | 6.48±1.30 | 17.82±1.75 | 61.77±0.84 | 6.30±2.59 | 66.52±0.69 | 40.62±1.97 | 24.26±0.99 | 35.27±0.63 |
| CBraMod | 32.47±0.14 | 18.30±0.29 | 32.92±0.27 | 70.58±1.10 | 30.11±2.58 | 80.94±1.72 | 25.98±0.39 | 6.35±0.56 | 20.40±1.01 |
| CSBrain | 22.63±0.75 | 4.06±0.86 | 19.88±1.48 | 71.39±1.22 | 26.88±2.56 | 76.52±1.35 | 43.50±0.22 | 25.41±0.14 | 35.33±0.60 |
| REVE | 52.64±1.14 | 37.94±1.34 | 52.26±0.95 | 50.55±0.09 | 22.76±3.23 | 51.24±0.37 | 43.68±0.70 | 28.43±0.96 | 43.96±0.76 |
| CodeBrain | 54.34±0.58 | 42.29±1.73 | 53.41±0.81 | 73.64±0.51 | 21.17±6.11 | 78.63±0.75 | 43.34±0.69 | 30.10±1.04 | 45.74±0.55 |
| EEG-GPRL (ours) | 56.37±1.43 | 46.76±2.59 | 59.85±1.85 | 74.80±0.63 | 33.89±0.59 | 84.12±2.11 | 45.89±0.78 | 30.68±1.24 | 46.67±1.10 |
Response to W1 & Q2
W1.1 & Q2.1. Feature Standardization or L2 Normalization in KNN
We appreciate the reviewer's concern that feature scaling may affect KNN comparisons, particularly since VISReg regularizes the feature distribution. We therefore re-evaluated all models under two configurations:
All models use identical downstream splits and KNN settings (K=3,5,7,9,11). Detailed results are provided in Tables 1–2 below.
Feature normalization indeed affects several baselines, potentially due to differences in feature scales. Nevertheless, EEG-GPRL retains the highest mean balanced accuracy across nine datasets under both configurations, outperforming the other baselines. These results further support the robustness of EEG-GPRL's representation transferability across different normalization settings.
We appreciate your suggestion, which helps us establish a fairer and more reliable comparison across baseline. We have update the paper with the feature-normalized results.
W1.2 & Q2.2. Common Linear Probing
We agree that the backbone-dependent hidden dimensions of the original MLP readout may introduce differences in classifier capacity. To address this concern, we additionally evaluated all models using a common linear-probing protocol, freezing the pretrained encoders and replacing the nonlinear MLP heads with single-layer linear readout. The mean Bal. Acc. summarized below, with detailed results provided in Table 3 below.
Mean linear-probing balanced accuracy across nine datasets (%).
EEG-GPRL achieves the highest mean balanced accuracy (61.44%), outperforming the other baselines. As shown in Table 3, EEG-GPRL achieves the highest or tied-highest performance on eight datasets and ranks second on PhysioNet-MI.
These results further demonstrate that EEG-GPRL's advantage persists with a simple linear readout, rather than depending on nonlinear MLP classifier capacity.
Q2.3. Results of CBraMod in Mumtaz2016
Thanks for pointing this out. We checked CBraMod's confusion matrices across five seeds (0 = HC, 1 = MDD. rows = true labels, columns = predictions):
CBraMod predicts most samples as Class 0 (HC), resulting in nearly 100% recall for Class 0 but very low recall for Class 1 (MDD). Since balanced accuracy averages the recall of both classes, Bal.Acc.=2Recall0+Recall1, its Bal. Acc. remains close to 50%. However, the high AUROC (97.37%) and AUPR (97.01%) suggest that cbramod can distinguish the two classes well, but the default threshold (0.5) is not suitable for CBraMod.
We therefore re-evaluated all models using thresholds selected on the validation set:
With the validation-selected threshold, CBraMod's Bal. Acc. increases from 53.34% to 80.86%, indicating that threshold selection accounts for much of the original gap. EEG-GPRL maintains the highest balanced accuracy (95.51%) under the same protocol. We will update the binary-classification results using validation-selected thresholds and clarified the evaluation protocol in the revised manuscript.