- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
In this work, the authors first show that the final layer classification readout is crucial for EEG FMs on downstream tasks and even after accounting for this, intermediate layers retain complementary task-predictive information that is not fully recoverable from the final representation. They also show this task-predictive information can partly transfer across downstream datasets.
Critical strengths
- The authors disentangle the contributions of readout design and layer-wise representations to downstream decoding performance. This is an interesting and often overlooked aspect in EEG FM evaluation. The experiments clearly demonstrate that both readout selection and representation depth can substantially influence performance.
- The cross-dataset transfer experiments provide additional evidence that task-relevant representations can generalize beyond a single dataset. The finding that source-selected layers and predictive directions remain useful on another dataset of the same task strengthens the results and highlights the potential transferability of these representations.
Critical weaknesses
- Limited mechanistic insight and novelty.
While the authors demonstrate that predictive information is distributed across layers, similar findings have been established in vision and language models. I feel that the investigation stops at a relatively superficial level, mainly demonstrating where predictive information can be accessed rather than explaining what that information represents or why it is distributed across depth. More specifically, for EEG, different downstream tasks rely on distinct physiological features. For example, ERP decoding depends on waveform morphology, peak amplitudes, and latencies, whereas MI relies more heavily on sensorimotor oscillations and their temporal dynamics. Investigating how these features are represented and transformed across layers would provide a much more meaningful explanation of the observed phenomenon. Furthermore, the analysis is not sufficiently connected to the pretraining objectives. MAE, JEPA, contrastive and autoregressive objectives may encourage different representations and potentially different depth-dependent behaviors. Although the evaluated models have different pretraining strategies, the current experiments do not disentangle the effects of the pretraining objectives from differences in architecture and tokenization.
- Limited practical relevance beyond frozen representation probing
The study only focuses on frozen EEG foundation models. However, previous benchmarks have shown that linear probing often performs substantially worse than full or parameter-efficient fine-tuning on downstream EEG tasks (Yang et al., 2026; Liu et al., 2026; Lu et al., 2026). This raises questions about the practical implications of the findings. During fine-tuning, the model can refit its representations according to downstream data, potentially making previously inaccessible task-relevant information available at the final layer. Consequently, the intermediate-layer advantages observed under frozen probing may diminish or even disappear after fine-tuning. While the paper provides useful guidance on improving frozen feature extraction through layer selection and better readouts, it remains unclear how these findings could improve EEG foundation models under more practical adaptation settings or inform the development of better pretrained representations.
References
Yang, Liuyin, et al. "Are EEG foundation models worth it? comparative evaluation with traditional decoders in diverse BCI tasks." The Fourteenth International Conference on Learning Representations. 2026.
Liu, Dingkun, et al. "EEG-FM-Compass: progress, benchmarking, and future directions for EEG foundation models." National Science Review (2026): nwag466.
Lu, Ziling, et al. "OmniEEG-Bench: A Standardized Evaluation Benchmark for EEG Foundation Models." arXiv preprint arXiv:2606.00815 (2026).
Questions for authors
-
Can the authors investigate which physiologically meaningful EEG features are represented at different depths, such as ERP waveform morphology and latency, MI-related mu/beta oscillations, or sleep-specific spectral and temporal patterns? Could these differences help explain the task-dependent layer preferences?
-
To what extent is the observed distribution of predictive information determined by the pretraining objective? Would reconstruction-based, latent-prediction, and autoregressive models show similar depth-dependent behavior under a controlled comparison using the same backbone and pretraining data?
-
Do the intermediate-layer advantages and complementary residual features persist after partial or full fine-tuning? In particular, does adapting the final layers to downstream supervision reduce the additional benefit of earlier representations?
-
Have the authors examined whether layers selected for one task remain useful for another, such as selecting an intermediate layer on MI and evaluating it on ERP? This could help distinguish generally transferable representations from task-specific layer preferences.
-
Interpretation of Figure 4: Since most of the performance degradation caused by removing source-derived task directions can be recovered through classifier refitting, to what extent do these directions represent uniquely valuable task information rather than features preferentially used by a particular classifier? Could the authors further clarify what these intervention results establish about the underlying representations?
6 comments
Rebuttal from the Author to Reviewer 7dth (2/2)
3. Persistence after partial fine-tuning
Adapting final layers reduces the N170 depth advantage without eliminating it, and residual utility persists. We fine-tuned CBraMod's final two blocks on OpenBMI and N170 (3 seeds, first 10 blocks frozen, development splits for early stopping and layer selection), then froze the adapted encoder and refitted all diagnostic probes, preprocessing, and cross-layer mappings.
| Task / metric | Final (frozen → adapted) | Selected (frozen → adapted) | Post-adaptation depth gain [95% CI] |
|---|---|---|---|
| OpenBMI / BA | 76.09 → 77.92 | 76.17 → 77.92 | 0.00 [0.00, 0.00] |
| N170 / AUROC | 68.77 → 74.53 | 77.45 → 77.45 | +2.92 [1.34, 4.40] |
Since N170's selected layer is in the frozen blocks, its score stays 77.45 by construction. Key findings:
- Adapting the final two blocks recovers +5.76% [4.52, 6.98] at N170's final layer (
68.77 → 74.53), improving feature accessibility. - A +2.92% [1.34, 4.40] depth gap persists (down from ~8.67%).
- On the adapted encoder, intermediate residuals still beat the adapted final layer by +0.88% [0.37, 1.39] and shuffled residuals by +1.84% [1.01, 2.99].
On OpenBMI, final and selected scores match after adaptation, and residual augmentation yields −0.41% [−0.75, −0.08], closing a negligible frozen gap. Thus, depth and readout choices should be reassessed after adaptation. We will restrict our claim to partial fine-tuning in the revision.
4. Directional layer reuse across tasks
We added MI-to-ERP cross-task transfer using CBraMod (OpenBMI→N170, OpenBMI→N400, N170→OpenBMI). Source depth is the mode of source-validation-selected layers (ties broken toward shallower depth). On target tasks, we fix the target's validation-selected final readout and refit classifiers, so only layer identity transfers (unlike Section 4.3's fixed source-interface transfer).
| Transfer / metric | Source − final [95% CI] | Source − random layers [95% CI] | Source − first layer [95% CI] |
|---|---|---|---|
| OpenBMI→N170 / AUROC | +9.50 [7.59, 11.35] | +1.73 [0.87, 2.48] | +0.11 [−0.67, 0.87] |
| OpenBMI→N400 / AUROC | +6.76 [5.62, 7.89] | +1.89 [1.43, 2.32] | +0.12 [−0.75, 0.99] |
| N170→OpenBMI / BA | −0.26 [−1.13, 0.60] | +0.00 [−0.44, 0.43] | +0.69 [0.02, 1.35] |
Random-layer baseline averages all non-final layers (N170/N400: 40 subjects; OpenBMI: 54). MI-selected depth transfers well to both ERP tasks, outperforming final and random layers while matching the first layer (+0.11 and +0.12 pp, CIs spanning zero), indicating a reusable shallow region. N170→OpenBMI yields no gain. Cross-task layer reuse is directional and depends on target signal properties (added to Section 4.3 and Appendix F.2.3).
5. What Figure 4 establishes after classifier refitting
We agree that the large drop under a frozen classifier reflects reliance on removed coordinates, while post-refit recovery shows alternative predictive features remain.
To isolate direction utility beyond classifier dependence, we added equal-rank deletion (removing 4 dimensions from the source-fitted PCA200 space, averaged over 20 random and 20 label-shuffled bases; energy-matched perturbations reported separately) and positive augmentation (appending 4 source residual coordinates to target final features).
| HMC→ISRUC model | 4-coord. aug. − final [95% CI] | Extra post-refit loss vs. random deletion [95% CI] |
|---|---|---|
| CBraMod | +1.17 [0.72, 1.69] | 1.70 [1.24, 2.20] |
| REVE | +0.67 [0.30, 1.07] | 0.78 [0.31, 1.28] |
| LaBraM | +0.97 [0.67, 1.28] | 2.17 [1.70, 2.65] |
Across 98 ISRUC subjects (paired bootstrap on macro-F1), augmentation also beats equal-rank random augmentation by 1.01%, 0.63%, and 0.92%. Positive augmentation and persistent post-refit excess deletion loss show that source residual directions carry genuine, transferable utility under tested interfaces. Recovery and transferable utility coexist across three model families. We will revise Section 4.4, Figure 4, and Tables S15–S16 accordingly.
Thank you for the detailed response and the additional experiments. I appreciate the authors' openness in discussing the results, including where the new findings challenge the original claims. I have also read the responses to the other reviewers. Unfortunately, the new controls have made me less convinced of the paper's main claims.
-
Complementary information across layers. In the response to Reviewer PC6L, when the final layer is kept at its native dimension, the residual gains drop from +9.57 / +8.96 / +6.11 / +2.12 to +1.22 / +0.48 / +0.71 / −0.01. This suggests that much of the apparent "complementary" information comes from compressing the final layer to 200 PCA dimensions, rather than from information that the final layer does not contain. The remaining gains are around 1 percentage point or less. I think this substantially weakens the paper's second main claim. Could the authors also clarify whether the +0.88 residual gain after fine-tuning (reported in the response to my review) was obtained using the PCA200 final layer?
-
Physiological diagnostics (my Q1). Thank you for adding these analyses. However, the mean PO8 amplitude in the 150–200 ms window is a linear function of the raw input, while band power is also relatively straightforward to extract from the signal. The fact that early layers decode these features better (R² around 0.6 in H01–H05 versus 0.14 at H12) may simply indicate that deeper layers progressively lose information about the original input. This does not yet tell us much about what different layers actually represent, or why. I think a raw-input baseline using the same ridge readout, together with a randomly initialized encoder, would be important for interpreting these results.
-
Pretraining objectives (my Q2). The controlled MAE/JEPA comparison is useful. However, on N170, the randomly initialized encoder (70.59) outperforms the final layers of both MAE (69.42) and JEPA (63.82), while the selected layers reach around 74.4. Together with point 2 and the cross-task results, where the source-selected layer performs almost identically to the first layer (+0.11 and +0.12, with intervals including zero), this suggests a simpler interpretation: for ERP tasks, layers closer to the input may work best because deeper layers in the current models discard waveform-level information. I find this observation interesting in itself, but it supports a different conclusion than the claim that predictive information is distributed across depth.
-
Fine-tuning (my Q3). In the new experiments, only the last two blocks are adapted, while the selected N170 layer lies within the frozen blocks and therefore cannot change. This leaves my original question largely unanswered, as the experiment does not test full fine-tuning or parameter-efficient adaptation methods such as LoRA, which are more representative of the practical settings I was interested in.
-
Minor clarification. The REVE OpenBMI→BCIC-IV-2a source-minus-final gain is reported as +2.28 [0.42, 4.24] in the response to Reviewer PC6L, but as +2.74 in the new transfer table. Could the authors clarify the discrepancy?
Overall, I think the new experiments reveal interesting findings, but they also suggest the paper needs a more substantial reframing, along the lines the authors themselves have outlined and that Reviewer 3nUD has also suggested. In particular, the distinction between genuinely complementary information across layers and the preservation of input-level information in earlier layers seems important for interpreting the results.
Since a revised manuscript reflecting this reframing is not yet available, I cannot assess how well the authors would address these issues in the current round. I therefore maintain my rating. Nevertheless, I appreciate the authors' efforts and believe this work could become a strong paper with clearer interpretation and more carefully scoped claims.
Rebuttal from the Author to Reviewer 7dth (1/2)
Dear Reviewer,
Thank you for recognizing the value of separating readout design from representation depth and for highlighting our cross-dataset transfer findings. Your questions pinpointed three decisive steps toward deepening our paper:
While our primary contribution establishes how readout choice, layer depth, and source-defined predictive directions govern frozen EEG evaluation and reuse, these new experiments anchor our observations in concrete physiological diagnostics and demonstrate how depth accessibility shifts with pretraining and adaptation. We agree that the original manuscript did not sufficiently connect these empirical patterns to physiological correlates or controlled pretraining comparisons.
We have completed new experiments addressing all three questions, alongside real MI–ERP cross-task layer transfer and equal-rank direction-intervention controls. We will integrate these results into the revised manuscript with the scope of each conclusion explicitly stated.
1. Physiologically grounded representation diagnostics across depth
We added direct decoding of two predefined physiological measurements across all twelve CBraMod layers using ridge regression and fixed readouts within participant-disjoint splits: log mean C3/C4 power over 8–30 Hz for OpenBMI (spanning the mu/beta range) and baseline-corrected mean PO8 amplitude over 150–200 ms for N170.
The layerwise profiles show that both physiological quantities are more accessible at earlier depths. For N170 amplitude, R² stays at approximately 0.59–0.63 in layers H01–H05 before dropping to 0.14 at H12, and combining final-layer features with intermediate residuals restores R² to 0.60. For OpenBMI band power, the profile is less monotonic (0.44 at H02, 0.33 at H06, and 0.24 at H12). The sharper depth decay of N170-amplitude accessibility aligns closely with the larger classification depth benefit observed on N170, providing a concrete physiological association for our task-dependent findings.
2. Controlled comparison of pretraining schemes (MAE vs. JEPA)
We agree that comparing the five original foundation models cannot isolate pretraining objectives from differences in architecture, tokenization, and training data. In order to make direct mechanism analysis, We therefore trained paired MAE and JEPA models with an identical eight-block encoder, tokenizer, HBN data pool, 25-channel input protocol, sampling stream, temporal masking, optimizer schedule, and nominal data exposure across three paired seeds (implementation details will be listed in revised manuscript). MAE reconstructs masked waveform patches, whereas JEPA predicts normalized teacher representations.
For each model, depth gain is measured after fixing the validation-selected final-layer readout, and residual gain is final-plus-residual performance minus the final-layer baseline. The table reports JEPA-minus-MAE differences in these gains.
Both pretraining schemes exhibit depth benefits, with markedly larger JEPA-over-MAE depth and residual gains on the ERP tasks (while OpenBMI differences are smaller and span zero). Crucially, inspecting the absolute scores reveals a central evaluation insight: pretraining objectives can differ primarily in how predictive information is distributed across depth rather than in peak representational quality. On N170, MAE improves from 69.42 (final layer) to 74.42 (selected layer), whereas JEPA improves from 63.82 to 74.47; by comparison, a randomly initialized encoder scores 70.59 at the final layer and 70.47 after depth selection. Thus, evaluating only the final layer would make JEPA appear worse than both MAE and a random encoder, whereas accounting for depth shows that both pretrained models converge to virtually identical selected-layer performance and clearly outperform the random baseline.