- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This work proposes DLink, a distillation framework for EEG-FMs that does not rely only on the last layer embeddings but has router to mix information from all layers accordingly. This is mostly based to the fact that EEG-FMs hold task-related information in the middle layers of the network.
Critical strengths
The paper is well-written and with a clear motivation. The authors identified that task-related information resides in the intermediate layers and designed a framework to make use of this previous finding and train better student models.
Critical weaknesses
-
Although I do understand the motivation about each component in Figure 2, it is not clear to be why a simpler architecture would not suffice. For example, get all the layer embeddings and use attention mechanisms. Or use interpretability tools to locate the most informative layer and use only that in the traditional distillation fashion.
-
Only three EEG-FMs are reported. Table 1 uses only CBraMod as backbone.
Questions for authors
-
Can the authors provide some evidence that the router is necessary and a simpler strategy would not be enough ?
-
Can the authors include more EEG-FM not only CBraMod as backbone to demonstrate the generalisation of the framework ?
-
Table 1 shows fine-tuned results. How about frozen EEG-FM / head-only results ?
-
Why in Table 2, why CBraMod, CSBrain and LaBraM results change between DLink-S and -M ?
6 comments
# Response to Reviewer 9Pxs(2/2)
W2 & Q2: Generalization beyond CBraMod
Thank you for encouraging broader teacher evaluation. We would like to highlight Appendix Table D1 of the original submission, which reports the complete comparison of CBraMod, CSBrain, and LaBraM as alternative teachers across all six datasets (18 dataset-teacher settings). Each teacher is used separately to train the same compact student. Table 1 fixes CBraMod for the main comparison, while Table D1 evaluates teacher generalization across datasets. The original Table 2 additionally compares all three teachers at both student capacities on FACED.
To strengthen this evaluation, we have now added REVE (El Ouahidi et al., 2025) as a fourth teacher on FACED. The following table summarizes ACC-B for all four teachers and the corresponding supervised students. Values are mean standard deviation over five runs.
| Teacher / training | DLink-S ACC-B (%) | DLink-M ACC-B (%) |
|---|---|---|
| Supervised, no KD | 41.61 0.76 | 47.35 1.33 |
| CBraMod | 43.46 0.80 | 49.81 1.10 |
| CSBrain | 43.48 0.66 | 49.62 0.66 |
| LaBraM | 42.28 1.14 | 49.20 1.29 |
| REVE (new) | 43.12 0.66 | 49.92 0.85 |
REVE-based distillation improves mean ACC-B over supervised training by 1.51 points for S and 2.57 points for M. All four teachers improve over the corresponding supervised student at both capacities. These results extend the evidence that DLink can transfer knowledge from different EEG foundation models into a compact student.
Q3: Distillation from a frozen pretrained teacher
Thank you for suggesting this evaluation. We additionally evaluate DLink-S on the four datasets in Table 1 using a frozen pretrained CBraMod checkpoint, without task-specific teacher fine-tuning. We retain the same validation-based hyperparameter-search procedure, student architecture, and distillation configuration. The table reports the resulting student's test ACC-B as mean standard deviation.
| Dataset | Pretrained teacher | DLink-S ACC-B (%) |
|---|---|---|
| FACED | CBraMod | 42.46 1.56 |
| mental-arithmetic | CBraMod | 61.39 2.07 |
| Siena | CBraMod | 72.91 5.09 |
| SHU-MI | CBraMod | 62.49 1.85 |
Q4: Why do teacher-labelled rows differ between DLink-S and DLink-M?
Thank you for asking about this comparison. Table 2 separates teacher identity from student capacity, as detailed below.
a. Each entry reports the distilled student's test performance. In Table 2, CBraMod, CSBrain, and LaBraM identify the teacher used during training. Their rows report the resulting DLink-S or DLink-M student, rather than the teacher's own classifier performance. For example, the CBraMod row reports 43.46% for the S student and 49.81% for the M student trained with that teacher.
b. S and M are different student capacities. They share the project-then-compress design, but differ in Transformer depth, compression width, and retained latent resolution (Appendix B.1). On FACED, S uses 0.2052M parameters, while M uses 0.6780M parameters (3.30× as many parameters). Their different capacity and compression settings yield different student results even with the same teacher. Within each capacity, all teacher comparisons share the same student architecture and inference cost.
The comparisons above explicitly label the teacher and the evaluated student configuration to make this distinction easy to follow.
Response to authors
I thank the authors for their time and effort in this rebuttal response as well as the extra experiments. Based on the new results the difference between pre-trained CBraMod and fine-tuned CBraMod as a teacher on FACED is 43.46-42.46 = 1% (DLink-S ACC-B). So the DLink can cover for the missing fine-tuning information. That is a very interesting finding but the ablation was done with only one EEG-FM so not a generic observation and it was not tested with other baseline methods if the same hypothesis holds there as well. I will raise my score to 3 but I still think the gains reported in W1 & Q1 to be small relative to other methodologies for a clear acceptance.
# Response to Reviewer 9Pxs(1/2)
Thank you for your time and constructive suggestions. The original submission includes simpler aggregation and adaptive-distillation controls in Tables 1, 3, and F2, and cross-teacher comparisons in Tables 2 and D1. We highlight these comparisons below, clarify the teacher/student labels, and extend the evaluation with a fourth foundation model, REVE, and frozen pretrained-teacher experiments.
W1 & Q1: Why use the router rather than simpler layer aggregation?
Thank you for suggesting these alternatives. We address this question from three perspectives: design flexibility, empirical benefits over alternative strategies, and deployment efficiency.
a. Design: sample-wise aggregation provides greater flexibility. Fixed Best-Val selects one layer according to aggregate validation performance and uses it for every input. DLink instead learns an input-conditioned mixture of teacher layers, allowing their contributions to vary across samples. This provides flexibility beyond selecting a single globally preferred layer or using one shared mixture for all inputs.
b. Experiments: greater benefits than the compared alternatives. Tables 1, 3, and F2 already evaluate fixed-layer selection, uniform averaging, shared learned weights, and adaptive distillation. These include ReviewKD (Chen et al., 2021), which incorporates attention-based multi-level feature fusion, and S-A KD (Song et al., 2022), which adapts distillation locations to individual samples. We summarize these comparisons below using the same compact student and teacher. Values are test ACC-B, reported as mean ± standard deviation.
DLink improves mean ACC-B over ReviewKD by 2.24 points on FACED and 1.74 points on SHU-MI. Its gains over S-A KD are 2.37/1.86 points, over averaging 0.98/7.05 points, and over Fixed Best-Val 0.36/6.08 points, respectively. The extended Fixed Best-Val comparison also favors DLink in mean ACC-B across all six datasets, with gains of 0.36-11.77 points. Table F2 also reports Global Learnable, which uses one shared layer-weight vector, at 41.88 ± 1.13% ACC-B on FACED.
c. Deployment: improved performance without additional inference cost. The router operates only during distillation and is discarded together with the teacher after training. The resulting student therefore retains the same inference architecture and parameter count as the corresponding standalone student. Across six datasets, DLink-S achieves a 134.5-fold parameter reduction relative to CBraMod (0.0953M vs. 12.817M, based on dataset means). Figures 1 and 5 visualize the accuracy-efficiency comparison, with detailed CPU median forward latencies in Appendix Table G1. The performance gains above require no additional routing computation at deployment.