acceptodds
Under review as a conference paper at ICLR 2027

DLink: Distilling Layer-wise and Dominant Knowledge from EEG Foundation Models

Abstract

EEG foundation models (EFMs) provide strong representations for diverse EEG decoding tasks through large-scale pretraining and downstream fine-tuning. Through empirical analysis, we observe that (i) while lightweight models remain competitive, they are highly task-dependent. Retaining high-capacity EFMs as inference backbones incurs substantial computational overhead, making knowledge distillation a natural route for model compression; and (ii) direct distillation from a fixed teacher representation underutilizes EFM knowledge, as task-discriminative information is distributed across intermediate layers rather than concentrated in the final layer. These observations motivate DLink (Distilling Layer-wise and Dominant Knowledge), a spectrally guided distillation framework with input-conditioned layer routing for transferring EFM knowledge into compact students. DLink uses a router to aggregate teacher layers for each input, and aligns magnitude and phase spectra to mitigate compression-induced spectral distortion in learned representations. The routed teacher knowledge is internalized by a project-then-compress student; the teacher and router are used only during training. Experiments across six EEG benchmarks show that DLink improves compact students while remaining lightweight at inference. Further analyses support learned layer aggregation and spectral distillation, establishing DLink as a practical framework for distilling layer-wise dominant knowledge into compact EEG models.

Primary area:
applications to neuroscience & cognitive science
Keywords:
Electroencephalogram (EEG), Knowledge distillation, EEG foundation models

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_9PxsOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This work proposes DLink, a distillation framework for EEG-FMs that does not rely only on the last layer embeddings but has router to mix information from all layers accordingly. This is mostly based to the fact that EEG-FMs hold task-related information in the middle layers of the network.

Critical strengths

The paper is well-written and with a clear motivation. The authors identified that task-related information resides in the intermediate layers and designed a framework to make use of this previous finding and train better student models.

Critical weaknesses
  1. Although I do understand the motivation about each component in Figure 2, it is not clear to be why a simpler architecture would not suffice. For example, get all the layer embeddings and use attention mechanisms. Or use interpretability tools to locate the most informative layer and use only that in the traditional distillation fashion.

  2. Only three EEG-FMs are reported. Table 1 uses only CBraMod as backbone.

Questions for authors
  1. Can the authors provide some evidence that the router is necessary and a simpler strategy would not be enough ?

  2. Can the authors include more EEG-FM not only CBraMod as backbone to demonstrate the generalisation of the framework ?

  3. Table 1 shows fine-tuned results. How about frozen EEG-FM / head-only results ?

  4. Why in Table 2, why CBraMod, CSBrain and LaBraM results change between DLink-S and -M ?

6 comments
AuthorsOpenReview ↗

# Response to Reviewer 9Pxs(1/2)

Thank you for your time and constructive suggestions. The original submission includes simpler aggregation and adaptive-distillation controls in Tables 1, 3, and F2, and cross-teacher comparisons in Tables 2 and D1. We highlight these comparisons below, clarify the teacher/student labels, and extend the evaluation with a fourth foundation model, REVE, and frozen pretrained-teacher experiments.

W1 & Q1: Why use the router rather than simpler layer aggregation?

Thank you for suggesting these alternatives. We address this question from three perspectives: design flexibility, empirical benefits over alternative strategies, and deployment efficiency.

a. Design: sample-wise aggregation provides greater flexibility. Fixed Best-Val selects one layer according to aggregate validation performance and uses it for every input. DLink instead learns an input-conditioned mixture of teacher layers, allowing their contributions to vary across samples. This provides flexibility beyond selecting a single globally preferred layer or using one shared mixture for all inputs.

b. Experiments: greater benefits than the compared alternatives. Tables 1, 3, and F2 already evaluate fixed-layer selection, uniform averaging, shared learned weights, and adaptive distillation. These include ReviewKD (Chen et al., 2021), which incorporates attention-based multi-level feature fusion, and S-A KD (Song et al., 2022), which adapts distillation locations to individual samples. We summarize these comparisons below using the same compact student and teacher. Values are test ACC-B, reported as mean ±\pm standard deviation.

StrategyLayer-selection / aggregation ruleFACED ACC-B (%)SHU-MI ACC-B (%)
Fixed AvgArithmetic mean over layers42.48 ±\pm 1.1455.84 ±\pm 0.98
Fixed Best-ValOne validation-selected layer43.10 ±\pm 0.9756.81 ±\pm 0.93
ReviewKDAttention-based multi-level feature fusion41.22 ±\pm 0.7761.15 ±\pm 0.99
S-A KDSample-adaptive multi-layer distillation41.09 ±\pm 0.7661.03 ±\pm 0.68
Uniform PriorInput-conditioned router with uniform guidance42.83 ±\pm 0.8161.63 ±\pm 0.88
Router w/o Lprior\mathcal{L}_{\mathrm{prior}}Same router without spectral guidance39.68 ±\pm 1.2358.97 ±\pm 1.20
DLinkInput-conditioned routing with spectral guidance43.46 ±\pm 0.8062.89 ±\pm 1.17

DLink improves mean ACC-B over ReviewKD by 2.24 points on FACED and 1.74 points on SHU-MI. Its gains over S-A KD are 2.37/1.86 points, over averaging 0.98/7.05 points, and over Fixed Best-Val 0.36/6.08 points, respectively. The extended Fixed Best-Val comparison also favors DLink in mean ACC-B across all six datasets, with gains of 0.36-11.77 points. Table F2 also reports Global Learnable, which uses one shared layer-weight vector, at 41.88 ±\pm 1.13% ACC-B on FACED.

c. Deployment: improved performance without additional inference cost. The router operates only during distillation and is discarded together with the teacher after training. The resulting student therefore retains the same inference architecture and parameter count as the corresponding standalone student. Across six datasets, DLink-S achieves a 134.5-fold parameter reduction relative to CBraMod (0.0953M vs. 12.817M, based on dataset means). Figures 1 and 5 visualize the accuracy-efficiency comparison, with detailed CPU median forward latencies in Appendix Table G1. The performance gains above require no additional routing computation at deployment.

AuthorsOpenReview ↗

# Response to Reviewer 9Pxs(2/2)

W2 & Q2: Generalization beyond CBraMod

Thank you for encouraging broader teacher evaluation. We would like to highlight Appendix Table D1 of the original submission, which reports the complete comparison of CBraMod, CSBrain, and LaBraM as alternative teachers across all six datasets (18 dataset-teacher settings). Each teacher is used separately to train the same compact student. Table 1 fixes CBraMod for the main comparison, while Table D1 evaluates teacher generalization across datasets. The original Table 2 additionally compares all three teachers at both student capacities on FACED.

To strengthen this evaluation, we have now added REVE (El Ouahidi et al., 2025) as a fourth teacher on FACED. The following table summarizes ACC-B for all four teachers and the corresponding supervised students. Values are mean ±\pm standard deviation over five runs.

Teacher / trainingDLink-S ACC-B (%)DLink-M ACC-B (%)
Supervised, no KD41.61 ±\pm 0.7647.35 ±\pm 1.33
CBraMod43.46 ±\pm 0.8049.81 ±\pm 1.10
CSBrain43.48 ±\pm 0.6649.62 ±\pm 0.66
LaBraM42.28 ±\pm 1.1449.20 ±\pm 1.29
REVE (new)43.12 ±\pm 0.6649.92 ±\pm 0.85

REVE-based distillation improves mean ACC-B over supervised training by 1.51 points for S and 2.57 points for M. All four teachers improve over the corresponding supervised student at both capacities. These results extend the evidence that DLink can transfer knowledge from different EEG foundation models into a compact student.

Q3: Distillation from a frozen pretrained teacher

Thank you for suggesting this evaluation. We additionally evaluate DLink-S on the four datasets in Table 1 using a frozen pretrained CBraMod checkpoint, without task-specific teacher fine-tuning. We retain the same validation-based hyperparameter-search procedure, student architecture, and distillation configuration. The table reports the resulting student's test ACC-B as mean ±\pm standard deviation.

DatasetPretrained teacherDLink-S ACC-B (%)
FACEDCBraMod42.46 ±\pm 1.56
mental-arithmeticCBraMod61.39 ±\pm 2.07
SienaCBraMod72.91 ±\pm 5.09
SHU-MICBraMod62.49 ±\pm 1.85

Q4: Why do teacher-labelled rows differ between DLink-S and DLink-M?

Thank you for asking about this comparison. Table 2 separates teacher identity from student capacity, as detailed below.

a. Each entry reports the distilled student's test performance. In Table 2, CBraMod, CSBrain, and LaBraM identify the teacher used during training. Their rows report the resulting DLink-S or DLink-M student, rather than the teacher's own classifier performance. For example, the CBraMod row reports 43.46% for the S student and 49.81% for the M student trained with that teacher.

b. S and M are different student capacities. They share the project-then-compress design, but differ in Transformer depth, compression width, and retained latent resolution (Appendix B.1). On FACED, S uses 0.2052M parameters, while M uses 0.6780M parameters (3.30× as many parameters). Their different capacity and compression settings yield different student results even with the same teacher. Within each capacity, all teacher comparisons share the same student architecture and inference cost.

The comparisons above explicitly label the teacher and the evaluated student configuration to make this distinction easy to follow.

Reviewer_9PxsOpenReview ↗

Response to authors

I thank the authors for their time and effort in this rebuttal response as well as the extra experiments. Based on the new results the difference between pre-trained CBraMod and fine-tuned CBraMod as a teacher on FACED is 43.46-42.46 = 1% (DLink-S ACC-B). So the DLink can cover for the missing fine-tuning information. That is a very interesting finding but the ablation was done with only one EEG-FM so not a generic observation and it was not tested with other baseline methods if the same hypothesis holds there as well. I will raise my score to 3 but I still think the gains reported in W1 & Q1 to be small relative to other methodologies for a clear acceptance.

Reviewer_2yvmOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes to distill EEG foundation models to small models that can tackle different EEG tasks while compressing the model size. The proposed method includes three parts: 1) an input-conditioned layer router that performs a soft-weighted aggregation of the teacher’s hidden representations across layers; 2) an alignment of the student’s projected features with the aggregated teacher representations is achieved by performing an FFT along the time-latent axis, and 3) a student architecture characterized by an initial projection stage followed by a compression stage. Experiments on six tasks demonstrate the strong performance of the proposed method.

Critical strengths

1)This paper focuses on an important research question. Given the high inference overhead of EEG foundation models, the strategy of using it as a "teacher" during training, while retaining only a smaller model for deployment, is a well-defined approach. 2) The statement that "intermediate layers often possess greater discriminative power than the final layer" serves as a sound starting point for this work. 3) The experimental protocol is rigorous. It employs a two-stage hyperparameter selection process using only the training and validation sets, utilizes five different random seeds, and ensures data partitioning is subject-based. The inclusion of detailed information regarding data splits, licenses, and computational resources in the appendix further enhances reproducibility. 4) The ablation studies are comprehensive, covering a wide range of factors: freezing specific layers, uniform averaging, layer-wise scoring, "Routed L2" mechanisms, single-route inputs, isolating amplitude, and phase components.

Critical weaknesses
  1. The router module may not be significant. The module assigns nearly equal weights to each layer. The variance between these weights is negligible (only about 0.002), meaning the mechanism is effectively equivalent to a simple average across all layers.
  2. The claim that the router "selects the optimal layer for each input" may be an overclaim, as the weights remain virtually unchanged regardless of the input.
  3. It is possible that the router is simply selecting "simple" layers, as the router makes its choices based on the characteristics of the small model itself.
  4. The explanation of the "frequency trick" is insufficient. Mathematically, matching in the frequency domain is fundamentally equivalent to matching in the spatial or standard) domain.
  5. Most of the performance gains are too marginal to be convincing.
  6. The paper does not report the performance of the teacher model. Without this information, it is impossible to determine the extent to which the small model has bridged the performance gap.
  7. Paper survey is insufficient, more related works on model distillation or compression are needed.
Questions for authors

Please kindly refer to the weakness.

4 comments
AuthorsOpenReview ↗

# Response to Reviewer 2yvm(1/3)

Thank you for the time and effort you devoted to reviewing our work. We appreciate your detailed questions about the routing mechanism, spectral objective, and practical value of compact distillation. Below, we address these points and provide additional diagnostics, teacher results, and compression experiments to support the discussion.

Q1: Near-uniform weights and router significance

Our routing ablations, batch-mean control, and student-parameter diagnostics jointly show that the router affects learned student parameters and that sample-specific aggregation improves ACC-B, although the average layer weights differ only slightly from uniform. We explain the mechanism and empirical evidence below.

a. Mechanism: input-conditioned aggregation of teacher representations. For each input xx, the router computes a weight vector w(x)w(x) from that sample's student features and spectral cues. These weights determine the contribution of each teacher layer to the aggregated supervision, allowing the layer mixture to depend on the input. Similar mean weights across layers are compatible with this input-conditioned computation.

For uniform weights ul=1/Lu_l=1/L, the corresponding difference in teacher supervision is

fTroute(x)−fTavg(x)=∑l(wl(x)−ul)fT(l)(x).f_T^{\mathrm{route}}(x)-f_T^{\mathrm{avg}}(x)=\sum_l\bigl(w_l(x)-u_l\bigr)f_T^{(l)}(x).

Each weight adjustment changes the contribution of its corresponding teacher layer. Its effect depends jointly on the weight difference and the layer representation being aggregated. We quantify the practical consequences through the following performance controls and student-parameter diagnostics.

b. Empirical evidence. We examine both downstream prediction quality and the student parameters obtained under alternative aggregation strategies.

(i) ACC-B perspective. Fixed Avg tests exact averaging; Global Learnable uses one shared learned mixture. Uniform Prior retains the router with a uniform guidance target (Table 3; Tables F2/E6).

We add Batch-mean routing: at every training step, each sample's layer weights for feature KD are replaced by the batch mean wˉB=1∣B∣∑i∈Bw(xi)\bar w_B=\frac{1}{|B|}\sum_{i\in B}w(x_i). The mean retains gradients and the router continues training. The spectral prior is still computed on each sample's original router output. Values are test mean ±\pm sample SD over five runs.

MethodFACED ACC-B (%)SHU-MI ACC-B (%)
Fixed Avg42.48 ±\pm 1.1455.84 ±\pm 0.98
Uniform Prior42.83 ±\pm 0.8161.63 ±\pm 0.88
Batch-mean routing (new)40.12 ±\pm 0.6260.68 ±\pm 1.78
DLink43.46 ±\pm 0.8062.89 ±\pm 1.17

DLink exceeds Batch-mean routing by 3.34 points on FACED and 2.21 points on SHU-MI. This supports the benefit of preserving sample-specific teacher mixtures while retaining router training and the sample-wise prior. DLink also improves over Fixed Avg by 0.98/7.05 points and Uniform Prior by 0.63/1.26 points (FACED/SHU-MI), and Global Learnable by 1.58 points on FACED.

(ii) Parameter-similarity perspective. We compare two FACED students trained from the same initialization: DLink, using learned spectral-prior routing weights ww, and Fixed Avg, using fixed uniform layer weights ul=1/Lu_l=1/L. Both students have 205,240 trainable parameters. Let θD\theta_D and θA\theta_A denote their final parameter vectors, and θ0\theta_0 their common initialization. The table distinguishes layer-weight distributions, final student parameters, and parameter updates from initialization.

Quantities compared (DLink vs. Fixed Avg)Comparison metricValue
Corresponding layer weights: wlw_l vs. ulu_lMean absolute difference across layers0.00329
Layer weights: ww vs. uuCosine similarity0.9980
Final parameters: θD\theta_D vs. θA\theta_AL2 distance, ∥θD−θA∥2\lVert\theta_D-\theta_A\rVert_252.216
Final parameters: θD\theta_D vs. θA\theta_ARelative distance, ∥θD−θA∥2/∥θA∥2\lVert\theta_D-\theta_A\rVert_2/\lVert\theta_A\rVert_2143.05%
Final parameters: θD\theta_D vs. θA\theta_ACosine similarity0.3104
Updates: θD−θ0\theta_D-\theta_0 vs. θA−θ0\theta_A-\theta_0Cosine similarity0.1583

Although corresponding layer weights differ by only 0.00329 on average (cosine similarity 0.9980), the final student parameters and update directions differ substantially in this diagnostic pair. Their cosine similarities are 0.3104 and 0.1583, respectively. These measurements characterize differences in learned parameters and net updates; the matched held-out accuracy controls above provide the corresponding performance evidence.

AuthorsOpenReview ↗

# Response to Reviewer 2yvm(2/3)

Q2-Q3: Input conditioning and informative teacher layers

Thank you for raising these questions about input conditioning and layer informativeness. We clarify the routing mechanism and provide additional analyses below.

a. Input-conditioned soft aggregation. The router computes w(x)=softmax⁡(RθR(z(x))/τ)w(x)=\operatorname{softmax}(R_{\theta_R}(z(x))/\tau) from pooled student features and their spectra. This denotes soft aggregation, not a uniquely optimal layer. Inputs can share an argmax while differing in full weights: FACED samples have w9=0.07967±0.00176w_9=0.07967\pm0.00176 (Table E2). Teacher-derived ql(x)∝exp⁡(slT(x)/τq)q_l(x)\propto\exp(s_l^T(x)/\tau_q) supplies additional guidance; independent probes assess informativeness.

b. All 18 settings. Numbers denote layer indices. Router rankings/weights use validation-sample means; probes use validation ACC-B. ρ\rho averages per-sample spectral-score/router rank correlations.

Dataset / teacherNo-prior top-1 (weight %)Prior top-3 (top-1 weight %)Probe top-3ρ\rho (prior / uniform)
FACED / CBraMod0 (100.00)9,8,10 (8.59)9,8,10+0.505 / +0.493
FACED / CSBrain0 (100.00)12,11,3 (7.69)12,13,11+0.496 / +0.451
FACED / LaBraM0 (93.21)12,11,9 (9.32)12,11,7+0.181 / -0.426
mental-arithmetic / CBraMod7 (99.89)9,8,6 (7.79)12,13,9+0.183 / +0.079
mental-arithmetic / CSBrain7 (99.98)10,11,12 (7.96)6,7,8+0.200 / +0.200
mental-arithmetic / LaBraM0 (53.66)12,11,10 (9.64)12,11,8-0.126 / -0.450
Mumtaz2016 / CBraMod3 (92.85)12,8,7 (8.10)12,13,11+0.231 / +0.205
Mumtaz2016 / CSBrain0 (94.46)12,11,2 (8.04)5,1,6-0.006 / +0.008
Mumtaz2016 / LaBraM0 (95.30)9,3,10 (8.47)7,9,8+0.508 / +0.207
PhysioNet-MI / CBraMod0 (99.88)9,10,8 (7.21)9,12,10+0.736 / +0.650
PhysioNet-MI / CSBrain0 (99.28)11,10,12 (7.45)12,13,11+0.353 / +0.211
PhysioNet-MI / LaBraM0 (79.22)12,0,11 (8.35)12,11,9+0.903 / -0.385
SHU-MI / CBraMod9 (100.00)7,9,11 (7.59)12,10,11+0.347 / +0.361
SHU-MI / CSBrain0 (99.97)11,10,5 (7.61)12,13,8+0.259 / +0.271
SHU-MI / LaBraM0 (80.24)12,11,10 (10.33)12,11,10-0.348 / -0.392
Siena / CBraMod1 (99.88)13,12,11 (12.19)12,13,5+0.368 / +0.172
Siena / CSBrain5 (30.50)12,13,11 (12.66)12,13,11+0.069 / +0.043
Siena / LaBraM11 (58.73)0,1,11 (9.17)12,0,1+0.604 / -0.687
  1. Collapse: without the prior, 13/18 concentrate at least 90% mean weight on one layer; only one collapsed case selects a probe top-3 layer. With the prior, none collapses (maximum 12.66%).
  2. Independent probing: the prior's highest-weight layer is probe top-3 in 14/18; router/probe rankings correlate positively in 15/18.
  3. Guidance alignment: mean sample-wise ρ\rho is +0.304 versus +0.056 under Uniform Prior, higher in 14/18 (one tie; three small decreases).
  4. Predictive effect: removing the prior costs 3.78 ACC-B points on FACED and 3.92 on SHU-MI.

Table E5/Fig. E1 report validation analyses; Tables E1-E4 describe training dynamics.

Q5: Accuracy-efficiency improvements

Thank you for raising this concern. DLink-S achieves the highest mean ACC-B on all six datasets among the evaluated compact-model and KD baselines. We highlight this accuracy-efficiency trade-off in Figures 1 and 5, add parameter counts to Tables 1/2/A5, and provide CPU inference latencies in Appendix Table G1.

Using six-dataset mean parameter counts, DLink-S uses 51.2% of EEGConformer's parameters and achieves a 134.5-fold reduction relative to CBraMod (0.0953M vs. 12.817M). These comparisons highlight improved compact-student accuracy at a substantially smaller parameter budget.

Q6: Fine-tuned teacher performance

Test ACC-B is mean ±\pm SD over five seeds; gap = CBraMod FT minus DLink-S.

DatasetCBraMod FT (%)DLink-S (%)Gap (pp)
FACED51.46 ±\pm 1.4843.46 ±\pm 0.808.00
mental-arithmetic63.33 ±\pm 1.8662.21 ±\pm 5.141.12
Mumtaz201690.64 ±\pm 1.3690.17 ±\pm 1.530.47
PhysioNet-MI61.51 ±\pm 0.3460.83 ±\pm 0.620.68
SHU-MI62.72 ±\pm 0.8062.89 ±\pm 1.17-0.17
Siena78.94 ±\pm 1.6678.79 ±\pm 2.630.15

On FACED, DLink-M narrows the gap to 1.65 points with a 26.1-fold parameter reduction relative to CBraMod (0.678M vs. 17.72M parameters). Table 2 shows mean ACC-B gains with all three teachers at both student capacities.

AuthorsOpenReview ↗

# Response to Reviewer 2yvm(3/3)

Q4: Frequency-domain equivalence and compression

Thank you for raising this question. We address the claimed equivalence between frequency-domain and time-domain matching from three perspectives: mathematical formulation, motivation under compression, and empirical comparisons.

a. Mathematics: DLink's magnitude-phase loss differs from pointwise feature L2. Parseval's identity establishes equivalence between time-domain squared L2 and squared L2 applied directly to complex Fourier coefficients. For aligned student features fSpf_S^p, routed teacher features fT∗f_T^{\ast}, and a unitary full DFT,

∥FN(fSp)−FN(fT∗)∥F2=∥fSp−fT∗∥F2.\lVert\mathcal{F}_N(f_S^p)-\mathcal{F}_N(f_T^{\ast})\rVert_F^2=\lVert f_S^p-f_T^{\ast}\rVert_F^2.

DLink computes separate losses on spectral magnitudes and sine/cosine-encoded phases. These nonlinear mappings define a different discrepancy from direct complex-coefficient L2. DLink's magnitude-phase objective is therefore not generally equivalent to pointwise time-domain feature L2 under Parseval's identity.

b. Method motivation: spectral supervision before feature downsampling. The compact student reduces temporal-latent resolution during compression. Without sufficient attenuation, components above the downsampled Nyquist limit can fold into lower frequencies, causing aliasing. This motivates aligning representation spectra before compression, following the feature-level anti-aliasing motivation of Zhang (2019), Making Convolutional Networks Shift-Invariant Again.

c. Empirical evidence: lower spectral distortion accompanies higher student accuracy. We compare DLink, pointwise Routed L2, and non-Fourier Temporal Difference under the same student, teacher, and router setup. Temporal Difference uses MSE(FS,FT∗)+MSE(ΔsFS,ΔsFT∗)\mathrm{MSE}(F_S,F_T^{\ast})+\mathrm{MSE}(\Delta_sF_S,\Delta_sF_T^{\ast}) to incorporate temporal structure without Fourier-domain matching.

FACED objectiveACC-B (%)RhighR_{\mathrm{high}} (%)DspecD_{\mathrm{spec}}
Routed L240.64 ±\pm 0.4036.45 ±\pm 2.721.8993 ±\pm 0.0044
Temporal Difference41.29 ±\pm 0.5131.92 ±\pm 2.881.8763 ±\pm 0.0170
DLink43.46 ±\pm 0.8024.21 ±\pm 0.561.5790 ±\pm 0.0169

RhighR_{\mathrm{high}} is above-cutoff energy; DspecD_{\mathrm{spec}} is mapped spectral MSE. Both use test samples and validation-selected checkpoints: sample means within each seed, then five-seed mean/SD (Table F4).

Across the three objectives, lower distortion accompanies higher method-mean ACC-B. Against Temporal Difference, DLink reduces DspecD_{\mathrm{spec}} by 0.2973 and high-frequency energy by 7.71 points, gaining 2.17 ACC-B points. SHU-MI ACC-B is 62.89%, versus Temporal Difference/Routed L2 at 61.39%/61.04%. Table F2 additionally isolates magnitude, phase, and ordering.

Q7: Related work and additional compression experiments

Thank you for suggesting a broader discussion of related work. Sections 5.1-5.2 already discussed knowledge distillation, including feature/logit transfer, layer-wise transfer, and frequency-aware KD. We have now added a discussion of model compression, including SmoothQuant (Xiao et al., 2023) and DepGraph (Fang et al., 2023).

The large parameter and computation gap between EEG foundation models and compact EEG students makes aggressive compression challenging. The maximum feasible extent of structured pruning depends on the architecture, dependencies between layers, and minimum channel constraints; increasing the requested pruning rate therefore need not yield a model as small as the target student. Quantization can reduce weight-storage bytes without reducing the parameter count. These constraints motivate evaluating achieved parameter counts and accuracy together.

We added DepGraph experiments. Rates are requested settings; 99.99% uses an eight-channel minimum. Counts describe achieved models; ACC-B is mean ±\pm SD.

DatasetMethod / settingParams (M)ACC-B (%)Params / DLink-S
FACEDDepGraph 90%1.891040.50 ±\pm 1.989.21x
FACEDDepGraph 99.99%, min 81.584927.51 ±\pm 0.707.72x
FACEDDLink-S0.205243.46 ±\pm 0.801x
SHU-MIDepGraph 90%1.931260.52 ±\pm 1.3228.56x
SHU-MIDepGraph 99.99%, min 81.277560.07 ±\pm 0.8418.90x
SHU-MIDLink-S0.067662.89 ±\pm 1.171x

DLink-S has higher ACC-B with fewer parameters in both tested settings. The 99.99% request still retains 7.72x/18.90x its parameters on FACED/SHU-MI. Quantization reduces bytes per weight; pruning respects structural/channel constraints. DLink directly trains a compact model and complements both.

Dear Reviewer 2yvm,

Thank you again for your time and constructive comments on our paper. As the discussion period is closing soon, we wanted to send a brief, gentle reminder regarding our response and updated manuscript. We hope our rebuttal has addressed your concerns and would greatly appreciate it if you could consider them in your final assessment. Thank you!

Reviewer_7fhyOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes DLink, a knowledge-distillation framework designed to transfer representations from pretrained EEG foundation models into compact student architectures.

The authors motivate their method through two observations: first, pretrained EEG foundation models can impose substantial computational overhead during inference; second, task-discriminative information may be distributed across intermediate network layers rather than concentrated in the final representation.

DLink addresses these observations through three components. An input-conditioned layer router assigns weights to intermediate teacher representations. A representation-spectrum distillation objective aligns the magnitude and phase spectra of teacher and student features along their temporal-latent dimensions. Finally, a project-then-compress student architecture transfers the distilled information into a lightweight prediction model.

The teacher and router are discarded after training, leaving only the student for inference.

The proposed method is evaluated across six EEG benchmarks covering emotion recognition, mental arithmetic, depression classification, motor imagery, and seizure prediction. The experiments compare DLink against supervised lightweight networks and several existing knowledge-distillation methods. Additional analyses investigate layer routing, spectral objectives, student capacity, and inference efficiency.

Critical strengths

S1. A practical distillation framework that removes the teacher from the inference pipeline.

S2. Reasonably extensive comparisons and component-level analysis.

Critical weaknesses

W1. The proposed spectral-concentration prior is not convincingly justified as a measure of task-discriminative teacher knowledge.

A central methodological claim is that DLink identifies and transfers dominant knowledge from different layers of an EEG foundation model.

However, the spectral-concentration score used to guide the router measures how concentrated a teacher representation's Fourier energy is across its temporal-latent frequency components. The manuscript does not establish why this statistic should reliably correspond to the amount of task-relevant or transferable information in a representation.

A representation with highly concentrated spectral energy may simply be smooth, redundant, or dominated by low-frequency variation. Conversely, important discriminative information may reside in relatively weak or broadly distributed components.

Therefore, spectral concentration is not intrinsically a measure of feature quality, semantic importance, or class-discriminative information.

The authors provide an illustrative experiment in which the router's preferred layer coincides with a strong linear-probe layer, but this is insufficient to establish a general relationship across tasks, teachers, and datasets.

The reported ablation results also warrant a more cautious interpretation. For example, on FACED, the complete DLink method achieves 43.46% balanced accuracy, compared with 43.10% for fixed best-validation layer selection and 42.73% for the Mean Power routing variant. These margins are modest relative to the reported variation.

Moreover, the router is optimized jointly with the student and supervised classification objective. It is therefore unclear whether the learned layer preferences arise from the proposed spectral prior or are largely determined by downstream supervision.

This issue is important because the identification of dominant teacher knowledge is presented as a central scientific contribution rather than simply an implementation choice.

W2. The advantages of representation-spectrum matching are insufficiently isolated, and the overall improvements are not uniformly compelling.

The proposed magnitude-and-phase distillation objective is motivated by the possibility that student compression distorts the spectral structure of teacher representations.

However, the paper does not provide a sufficiently direct demonstration that the measured representation-spectrum distortion is responsible for downstream prediction errors or that reducing this distortion reliably preserves task-relevant information.

The Fourier transform is applied along a learned temporal-latent dimension rather than directly to the original EEG signal. Consequently, the resulting frequencies do not necessarily correspond to physiological rhythms or uniquely meaningful temporal structures.

Although the authors acknowledge this distinction, the connection between preserving these spectra and improving EEG decoding remains primarily empirical.

The ablations comparing magnitude-only, phase-only, and shuffled-ordering objectives provide some support for the full objective, but the observed performance differences are relatively modest.

For example, Appendix F.6 reports FACED balanced accuracies of 42.34%, 42.39%, and 42.86% for magnitude-only, phase-only, and shuffled-ordering controls, respectively, compared with 43.46% for DLink. The relatively small difference between the complete objective and the shuffled-ordering control raises questions about the extent to which physically meaningful temporal ordering is essential to the method.

Furthermore, the overall performance advantage is uneven across datasets. On FACED, DLink improves balanced accuracy over the matched supervised student from 41.61% to 43.46%. On mental arithmetic, the increase is only from 60.90% to 62.21%, while AUROC changes from 69.29% to 71.49% with substantial variation.

On several tasks, independently trained lightweight architectures remain competitive, and some alternative methods achieve stronger secondary metrics.

The results therefore establish that DLink can improve compact student training, but they do not conclusively demonstrate that its specific spectral formulation provides a robust advantage over simpler intermediate-feature distillation and regularization approaches.

Questions for authors

Q1. What theoretical or empirical evidence supports spectral concentration as a proxy for task-discriminative teacher information? Can the authors report correlations between spectral-concentration scores, layer-wise linear-probe accuracy, and actual student improvement across all evaluated datasets and teacher models?

Q2. How much does the input-conditioned router improve over a simple learned global layer weighting or a fixed best-validation-layer strategy when all alternatives are evaluated with matched computational budgets and optimization procedures? Are the improvements statistically significant across datasets?

Q3. Can the authors directly quantify the relationship between representation-spectrum distortion and downstream prediction quality? In particular, does a reduction in the proposed spectral discrepancy consistently predict better student accuracy, and how does the method compare with alternative feature-alignment objectives that preserve temporal structure without Fourier-domain phase matching?

Q4. Does DLink consistently improve the accuracy–efficiency Pareto frontier across different student capacities and teacher architectures? Can the authors provide additional matched-parameter comparisons, preferably with confidence intervals for both accuracy and efficiency, to establish that its advantages are not specific to the proposed student architecture?

5 comments
AuthorsOpenReview ↗

# Response to Reviewer 7fhy (1/4)

Thank you very much for your time and effort in reviewing our paper. We sincerely appreciate your feedback. We briefly clarify two points.

  1. Overall performance should be considered alongside model size and measured inference latency. DLink-S achieves stronger performance across all six benchmarks with a 134.5-fold parameter reduction relative to CBraMod (0.0953M vs. 12.817M, based on six-dataset means), and uses 51.2% of EEGConformer's parameters. Figures 1 and 5 visualize the accuracy-efficiency comparison across six datasets. Parameter counts are also reported in the results tables, with detailed CPU latency and model profiles in Appendix Table G1.

  2. Several comparisons raised in Q2–Q4 were already included in the original submission: the Fixed Best-Val and Global Learnable/Uniform Prior routing baselines (Table 3 and Table F2), temporal feature distillation (Routed L2, Table 3), and distillation with a Conformer student (Tables A6–A7). To address these concerns more fully, we expanded the ablations, added a non-Fourier temporal control experiment in Table F4, and provide statistical analyses below.

W2 & Q4: Pareto Frontier Across Capacities & Architecture-Agnostic Distillation

DLink changes the training supervision while retaining the student's inference architecture. Its accuracy gains therefore incur no additional inference parameters or routing computation at a fixed student configuration. We address the capacity, teacher, and architecture questions from the following perspectives:

a. Consistency across student capacities and teacher architectures. Table 2 evaluates two student capacities with three teacher architectures on FACED. All configurations within each capacity share the same inference architecture: 0.2052M parameters for DLink-S and 0.6780M for DLink-M, corresponding to 86.3-fold and 26.1-fold parameter reductions relative to the 17.72M-parameter CBraMod teacher. Both tables below report ACC-B means and 95% confidence intervals across five seeds.

Teacher / trainingDLink-S ACC-B (%) [95% CI]DLink-M ACC-B (%) [95% CI]
Supervised, no KD41.61 [40.67, 42.55]47.35 [45.70, 49.00]
CBraMod43.46 [42.47, 44.45]49.81 [48.44, 51.18]
CSBrain43.48 [42.66, 44.30]49.62 [48.80, 50.44]
LaBraM42.28 [40.86, 43.70]49.20 [47.60, 50.80]
  1. Improvements across all six capacity-teacher combinations. DLink improves mean ACC-B over supervised training by 0.67-1.87 points for DLink-S and 1.85-2.46 points for DLink-M, without increasing inference cost within either capacity.

  2. Accuracy-size trade-offs across capacities. On FACED, DLink-S reaches 43.46% ACC-B with 0.2052M parameters, compared with EEGConformer's 43.24% with 0.2180M. DLink-M reaches 49.81% with 0.6780M parameters, a 26.1-fold parameter reduction relative to CBraMod. Across all six benchmarks, DLink-S has higher mean ACC-B than EEGConformer while using 51.2% of its mean parameter count. Figures 1 and 5 visualize the accuracy-efficiency comparison; Appendix Table G1 provides the detailed per-dataset CPU latency profiles.

b. Matched-parameter comparisons with a different student architecture. The original Tables A6-A7 already evaluate EEGConformer as the student. We further extend this comparison to seven KD baselines on FACED. All methods use the same EEGConformer student with 0.218M parameters, directly testing whether DLink's advantage extends beyond the proposed student architecture.

MethodACC-B (%)95% CI
Supervised, no KD43.24[40.91, 45.57]
RKD40.90[39.63, 42.17]
FreeKD44.80[42.37, 47.23]
ReviewKD45.17[43.12, 47.22]
Logit-std45.18[43.53, 46.83]
CRD45.19[43.76, 46.62]
FitNets45.45[44.07, 46.83]
SA-KD45.49[44.53, 46.45]
DLink46.58[45.92, 47.24]
  1. Higher accuracy at identical inference cost. DLink improves ACC-B by 3.34 points over supervised EEGConformer and 1.09 points over the strongest KD baseline, SA-KD.

  2. Benefits extend across student architectures. Together with the two-capacity results, this matched-backbone comparison supports DLink's effectiveness beyond the proposed compact student.

AuthorsOpenReview ↗

# Response to Reviewer 7fhy (2/4)

W1 & Q1: Spectral-Concentration Prior and Correlation Analysis

To clarify the role and effectiveness of the spectral-concentration prior, we provide the following theoretical rationale and empirical evidence:

a. Theoretical rationale

In DLink, spectral concentration provides a sample-dependent heuristic routing signal, used jointly with classification and distillation supervision. For spectral-concentration scores sls_l and guidance distribution ql∝exp⁡(sl/τq)q_l\propto\exp(s_l/\tau_q), the prior loss becomes

Lprior=−H(w)−1τq∑lwlsl+log⁡Z.\mathcal{L}_{\mathrm{prior}} = -H(w) - \frac{1}{\tau_q}\sum_l w_l s_l + \log Z.

where Z=∑lexp⁡(sl/τq)Z=\sum_l\exp(s_l/\tau_q). It combines entropy regularization that discourages routing collapse with sample-dependent guidance toward spectrally concentrated teacher layers. Uniform Prior retains only the entropy term. Routing is jointly optimized with the classification and distillation objectives.

b. Empirical evidence

(i) Association with prediction correctness and layer informativeness

A layer may have high mean spectral concentration for both correctly and incorrectly classified samples. To examine whether concentration is associated with prediction success, we introduce the Spectral Concentration Gap (SCG), the difference between these two groups within the same layer:

SCGl=E[sl(X)∣y^l(X)=y]−E[sl(X)∣y^l(X)≠y].\mathrm{SCG}_l = \mathbb{E}[s_l(X)\mid\hat y_l(X)=y] - \mathbb{E}[s_l(X)\mid\hat y_l(X)\neq y].

Here, y^l(X)\hat y_l(X) is the prediction at layer ll. We compare SCG and linear-probe ACC-B rankings across all layers.

DatasetSCG Top-3 layersProbe Top-3 layers (ACC-B)SCG–Probe Spearman ρ\rho
FACEDL9L_9, L4,L5L_4, L_5L9L_9, L8,L10L_8, L_{10} (46.39%)+0.200
PhysioNet-MIL12,L9L_{12}, L_9, L13L_{13}L9,L12L_9, L_{12}, L10L_{10} (54.82%)+0.578
SienaL6L_6, L12,L13L_{12}, L_{13}L12,L13L_{12}, L_{13}, L5L_5 (60.68%)+0.547
Mumtaz2016L12,L13,L11L_{12}, L_{13}, L_{11}L12,L13,L11L_{12}, L_{13}, L_{11} (99.48%)+0.657
mental-arithL9,L12,L13L_9, L_{12}, L_{13}L12,L13,L9L_{12}, L_{13}, L_9 (79.09%)+0.815
SHU-MIL10,L12L_{10}, L_{12}, L13L_{13}L12,L10L_{12}, L_{10}, L11L_{11} (68.29%)+0.415

All 18 settings: Top-3 sets overlap in 14/18 settings (30/54 shared positions); mean SCG–Probe Spearman ρ=+0.371\rho=\mathbf{+0.371}, positive in 15/18 settings.

(ii) Routing behavior and alignment with informative layers

We compare mean validation router weights with and without the prior against probe rankings, and sample-wise spectral-score–router correlations against Uniform Prior.

DatasetNo-prior top layer (mean validation weight)Probe ACC-B (rank)Prior top-3 layers (top-layer mean validation weight)Probe top-3 layers (ACC-B)Sample-wise ρˉ\bar{\rho} (prior / uniform)
FACEDL0L_0 (100.00%)39.83% (14/14)L9,L8,L10L_9, L_8, L_{10} (8.59%)L9,L8,L10L_9, L_8, L_{10} (46.39%)+0.505 / +0.493
PhysioNet-MIL0L_0 (99.88%)44.53% (14/14)L9,L10L_9, L_{10}, L8L_8 (7.21%)L9L_9, L12L_{12}, L10L_{10} (54.82%)+0.736 / +0.650
SienaL1L_1 (99.88%)54.96% (13/14)L13,L12L_{13}, L_{12}, L11L_{11} (12.19%)L12,L13L_{12}, L_{13}, L5L_5 (60.68%)+0.368 / +0.172
Mumtaz2016L3L_3 (92.85%)89.30% (11/14)L12L_{12}, L8,L7L_8, L_7 (8.10%)L12L_{12}, L13,L11L_{13}, L_{11} (99.48%)+0.231 / +0.205
mental-arithL7L_7 (99.89%)61.16% (8/14)L9L_9, L8,L6L_8, L_6 (7.79%)L12,L13L_{12}, L_{13}, L9L_9 (79.09%)+0.183 / +0.079
SHU-MIL9L_9 (100.00%)67.42% (5/14)L7,L9L_7, L_9, L11L_{11} (7.59%)L12,L10L_{12}, L_{10}, L11L_{11} (68.29%)+0.347 / +0.361
All 18 settings13/18 collapsed (≥90%\geq90\%)1/18 in probe top-30/18 collapsed (max 12.66%)14/18 top-3 matchesHigher in 14/18 (p=0.0012p=0.0012)

In both tables, the six dataset rows report CBraMod results, while the summaries cover all 18 dataset–teacher settings. Bold layers indicate overlap between the two top-three sets being compared; parenthesized probe ACC-B denotes the best layer's accuracy.

AuthorsOpenReview ↗

# Response to Reviewer 7fhy (3/4)

W1 & Q1: Summary of Evidence and Conclusions (continued)

Together, the SCG analysis, routing analysis, and downstream ablations provide complementary evidence:

  1. Association with layer informativeness. SCG and probe rankings are positively correlated in 15/18 settings (mean ρ=+0.371\rho=\mathbf{+0.371}). The two top-three sets overlap in 14/18 settings (30/54 shared layer positions). Thus, higher-performing layers tend to have larger correct-minus-incorrect concentration gaps. Together with the routing analyses and downstream ablations below, these findings support spectral concentration as a heuristic proxy for task-discriminative teacher information.

  2. Routing stability and informative layer selection. Without the prior, 13/18 settings assign over 90% of the mean routing weight to one layer; among these collapsed cases, that layer appears in the probe top three only once. With the prior, no setting collapses (maximum single-layer mean weight: 12.66%), and the highest-weight layer falls within the probe top three in 14/18 settings. Router-weight and probe rankings are also positively correlated in 15/18 settings.

  3. Sample-dependent guidance. The mean sample-wise correlation between spectral-concentration scores and router weights is higher with the spectral prior than with Uniform Prior (+0.304 vs. +0.056; paired Wilcoxon p=0.0012p=0.0012), with improvements in 14/18 settings. This measures alignment with spectral guidance, complementing the SCG–probe and router–probe comparisons above.

  4. Downstream effectiveness. Removing Lprior\mathcal{L}_{\mathrm{prior}} reduces ACC-B by 3.78 percentage points on FACED and 3.92 points on SHU-MI, supporting its contribution to student performance.

The SCG analysis is included in Appendix Table E7; the routing analyses are included in Table E5 and Fig. E1.

W1 & Q2: Matched-Budget Routing Controls & Statistical Significance Across Datasets

To assess whether DLink’s gains extend beyond FACED and depend on its routing design, we clarify the controls and provide results across all six datasets.

a. Comparison rationale. Global Learnable and Uniform Prior were already included in Table F2, and Fixed Best-Val was reported in Table 3. Fixed Best-Val tests whether selecting a single layer by validation performance is sufficient, while Uniform Prior retains input-dependent routing and replaces spectral guidance with a uniform target. These comparisons use the same student architecture and inference budget.

b. Extended results. We extend the Fixed Best-Val and Uniform Prior comparisons to all six datasets. The table reports ACC-B means, standard deviations, 95% confidence intervals of the means, and paired comparisons. We use one-sided exact sign tests without multiple-comparison correction to assess the consistency of seed-level improvements. W-L-T denotes wins, losses, and ties for DLink.

BenchmarkDLink (Full) ACC-B [95% CI]Fixed Best-Val ACC-B [95% CI]Δ\Delta (pp; W-L-T; psignp_{\mathrm{sign}})Uniform Prior ACC-B [95% CI]Δ\Delta (pp; W-L-T; psignp_{\mathrm{sign}})
FACED43.46 ±\pm 0.80% [42.47, 44.45]43.10 ±\pm 0.97% [41.90, 44.30]+0.36; (3-2-0; p=0.5000p=0.5000)42.83 ±\pm 0.81% [41.82, 43.84]+0.63 (5-0-0; p=0.0313p=0.0313)
PhysioNet-MI60.83 ±\pm 0.62% [60.06, 61.60]58.63 ±\pm 0.68% [57.78, 59.47]+2.20 (5-0-0; p=0.0313p=0.0313)58.77 ±\pm 0.96% [57.58, 59.97]+2.06 (5-0-0; p=0.0313p=0.0313)
SHU-MI62.89 ±\pm 1.17% [61.44, 64.34]56.81 ±\pm 0.93% [55.66, 57.96]+6.08 (5-0-0; p=0.0313p=0.0313)61.63 ±\pm 0.88% [60.53, 62.73]+1.26 (5-0-0; p=0.0313p=0.0313)
Siena78.79 ±\pm 2.63% [75.52, 82.06]67.02 ±\pm 2.73% [63.63, 70.40]+11.77 (5-0-0; p=0.0313p=0.0313)63.77 ±\pm 1.85% [61.48, 66.07]+15.02 (5-0-0; p=0.0313p=0.0313)
Mumtaz201690.17 ±\pm 1.53% [88.27, 92.07]88.89 ±\pm 1.24% [87.35, 90.42]+1.28 (5-0-0; p=0.0313p=0.0313)89.69 ±\pm 0.15% [89.50, 89.88]+0.48 (3-2-0; p=0.5000p=0.5000)
mental-arith62.21 ±\pm 5.14% [55.83, 68.59]59.65 ±\pm 4.41% [54.18, 65.13]+2.56 (5-0-0; p=0.0313p=0.0313)60.63 ±\pm 4.23% [55.37, 65.88]+1.58 (3-1-1; p=0.3125p=0.3125)
  1. Higher mean ACC-B across all six datasets. DLink outperforms both controls in mean ACC-B on every dataset. The gains extend beyond FACED.

  2. Consistent improvements across seeds. Among the reported paired comparisons, the sign test favors DLink at the unadjusted 0.05 level in 5/6 comparisons against Fixed Best-Val and 4/6 against Uniform Prior. Each significant comparison has five wins and no losses (p=0.0313p=0.0313).

Comments on the paper

1 comment
AuthorsOpenReview ↗

# Summary of the Rebuttal

Dear Area Chair,

Thank you very much for your time and dedicated service in shepherding our submission. Reviewer 9Pxs engaged across two rounds, confirmed our rebuttal addressed their concerns, and raised their rating to 3. Reviewers 7fhy and 2yvm have not yet responded. Notably, none of the reviewers questioned our novelty, and the reviews recognized several strengths: a practical, teacher-free inference framework (7fhy S1; 2yvm S1), well-written presentation with a clear motivation and sound intermediate-layer premise (9Pxs; 2yvm S2), and rigorous experiments with comprehensive ablations (7fhy S2; 2yvm S3–S4).

Core Contributions & Recognized Strengths

Building on these recognized strengths, DLink eliminates EFM inference overhead while exploiting intermediate-layer knowledge via three coupled designs: (1) input-conditioned layer routing to dynamically aggregate dominant intermediate layers, (2) pre-compression spectral distillation to preserve feature structure during downsampling, and (3) a project-then-compress student that discards both teacher and router at inference. Across 6 benchmarks, DLink-S achieves the best mean ACC-B among compact/KD baselines with 134.5× fewer parameters than CBraMod (0.0953M vs. 12.817M), 51.2% of EEGConformer's size, and 3.53 ms mean CPU latency (vs. 21.16 ms CBraMod / 6.30 ms EEGConformer; Figs. 1, 5; Table G1).

Clarifications & New Evidence Addressing Reviewers 7fhy & 2yvm

1. Key Clarifications (Existing Controls & Methodological Scope)

  • Key controls (Fixed Best-Val, Fixed Avg, Uniform Prior, Global Learnable, Routed L2, ReviewKD, S-A KD), 18 dataset–teacher settings (Table D1), and EEGConformer student results (Tables A6–A7) were already in the original submission (7fhy Q2–Q4; 2yvm W1).
  • The router performs input-conditioned soft aggregation rather than hard single-layer selection (2yvm W1–W2; 7fhy W1). Spectral KD operates on learned temporal-latent features before compression (not raw EEG Hz) and is mathematically non-equivalent to time-domain L2 under Parseval's identity due to separate nonlinear losses on magnitudes and sin/cos-encoded phases (2yvm W4; 7fhy W2).

2. Spectral Prior & Dynamic Layer Routing (7fhy W1/Q1–Q2; 2yvm W1–W3)

  • 18-setting diagnostics: Our new Spectral Concentration Gap (SCG, Table E7) correlates positively with linear-probe rankings in 15/18 settings (mean ρ=+0.371; top-3 overlap 14/18). Without the prior, 13/18 settings collapse (≥90% weight on one layer; probe top-3 only 1/18). With it, 0/18 collapse (max 12.66%), top-1 hits probe top-3 in 14/18, and sample-wise spectral–router correlation beats Uniform Prior (+0.304 vs. +0.056, higher in 14/18; Table E5, Fig. E1).
  • Why small weight differences matter: A new Batch-mean routing control drops ACC-B by 3.34 pp (FACED) and 2.21 pp (SHU-MI). Despite a 0.00329 mean layer-weight difference from Fixed Avg (cosine 0.9980), final student parameters and updates diverge sharply (cosine 0.3104 / 0.1583).
  • 6-dataset significance: DLink wins mean ACC-B on 6/6 datasets, with exact sign-test significance (5/5 seeds) in 5/6 vs. Fixed Best-Val (up to +11.77 pp Siena, +6.08 pp SHU-MI) and 4/6 vs. Uniform Prior (up to +15.02 pp).

3. Pre-Compression Spectral KD vs. Non-Fourier Control (7fhy W2/Q3; 2yvm W4)

  • Added a non-Fourier Temporal Difference control (Table F5). DLink beats Temporal Difference and Routed L2 on FACED (43.46% vs. 41.29%/40.64%) and SHU-MI (62.89% vs. 61.39%/61.04%), while lowering above-cutoff energy R_high (24.21% vs. 31.92%/36.45%) and spectral MSE D_spec (1.5790 vs. 1.8763/1.8993), supporting pre-compression anti-aliasing.

4. Teacher Gaps, Generalization & Pruning Baselines (7fhy Q4; 2yvm W5–W7)

  • DLink-S stays within 0.15–1.12 pp of fine-tuned CBraMod on 4 tasks and surpasses it on SHU-MI (+0.17 pp); DLink-M narrows the FACED gap to 1.65 pp (26.1× reduction). Against DepGraph pruning (90%/99.99%), even 99.99% retains 7.72×/18.90× more parameters than DLink-S with lower ACC-B.
  • Added REVE as a 4th teacher (+1.51/+2.57 pp on FACED; all 4 teachers improve both capacities) and 7 KD baselines on a matched EEGConformer student (0.218M), where DLink reaches 46.58% (+3.34 pp vs. supervised; +1.09 pp vs. best KD baseline SA-KD).

We believe the revised manuscript and additional experiments substantiate DLink’s methodological contribution and practical value for efficient EEG decoding. We respectfully ask you to consider the full evidence in your independent assessment and support acceptance of our paper. Thank you for your time and careful consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.