- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
MoRSE trains a shared 1.97M-parameter EEG student across eight tasks while randomly varying the available electrode subset. Full-montage teachers provide task logits and token representations, with feature alignment taken from a never-fine-tuned backbone. The method combines labeled multi-task training with alignment on an additional unlabeled corpus. Evaluation includes 30 task-by-montage cells, held-out hardware layouts, comparisons with montage-sampled specialists, and two external sleep cohorts. The experiments measure accuracy, aggregate parameter storage, and performance when electrodes are removed. Its extensive ablations locate most of the distillation benefit in feature alignment rather than logit transfer.
Critical strengths
-
A practically meaningful robustness and consolidation result, tested against relevant alternatives. The same student handles eight tasks and multiple electrode budgets, with mean balanced accuracy of 0.6044 versus 0.6335 for the LaBraM per-cell specialists. The paper also compares against eight montage-sampled specialists, a stronger and more relevant baseline than specialists trained only at fixed montages. On 42 hardware-layout cells, montage sampling improves the student from 0.5143 to 0.5670. The unchanged sleep head also transfers to ISRUC and PhysioNet 2018; the largest sampling gains occur with two electrodes, where training electrodes are missing. These experiments directly address the claimed deployment problem.
-
Extensive, informative ablations and transparent negative findings. The appendix distinguishes teacher family from student initialization, separates feature alignment from additional unlabeled data, compares multi-task and single-task students, and reports unsuccessful interventions. Table 15 establishes that feature alignment helps even without the auxiliary corpus, while explicitly treating logit distillation's incremental gain as indistinguishable from zero. This level of analysis substantially increases the value of the work beyond presenting a compact model.
Critical weaknesses
-
The teacher-role explanation is stronger than the causal evidence for it. The abstract motivates never-fine-tuned alignment features through the effective-rank reduction after task fine-tuning, and the recipe is described as distillation from fine-tuned task experts. However, effective rank tracking class count does not by itself show that those representations are unsuitable alignment targets. The direct experiment needed for that explanation is a matched feature-alignment teacher comparison before versus after task fine-tuning, holding the teacher family, student, corpus, and losses fixed. Separately, Table 15 gives 0.6016 for feature-only training and 0.6044 for the full recipe, with the logit increment carried by one seed; the authors appropriately acknowledge this. The evidence therefore supports asymmetric feature distillation with montage sampling more strongly than a necessary dual-teacher transfer mechanism. I would accept a narrower explanation consistent with these results; demonstrating the stronger teacher-role claim would require the matched control above.
-
Aggregate retention ratios conceal meaningful task-level costs, and seed variation does not establish patient-level reliability. Retaining approximately 95% of raw balanced accuracy can sound closer to specialist performance than the absolute drop or chance-adjusted headroom warrants. The appendix already supplies headroom analyses, which is a strength, but they should be reflected in the main result: some full-montage tasks lose appreciably more than the mean, while others benefit from sharing. The 89× figure also concerns aggregate stored parameters for the specialist collection, not an 89× reduction in the cost of one inference; the separate runtime measurements are the relevant evidence for that claim. Finally, paired tests across five seeds quantify training variability on fixed test cohorts, not uncertainty over patients or sites. Please elevate the existing per-task/headroom results and add participant-level paired uncertainty for the principal robustness and external-cohort comparisons where feasible. This would clarify when the compact shared model is an acceptable replacement rather than imply a uniform trade-off across tasks.
Questions for authors
-
Is there a matched comparison using the same alignment teacher before versus after task fine-tuning? If not, will the authors frame effective-rank observations as motivation and remove claims that the never-fine-tuned teacher is required by those observations?
-
Can the main results show absolute per-task losses and chance-adjusted headroom next to the aggregate retention ratio, together with paired subject-level uncertainty for the external sleep and held-out-layout comparisons?
-
What behavior is expected for electrodes outside the trained slot vocabulary, or for reference changes beyond those evaluated? Please distinguish unseen subsets/layouts of supported electrodes from arbitrary unseen electrode coordinates and acquisition systems. A clear limitation is sufficient; a new broad benchmark is not required.
4 comments
Response to Reviewer qUa7 (2/2): W2, Q2 and Q3
W2 / Q2. Per-task costs, stored parameters and subject-level uncertainty.
-
Table 1 now gives chance-adjusted retention per task next to the absolute per-task differences and the aggregate, and the abstract leads with the absolute gap (0.029 balanced accuracy and 90% of the headroom above chance against the per-cell specialists, 0.010 and 96% against the eight montage-sampled specialists).
-
89× now refers to stored parameters. Per-inference cost is reported separately in Table 11, which now adds CPU latency. On one CPU core a window takes 5.9 ms for MoRSE and 16.8 ms for a LaBraM specialist, so per inference MoRSE is about 2.9 times faster, far less than 89×, and the text keeps the two apart.
-
Subject-level paired intervals for the held-out layouts are in the table below (all tasks in Appendix P). For the external sleep cohorts, the sampling gain at two electrodes is +0.1267 [+0.1149, +0.1386] on ISRUC (82 subjects) and +0.1163 [+0.1072, +0.1254] on PhysioNet 2018 (150 subjects). At four and six electrodes the gains are small (+0.0017 to +0.0123). Their subject-level intervals exclude zero in three of the four cases, but their seed-paired intervals include zero in all four, so we describe them as small and not resolved across training seeds.
-
Five seeds describe training variability on fixed test cohorts, and Section 3.1 now says so.
| Comparison | Δ BA | Subject 95% CI |
|---|---|---|
| Student vs 8 sampled specialists, 42 layouts | −0.0255 | [−0.0452, −0.0046] |
| Sampling vs no sampling (student), 42 layouts | +0.0528 | [+0.0365, +0.0671] |
Q3. Electrodes outside the vocabulary and other references. We added the limitation in Appendix J.3. MoRSE is validated on unseen subsets and layouts of the electrodes in its training vocabulary and with the references of the training data. Its behaviour for arbitrary new electrode coordinates, other references and other acquisition systems is not established. Within that scope we added two controlled estimates (Appendix N). For electrode identities the student does not know, we withheld four electrodes from the student throughout training and mapped them at test time to trained sites at increasing distance (26 to 192 mm). The loss grows with distance, and leaving the electrodes out did better than mapping them (by 0.0095 to 0.0694 on the cells that contain them), so Appendix J.3 advises leaving such electrodes out. For references, re-referencing the test data to the common average or to a single electrode, without retraining, lowers balanced accuracy by 0.024 and 0.036 at the full montages and by 0.026 and 0.020 on the held-out layouts. These estimates cover sites of the 10-10 system and two reference changes, and Appendix J.3 does not extend them to new coordinates or acquisition systems.
We hope these results address your concerns, and we would be glad to clarify any remaining point.
Thank you for the detailed response and the revision. I'll keep my score
Thank you for your follow-up
Dear Reviewer qUa7,
Thank you very much for reading our responses and the revision, and for your continued positive assessment. Your review described the matched comparison and stated which narrower explanation you would accept, and the revised paper now reflects both. We sincerely appreciate the time and care you have given to our paper.
Best regards, The Authors
Response to Reviewer qUa7 (1/2): W1 and Q1
Thank you for the clear review and for stating which narrower explanation you would accept. We answer each weakness (W) and question (Q) below. Changes are in blue in the revised PDF.
W1. Narrower explanation. We adopt it. The submitted and revised wording, side by side:
Logit distillation stays in the recipe as specified but is no longer described as necessary, because its increment over alignment is not resolved (+0.0028, Table 15). Appendix G.2 reports its per-task values as exploratory.
Q1. Matched comparison. Yes. We aligned the student to the same Gram backbone before versus after task fine-tuning, holding the teacher family, student, corpus and losses fixed as you specified, and repeated the comparison in the feature-only configuration without the corpus (Section 3.5 and Appendix K). Neither resolves an advantage for the never-fine-tuned Gram (full recipe +0.0001 [−0.0117, +0.0119]; feature-only −0.0042 [−0.0143, +0.0060], and −0.0028 [−0.0105, +0.0049] with ten seeds), and for LaBraM, run in the same way, the estimate favours the fine-tuned teacher across seeds (−0.0082 [−0.0144, −0.0020]) but not across subjects or on validation. We also measured the effective rank of Gram itself before and after fine-tuning. It falls strongly on some tasks (Mumtaz 61 to 3.4, ADFTD 61 to 5.7) but not on others (HMC 53 to 54, CHB-MIT 64 to 61, SEED-V 56 to 67), while LaBraM falls on all eight (Table 12). A further test shows where the rank observation does and does not hold. Students aligned to fine-tuned teachers inherit their compression (student effective rank lower by 12 to 55, all fifteen seed pairs), yet neither accuracy on the eight tasks nor a linear probe on tasks not used in distillation shows a resolved cost (+0.0047 [−0.0086, +0.0179]). Following the rule we fixed before running the controls, effective rank is framed as motivation, and the claim that the observations require a never-fine-tuned teacher is removed.
Continued in the next comment (2/2).