- Rating
- 2
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes CtxSleepNet, a multimodal sleep staging framework designed to improve cross-domain generalization. The model combines CNN, GRU, Transformer, and LSTM to extract features from EEG and EOG signals and capture temporal dependencies across sleep epochs. The authors also introduce TypCert to select target subjects for fine-tuning under limited labeling budgets and a domain statistics alignment strategy to improve zero-shot performance.
Critical strengths
-
The experiments are comprehensive, including comparisons with existing methods, cross-dataset evaluations, ablation studies, and sensitivity analysis.
-
The proposed method achieves promising cross-domain performance, improving the average zero-shot accuracy from 68.2% to 74.4% with domain adaptation.
Critical weaknesses
-
The methodological novelty is limited. The backbone mainly combines existing CNN, GRU, Transformer, and LSTM modules. Similar designs for epoch-level feature extraction and sequence modeling have already been explored in previous sleep staging studies.
-
The model is relatively complex, but the paper lacks comparisons with baselines in terms of FLOPs, parameter counts, and inference time. Although the model achieves good performance, the additional computational cost should also be evaluated.
Questions for authors
The ablation studies could be more comprehensive. While the authors evaluate different backbone components, more detailed ablations of the domain adaptation strategy are needed. Since DSA has already been studied in previous work, the authors should clarify how much improvement comes from stage reweighting and recording-wise BN correction. Similarly, TypCert is only compared with random selection, without separately evaluating the contributions of Typicality and Certainty.
4 comments
Thank you for the clarification and additional experiments. While the new results address my concerns about ablations and computational complexity, my concern regarding limited methodological novelty remains. I will maintain my original rating of 2 (Weak Reject).
We thank the reviewer for the follow-up and are glad that the new ablations and the complexity analysis resolved those concerns. Our first reply focused on experiments, so we would like to state the methodological contribution and its theoretical basis more precisely.
What is new. The backbone modules are standard and are not our main claim. The contribution is a budget-aware way to adapt a sleep-staging model to a new dataset, with one recipe from zero to 10% labels instead of a separate tool per regime:
| Component | Closest prior work | What is new | Evidence |
|---|---|---|---|
| Recording-wise BN correction | AdaBN, DSA | Per-recording correction relative to source reference moments | +6.5 vs. DSA |
| Stage-aware normalization | Prior correction of outputs | Stage weights inside the BN moments | +1.4 at 0%, +1.6 at 1–2% |
| TypCert | Random or active-learning sampling | Label-free subject selection targeting | Fewest runs below zero shot |
| Budget-dependent scope | One scope for all budgets | Scope set by | +8.4 / +8.0 on ISRUC targets |
Theoretical foundation. Each design choice follows from a result in Section 3, and each prediction is confirmed by the experiments.
Proposition 1 (value of cross-epoch context). Using neighboring epochs lowers the optimal weighted risk by exactly , the label information that the context carries beyond the epoch itself, and Appendix B.1 shows that a learned sequence encoder realizes this gain once it exceeds its compression and fitting errors. The gain is large when a stage is ambiguous within one epoch, as N1 is, which motivates separating within-epoch encoding from cross-epoch modeling. Consistently, the Sequence module adds 6.0 points of N1 F1 (Table 5).
Theorem 1 (cohort bias and adaptation capacity). The error of the labeled-cohort gradient decomposes exactly into three terms: the target signal outside the trainable subspace, , the cohort bias, bounded by , and the variance, bounded by . The local risk-decrease corollary in Appendix B adds that an update lowers target risk whenever the target gradient in the trainable subspace exceeds bias plus variance. This yields the two label-dependent designs.
- TypCert targets the bias terms. Lemma B.1 bounds the true cohort–target stage mismatch by plus posterior-error terms, and each greedy step of the rank rule picks a candidate that no other candidate improves on both terms. Empirically, TypCert attains the smallest of all rules at 1% (0.55 vs. 0.63 for typicality, 0.60 for certainty, and 0.83 for random) and the fewest runs below zero shot (9 and 2 of 60 vs. 18 and 13 for random).
- The update scope follows the variance term. Updating the encoder pays off only when its additional target signal exceeds the added bias and variance, and the variance bound is large when a cohort contains one or two subjects. Hence normalization and classifier updates at 1–2%, which with TypCert exceed random full fine-tuning by 8.4 and 8.0 points on ISRUC targets, and full adaptation at 5–10%, which reaches 79.6% at 10%, 2.1 points below in-domain training.
Why it matters. Without labels or parameter updates, mean accuracy over six transfers rises from 68.2% to 74.4%. The adaptation is not tied to our backbone and adds +7.2 [6.6, 7.8] points to a backbone trained with the SleepDG objective. With the same inputs, splits, and ensemble, CtxSleepNet also exceeds all six Table 2 baselines retrained with our pipeline (87.3% vs. at most 85.6%, see our reply to Reviewer THRu).
We will rewrite the contribution list in the Introduction around these points, add the table above to Section 2, and add a paragraph to Section 3 that maps each theoretical term to its design choice and empirical test. We hope this clarifies the methodological contribution and would be grateful if the reviewer would take it into account in the final assessment.
Dear Reviewer tY6P,
Thank you again for your thoughtful review and follow-up. Your comments have greatly improved the quality of our paper, leading to the ablations of the adaptation strategy, the complexity comparison, and a clearer statement of our methodological contribution and its theoretical basis. As the author-reviewer discussion period ends soon, we would greatly appreciate knowing whether our latest reply has addressed your concern about methodological novelty, and we look forward to your final evaluation.
We thank the reviewer for the constructive comments. All new experiments reuse the source ensembles, folds, and test sets of Table 3. Δ is the paired accuracy difference averaged over the six transfers, with 95% t-intervals over ten repetitions.
W1. Novelty. We agree that the backbone modules are standard, but how to design the main model architecture to effectively extract sleep signals is a meaningful engineering problem.. Each backbone component is ablated in Table 5, e.g., removing the cross-epoch Sequence module lowers N1 F1 by 6.0 points, consistent with Proposition 1. Our main methodological contributions are the two adaptation components. Recording-wise statistics alignment corrects BN statistics per recording relative to source reference moments and reweights stages to separate acquisition shift from stage composition. TypCert selects target subjects from unlabeled predictions and matches the update scope to the label budget. The ablations below show that each part contributes beyond DSA, BN re-estimation, and random selection.
W2. Complexity. On one RTX 6000 Ada GPU (one night = 1,000 epochs):
CtxSleepNet has fewer parameters than DeepSleepNet but more FLOPs, almost all in the within-epoch encoder (CNN 1.2, GRU 0.8, Transformer 1.6 GFLOPs), while the cross-epoch LSTM adds less than 0.01. A whole night is still scored in 0.2 s by one model and in 3.4 s by the ensemble with alignment. The ensemble members are ten checkpoints of one training run, so it adds no training cost, and where inference cost matters a single model keeps most of the accuracy (85.9% vs. 87.3% on SleepEDF-20, Table 5).
Q. Stage reweighting and recording-wise BN correction. Zero-shot, without parameter updates:
The recording-wise BN correction carries most of the gain. It exceeds DSA by +5.0 [4.2, 5.9] and plain re-estimation by +1.1 [0.9, 1.3], because the source reference makes the robust geometric-mean variance usable (its scale differs from the stored variance). Stage reweighting adds +1.4 [0.9, 1.9] inside the statistics, up to +3.9 on Phys→EDF. Without the BN correction it can only rescale the outputs as a prior correction, which lowers accuracy in all six transfers because the self-estimated prior amplifies the model's own bias. DSA pools one set of statistics over all target recordings. This helps on SleepEDF targets (+6.1, +3.2), whose recordings have similar amplitudes (across-recording CV 0.24), but not on ISRUC and Physio2018 targets (−7.0 to +0.8), whose amplitudes vary far more (CV 0.80 and 0.63), so the pooled statistics fit no single recording. With fine-tuning on identical cohorts and updates, reweighting changes accuracy by +1.6 [1.5, 1.8] at 1–2%, so it helps when labels are absent or scarce.
Q. Typicality vs. certainty. At 1%/2% budgets with the same update scope (normalization + classifier):
TypCert exceeds random by +1.8 [−0.1, 3.7] and +2.5 [0.4, 4.7] (EDF→ISRUC +7.8 and +8.4), certainty alone by +2.1 [0.9, 3.2] and +1.0 [0.6, 1.4]. Certainty alone can fall below random because it favors subjects the source model already finds easy. Their cohorts contain fewer N1 epochs (9% vs. 13% for random) and match the target stage distribution worst (true TV 0.18 vs. 0.15 for random at 1%), which biases the classifier update. We keep certainty because it complements typicality. Lemma B.1 bounds the true cohort–target mismatch by dTV+Hˉ, and typicality alone controls only the first term. TypCert thus matches typicality in stage composition (true TV 0.10 and 0.07 at 1% and 2%) with lower predictive entropy (0.46 vs. 0.56). It also has the fewest runs below zero-shot and the lowest across-repetition SD (2.3 vs. 3.0 and 3.5 at 1%). We will add the complexity table and both ablations to the revision.