- Rating
- 1(−1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1(−1)
Read review
Paper summary
This paper proposes SleepGPT, an autoregressive Transformer pretrained on expert-annotated sleep-stage sequences to learn transferable sleep-stage dynamics independently of physiological signals. The pretrained model is applied to two downstream tasks: sleep staging refinement through probability fusion with existing classifiers, and sleep disorder classification through a hierarchical sequence encoder.
Critical strengths
-
The paper decouples sleep-stage sequence modeling from physiological signal modeling and learns stage-transition dynamics through autoregressive pretraining. The learned knowledge is then transferred to two distinct downstream tasks.
-
The paper evaluates the transferability of the pretrained model across different downstream tasks and extends its experiments to wearable EEG recordings, demonstrating its potential applicability in different settings.
Critical weaknesses
-
The paper primarily evaluates SleepGPT with relatively dated baseline models, including YASA, TinySleepNet, and XSleepNet. It remains unclear whether the reported improvements reflect limitations in the temporal modeling capabilities of these baselines. In particular, Table 3 shows that the gains become considerably smaller when the baseline models already incorporate temporal modeling. It would be valuable to investigate whether SleepGPT can provide consistent improvements when integrated with more recent and stronger sleep staging models, such as OSF(ICML 2026).
-
Sleep stages exhibit strong temporal continuity and relatively structured transition patterns. It is therefore unclear whether large-scale autoregressive pretraining is necessary to capture these dependencies. The paper lacks comparisons with simpler sequence models or duration-aware transition models. Such comparisons are important to establish whether SleepGPT learns meaningful higher-order temporal dependencies beyond those captured by conventional transition-based approaches.
-
Although SleepGPT does not require ground-truth stage labels during sleep staging inference, it relies on previously predicted stage labels to estimate the next-stage distribution. This introduces a potential risk of error propagation, as the model is pretrained on expert-scored sequences but operates on potentially noisy predicted sequences at inference time. The paper does not sufficiently investigate the impact of accumulated staging errors on the learned sequence prior. Furthermore, the sleep disorder classification experiments rely exclusively on expert-scored hypnograms, leaving the effectiveness of the framework with automatically generated hypnograms unverified.
[1] Shuai Z, Xu Z, Yang D, et al. OSF: On pre-training and scaling of sleep foundation models[J]. arXiv preprint arXiv:2603.00190, 2026.
Questions for authors
Please see in Weaknesses
9 comments
Response to Comment 2
Comment 2. Simpler sequence and duration-aware priors
Thank you for suggesting simpler sequence and duration-aware controls. We address this concern by comparing SleepGPT with these priors in the primary staging settings and by evaluating the downstream value of its pretrained representation for disease classification.
We added a first-order Markov prior and a causal duration-conditioned prior that estimates the next-stage distribution from the current stage and its elapsed run length, capped at 120 epochs. Both controls are fitted on the same 5,214 SHHS-1 training recordings and evaluated using identical base scores and fusion-weight selection protocols. YASA uses a fixed prior weight of 0.15; the other settings use validation-selected weights.
The table below covers the primary external-staging and BOAS settings. Each result cell reports ACC (%) / MF1 (%) / κ. Bold marks the highest displayed value for each metric within a row.
| Primary setting | Baseline | Markov1 | Duration prior | SleepGPT |
|---|---|---|---|---|
| YASA → SleepEDF | 72.70 / 63.30 / 0.6110 | 77.06 / 66.71 / 0.6823 | 77.33 / 67.01 / 0.6857 | 76.90 / 68.20 / 0.6770 |
| YASA → MASS | 78.60 / 71.20 / 0.7000 | 77.78 / 69.54 / 0.6874 | 77.84 / 69.69 / 0.6880 | 80.20 / 73.60 / 0.7240 |
| YASA → Physio2018 | 71.40 / 66.20 / 0.6130 | 72.49 / 66.38 / 0.6254 | 72.59 / 66.58 / 0.6265 | 73.50 / 68.60 / 0.6400 |
| TinySleepNet, MASS → Physio2018 | 70.50 / 66.30 / 0.6000 | 71.11 / 66.96 / 0.6075 | 71.27 / 66.88 / 0.6088 | 72.10 / 68.00 / 0.6210 |
| TinySleepNet, Physio2018 → MASS | 74.10 / 67.10 / 0.6170 | 73.62 / 67.30 / 0.6091 | 73.62 / 67.30 / 0.6091 | 77.00 / 71.30 / 0.6650 |
| XSleepNet, MASS → Physio2018 | 70.30 / 65.80 / 0.5940 | 70.29 / 65.82 / 0.5936 | 70.29 / 65.82 / 0.5936 | 72.20 / 67.40 / 0.6170 |
| XSleepNet, Physio2018 → MASS | 74.40 / 68.50 / 0.6260 | 71.53 / 63.99 / 0.5763 | 71.57 / 63.99 / 0.5766 | 76.70 / 71.30 / 0.6610 |
| BOAS headband, TinySleepNet | 80.70 / 62.50 / 0.6520 | 80.59 / 62.76 / 0.6524 | 80.56 / 62.17 / 0.6510 | 81.30 / 64.50 / 0.6670 |
| BOAS headband, XSleepNet | 83.00 / 65.70 / 0.6940 | 82.53 / 66.85 / 0.6896 | 82.40 / 66.49 / 0.6874 | 83.90 / 67.80 / 0.7120 |
| BOAS PSG, TinySleepNet | 80.90 / 64.10 / 0.6600 | 79.48 / 63.19 / 0.6390 | 79.50 / 63.60 / 0.6402 | 81.60 / 65.60 / 0.6730 |
| BOAS PSG, XSleepNet | 83.40 / 68.80 / 0.7150 | 83.22 / 68.56 / 0.7113 | 83.23 / 68.54 / 0.7109 | 84.30 / 70.40 / 0.7280 |
The reported SleepGPT values are higher in ACC, MF1 and κ than both controls in ten of the eleven settings. On SleepEDF with YASA, the duration prior gives higher ACC and κ, while SleepGPT gives the highest MF1 (68.20% versus 67.01%). The neural classifiers in the cross-dataset and BOAS rows retain their temporal modules, so these comparisons also evaluate the additional value of the prior when the signal model already incorporates temporal context.
The downstream classification experiment tests a second use of SleepGPT: initializing the hierarchical classifier with its pretrained representation. The following table reports augmented CAP cross-validation results for abnormal versus normal sleep. For CAP automatic inputs, classifiers are trained and tested on the corresponding hypnogram source using 184 recordings with available signal data; these automatic-input results are means across five random seeds.
| Task | Initialization | Hypnogram input | AUC | BACC |
|---|---|---|---|---|
| CAP CV | From scratch | Expert-scored | 0.958 | 0.893 |
| CAP CV | SleepGPT | Expert-scored | 0.974 | 0.962 |
| CAP CV | SleepGPT | YASA | 0.895 | 0.846 |
| CAP CV | SleepGPT | YASA + Markov1 | 0.880 | 0.809 |
| CAP CV | SleepGPT | YASA + SleepGPT | 0.914 | 0.842 |
Bold marks the highest value within each task and input group (expert-scored or automatic). With expert-scored inputs, SleepGPT initialization raises AUC from 0.958 to 0.974 and BACC from 0.893 to 0.962 relative to the same hierarchical classifier trained from scratch on CAP. With initialization held fixed, SleepGPT-fused automatic inputs give an AUC of 0.914, compared with 0.895 for raw YASA and 0.880 for Markov1-fused inputs. This automatic-input advantage concerns AUC: BACC is higher than with Markov1 (0.842 versus 0.809), but slightly below raw YASA (0.846).
Together, the staging comparisons and downstream classification results support the practical value of learning reusable stage-sequence representations. The table provides descriptive comparisons with the tested transition and duration controls; whether a smaller model or a smaller pretraining corpus can achieve the same benefit remains an open question.
Response to Comment 3
Comment 3. Error propagation and automatically generated hypnograms
Thank you for raising the mismatch between expert-scored pretraining sequences and predicted histories at inference. We added controlled history and noise experiments to measure its impact, and evaluated disease classifiers trained and tested on automatically generated hypnograms on both CAP and MNC.
(a) Error propagation from predicted histories
During staging inference, SleepGPT uses previous fused predictions without access to expert stage labels. We compare three history sources while holding the current signal-classifier scores and prior weight fixed: preceding expert stages (an oracle diagnostic), preceding raw base predictions, and preceding fused predictions (the deployed closed loop). We also replace 10% or 20% of the tokens fed into the closed-loop history with a different stage to test sensitivity to additional history errors. The reference labels and current signal scores remain unchanged.
All entries below are ACC (%) at fixed . Noise columns are means across three random seeds; the other columns evaluate fixed models without added random corruption.
| Setting | Base ACC (%) | Expert history (%) | Raw-base history (%) | Closed loop (%) | Closed loop +10% noise (%) | Closed loop +20% noise (%) |
|---|---|---|---|---|---|---|
| MASS / TinySleepNet, non-sequential | 81.69 | 84.32 | 82.38 | 82.54 | 82.27 | 82.11 |
| MASS / XSleepNet, non-sequential | 80.44 | 83.48 | 81.29 | 81.47 | 81.25 | 81.08 |
| MASS / TinySleepNet, sequential | 84.61 | 85.83 | 84.47 | 84.45 | 84.50 | 84.50 |
| MASS / SleepFM | 76.22 | 80.45 | 76.12 | 77.29 | 76.51 | 76.42 |
The comparisons reveal three effects. First, replacing expert histories with closed-loop predictions reduces ACC by 1.38–3.16 percentage points across these settings, quantifying the performance gap associated with predicted histories. Second, closed-loop feedback outperforms raw-base histories in three of four settings, with the largest difference on SleepFM (77.29% versus 76.12%). Sequential TinySleepNet is the exception: closed-loop ACC is 84.45%, slightly below both raw-base history (84.47%) and the base classifier (84.61%). Third, additional history corruption reduces the benefit in the two non-sequential settings and SleepFM, but their ACC remains above the corresponding base classifiers even at 20% corruption.
Closed-loop fusion therefore retains an accuracy advantage over the base classifier in three of four settings even with 20% additional history corruption. This supports robustness to noisy histories under the tested conditions; the experiments measure overall performance rather than the temporal structure of error propagation.
(b) Disease classification from automatic hypnograms
We now train and test disease classifiers on automatically generated hypnograms, keeping the hierarchical classifier initialization fixed across the three input sources below. CAP evaluates abnormal versus normal sleep; MNC evaluates type-1 narcolepsy versus other hypersomnia cases and controls. Results are means across five random seeds for CAP and ten for MNC. Within each cohort, all input sources use the same recordings and ten-fold splits. CAP includes 184 recordings with matching signal data, and MNC includes 407. Staging fusion uses a fixed prior weight of 0.15.
| Task | Hypnogram input | AUC | BACC |
|---|---|---|---|
| CAP CV | YASA | 0.895 | 0.846 |
| CAP CV | YASA + Markov1 | 0.880 | 0.809 |
| CAP CV | YASA + SleepGPT | 0.914 | 0.842 |
| MNC CV | YASA | 0.873 | 0.772 |
| MNC CV | YASA + Markov1 | 0.864 | 0.765 |
| MNC CV | YASA + SleepGPT | 0.872 | 0.783 |
Bold marks the highest mean for each metric within each cohort. On CAP, SleepGPT-fused inputs increase AUC by 0.019 over raw YASA and 0.034 over Markov1-fused inputs. BACC is higher than with Markov1 but slightly below raw YASA. On MNC, fusion gives higher mean BACC but essentially unchanged AUC relative to YASA; paired analyses do not show a significant improvement over YASA.
These experiments directly address the previously unverified use of automatic hypnograms: the SleepGPT-initialized classifier achieves an AUC of 0.914 on CAP and 0.872 on MNC with SleepGPT-fused staging inputs, without expert-scored hypnograms as classifier inputs during either training or testing. They demonstrate downstream classification utility with predicted stages across two disease tasks, with an additional AUC benefit from staging fusion on CAP and nearly unchanged mean AUC relative to raw YASA on MNC.
Response to Comment 1
Comment 1. Recent staging models and gains with temporal modeling
Thank you for suggesting OSF. We added 15 settings across SleepEDF, MASS and Physio2018 to test whether SleepGPT complements recent pretrained signal models.
OSF uses the released frozen backbone with a linear or one-hidden-layer MLP staging head, fitted on training recordings and selected on validation recordings. SleepFM uses its released encoder and temporal staging head without retraining. Signal-classifier outputs are held fixed in every fusion comparison. SleepGPT and the Markov1/duration controls use the same SHHS training subset.
CV appears first, followed by source→target transfer and direct SleepFM evaluation. Cells report ACC (%)/MF1 (%)/κ. Δ is SleepGPT minus baseline, calculated before rounding: percentage points for ACC/MF1, absolute change for κ. Bold marks the best displayed value per metric within a row.
SleepGPT increases ACC in all 12 OSF settings (+0.18–3.26 points); CV gains are +0.25–1.18. For Physio2018→SleepEDF with OSF Linear, ACC rises from 74.14% to 77.40% and MF1 from 66.44% to 68.62%. Thus, gains extend beyond the older classifiers.
With the temporally informed SleepFM classifier, MASS improves in all three metrics (ACC +1.07 points; MF1 +1.00). Effects are smaller or negative on Physio2018 and SleepEDF, showing that a separate stage prior is useful in some settings even with temporal modeling, but its benefit depends on the classifier and cohort.
OSF fusion weights are selected on validation recordings, using source-cohort validation for transfer. SleepFM uses a fixed weight of 0.15 established from existing neural-classifier validation settings. SleepEDF CV groups both nights by participant, with validation participants drawn from the training partition. Evaluation uses the first-to-last actual sleep interval ±30 min. OSF maps Fpz-Cz/Pz-Oz to C3/C4 input slots as montage proxies, alongside EOG and EMG.