acceptodds
Under review as a conference paper at ICLR 2027

SleepGPT: Learning Transferable Sleep Stage Dynamics from Hypnograms

Abstract

Sleep stage sequences (hypnograms) encode temporal structure that is useful for both resolving ambiguous stage predictions and characterizing whole-night sleep architecture. However, existing sleep models typically learn temporal dependencies jointly with physiological signal representations, leaving open whether stage dynamics can be learned independently and transferred across models and tasks. We introduce SleepGPT, a compact autoregressive transformer pretrained exclusively on 5.86 million expert-scored stage annotations from 5,793 overnight recordings. SleepGPT learns stage dynamics through next-stage prediction and supports two complementary forms of transfer. For sleep staging, the frozen model provides a sequence prior that is combined with signal-based predictions from independently trained PSG classifiers; for sleep disorder classification, its pretrained encoder represents local hypnogram segments within a hierarchical whole-night model. Across seven external staging comparisons involving three classifiers and three datasets, SleepGPT improves accuracy by 1.6-4.2 percentage points without target-cohort retraining. Improvements persist for classifiers that already model temporal EEG context and for simultaneous PSG and wearable EEG recordings. For hypnogram-based disorder classification, SleepGPT initialization improves balanced accuracy over the same hierarchical architecture trained from scratch, with the gains extending to independent external cohorts. These results show that transferable temporal representations can be learned directly from expert-scored hypnograms, independently of physiological signal modeling, and reused for both epoch-level inference and recording-level prediction.

Primary area:
applications to neuroscience & cognitive science
Keywords:
Autoregressive sequence pretraining, hypnogram, automated sleep staging, sleep disorder classification

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_iF3NOpenReview ↗
Rating
1(−1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
1(−1)
Read review
Paper summary

This paper proposes SleepGPT, an autoregressive Transformer pretrained on expert-annotated sleep-stage sequences to learn transferable sleep-stage dynamics independently of physiological signals. The pretrained model is applied to two downstream tasks: sleep staging refinement through probability fusion with existing classifiers, and sleep disorder classification through a hierarchical sequence encoder.

Critical strengths
  1. The paper decouples sleep-stage sequence modeling from physiological signal modeling and learns stage-transition dynamics through autoregressive pretraining. The learned knowledge is then transferred to two distinct downstream tasks.

  2. The paper evaluates the transferability of the pretrained model across different downstream tasks and extends its experiments to wearable EEG recordings, demonstrating its potential applicability in different settings.

Critical weaknesses
  1. The paper primarily evaluates SleepGPT with relatively dated baseline models, including YASA, TinySleepNet, and XSleepNet. It remains unclear whether the reported improvements reflect limitations in the temporal modeling capabilities of these baselines. In particular, Table 3 shows that the gains become considerably smaller when the baseline models already incorporate temporal modeling. It would be valuable to investigate whether SleepGPT can provide consistent improvements when integrated with more recent and stronger sleep staging models, such as OSF(ICML 2026).

  2. Sleep stages exhibit strong temporal continuity and relatively structured transition patterns. It is therefore unclear whether large-scale autoregressive pretraining is necessary to capture these dependencies. The paper lacks comparisons with simpler sequence models or duration-aware transition models. Such comparisons are important to establish whether SleepGPT learns meaningful higher-order temporal dependencies beyond those captured by conventional transition-based approaches.

  3. Although SleepGPT does not require ground-truth stage labels during sleep staging inference, it relies on previously predicted stage labels to estimate the next-stage distribution. This introduces a potential risk of error propagation, as the model is pretrained on expert-scored sequences but operates on potentially noisy predicted sequences at inference time. The paper does not sufficiently investigate the impact of accumulated staging errors on the learned sequence prior. Furthermore, the sleep disorder classification experiments rely exclusively on expert-scored hypnograms, leaving the effectiveness of the framework with automatically generated hypnograms unverified.

[1] Shuai Z, Xu Z, Yang D, et al. OSF: On pre-training and scaling of sleep foundation models[J]. arXiv preprint arXiv:2603.00190, 2026.

Questions for authors

Please see in Weaknesses

9 comments
AuthorsOpenReview ↗

Response to Comment 1

Comment 1. Recent staging models and gains with temporal modeling

Thank you for suggesting OSF. We added 15 settings across SleepEDF, MASS and Physio2018 to test whether SleepGPT complements recent pretrained signal models.

OSF uses the released frozen backbone with a linear or one-hidden-layer MLP staging head, fitted on training recordings and selected on validation recordings. SleepFM uses its released encoder and temporal staging head without retraining. Signal-classifier outputs are held fixed in every fusion comparison. SleepGPT and the Markov1/duration controls use the same SHHS training subset.

CV appears first, followed by source→target transfer and direct SleepFM evaluation. Cells report ACC (%)/MF1 (%)/κ. Δ is SleepGPT minus baseline, calculated before rounding: percentage points for ACC/MF1, absolute change for κ. Bold marks the best displayed value per metric within a row.

EvaluationModelBaseline+ Markov1+ Duration+ SleepGPTΔ vs. Baseline
CV/SleepEDFOSF Linear82.21/75.28/0.752883.19/74.71/0.763583.33/75.21/0.765883.39/75.37/0.7667+1.18/+0.09/+0.0138
CV/SleepEDFOSF MLP82.07/74.80/0.750482.87/73.80/0.758383.00/74.31/0.760483.00/74.53/0.7609+0.93/-0.27/+0.0104
CV/MASSOSF Linear86.13/81.64/0.801486.40/81.35/0.803986.42/81.42/0.804186.38/81.31/0.8033+0.25/-0.33/+0.0019
CV/MASSOSF MLP86.18/81.83/0.802486.52/81.64/0.806086.52/81.70/0.805986.56/81.76/0.8066+0.39/-0.07/+0.0042
MASS → SleepEDFOSF Linear68.08/60.22/0.556469.83/60.77/0.574669.82/61.18/0.576369.86/61.60/0.5774+1.78/+1.38/+0.0210
MASS → SleepEDFOSF MLP73.47/63.29/0.628174.53/63.28/0.639374.67/63.50/0.641475.04/63.90/0.6460+1.56/+0.61/+0.0180
Physio2018 → SleepEDFOSF Linear74.14/66.44/0.643376.32/67.64/0.669076.67/68.04/0.673777.40/68.62/0.6829+3.26/+2.18/+0.0397
Physio2018 → SleepEDFOSF MLP74.28/65.37/0.644475.37/65.86/0.656776.14/66.39/0.666276.12/66.55/0.6662+1.84/+1.18/+0.0218
Physio2018 → MASSOSF Linear79.55/71.55/0.705479.80/71.25/0.706979.79/71.22/0.706479.74/71.14/0.7050+0.19/-0.40/-0.0004
Physio2018 → MASSOSF MLP80.03/72.01/0.711180.21/71.92/0.712480.16/71.69/0.710880.21/71.82/0.7119+0.18/-0.18/+0.0008
MASS → Physio2018OSF Linear62.13/58.06/0.494363.70/58.11/0.509563.52/58.42/0.508463.36/58.56/0.5073+1.23/+0.50/+0.0130
MASS → Physio2018OSF MLP62.85/59.17/0.502964.24/59.32/0.516864.25/59.41/0.517064.33/59.47/0.5178+1.48/+0.30/+0.0149
Pretrained → SleepEDFSleepFM71.54/64.40/0.596771.24/62.19/0.586771.45/61.70/0.588871.43/61.97/0.5891-0.11/-2.43/-0.0076
Pretrained → MASSSleepFM76.22/70.27/0.666077.02/71.02/0.674177.16/71.12/0.675777.29/71.27/0.6776+1.07/+1.00/+0.0116
Pretrained → Physio2018SleepFM76.04/72.87/0.675876.02/72.22/0.673276.08/72.27/0.673776.13/72.40/0.6746+0.09/-0.47/-0.0012

SleepGPT increases ACC in all 12 OSF settings (+0.18–3.26 points); CV gains are +0.25–1.18. For Physio2018→SleepEDF with OSF Linear, ACC rises from 74.14% to 77.40% and MF1 from 66.44% to 68.62%. Thus, gains extend beyond the older classifiers.

With the temporally informed SleepFM classifier, MASS improves in all three metrics (ACC +1.07 points; MF1 +1.00). Effects are smaller or negative on Physio2018 and SleepEDF, showing that a separate stage prior is useful in some settings even with temporal modeling, but its benefit depends on the classifier and cohort.

OSF fusion weights are selected on validation recordings, using source-cohort validation for transfer. SleepFM uses a fixed weight of 0.15 established from existing neural-classifier validation settings. SleepEDF CV groups both nights by participant, with validation participants drawn from the training partition. Evaluation uses the first-to-last actual sleep interval ±30 min. OSF maps Fpz-Cz/Pz-Oz to C3/C4 input slots as montage proxies, alongside EOG and EMG.

AuthorsOpenReview ↗

Response to Comment 2

Comment 2. Simpler sequence and duration-aware priors

Thank you for suggesting simpler sequence and duration-aware controls. We address this concern by comparing SleepGPT with these priors in the primary staging settings and by evaluating the downstream value of its pretrained representation for disease classification.

We added a first-order Markov prior and a causal duration-conditioned prior that estimates the next-stage distribution from the current stage and its elapsed run length, capped at 120 epochs. Both controls are fitted on the same 5,214 SHHS-1 training recordings and evaluated using identical base scores and fusion-weight selection protocols. YASA uses a fixed prior weight of 0.15; the other settings use validation-selected weights.

The table below covers the primary external-staging and BOAS settings. Each result cell reports ACC (%) / MF1 (%) / κ. Bold marks the highest displayed value for each metric within a row.

Primary settingBaselineMarkov1Duration priorSleepGPT
YASA → SleepEDF72.70 / 63.30 / 0.611077.06 / 66.71 / 0.682377.33 / 67.01 / 0.685776.90 / 68.20 / 0.6770
YASA → MASS78.60 / 71.20 / 0.700077.78 / 69.54 / 0.687477.84 / 69.69 / 0.688080.20 / 73.60 / 0.7240
YASA → Physio201871.40 / 66.20 / 0.613072.49 / 66.38 / 0.625472.59 / 66.58 / 0.626573.50 / 68.60 / 0.6400
TinySleepNet, MASS → Physio201870.50 / 66.30 / 0.600071.11 / 66.96 / 0.607571.27 / 66.88 / 0.608872.10 / 68.00 / 0.6210
TinySleepNet, Physio2018 → MASS74.10 / 67.10 / 0.617073.62 / 67.30 / 0.609173.62 / 67.30 / 0.609177.00 / 71.30 / 0.6650
XSleepNet, MASS → Physio201870.30 / 65.80 / 0.594070.29 / 65.82 / 0.593670.29 / 65.82 / 0.593672.20 / 67.40 / 0.6170
XSleepNet, Physio2018 → MASS74.40 / 68.50 / 0.626071.53 / 63.99 / 0.576371.57 / 63.99 / 0.576676.70 / 71.30 / 0.6610
BOAS headband, TinySleepNet80.70 / 62.50 / 0.652080.59 / 62.76 / 0.652480.56 / 62.17 / 0.651081.30 / 64.50 / 0.6670
BOAS headband, XSleepNet83.00 / 65.70 / 0.694082.53 / 66.85 / 0.689682.40 / 66.49 / 0.687483.90 / 67.80 / 0.7120
BOAS PSG, TinySleepNet80.90 / 64.10 / 0.660079.48 / 63.19 / 0.639079.50 / 63.60 / 0.640281.60 / 65.60 / 0.6730
BOAS PSG, XSleepNet83.40 / 68.80 / 0.715083.22 / 68.56 / 0.711383.23 / 68.54 / 0.710984.30 / 70.40 / 0.7280

The reported SleepGPT values are higher in ACC, MF1 and κ than both controls in ten of the eleven settings. On SleepEDF with YASA, the duration prior gives higher ACC and κ, while SleepGPT gives the highest MF1 (68.20% versus 67.01%). The neural classifiers in the cross-dataset and BOAS rows retain their temporal modules, so these comparisons also evaluate the additional value of the prior when the signal model already incorporates temporal context.

The downstream classification experiment tests a second use of SleepGPT: initializing the hierarchical classifier with its pretrained representation. The following table reports augmented CAP cross-validation results for abnormal versus normal sleep. For CAP automatic inputs, classifiers are trained and tested on the corresponding hypnogram source using 184 recordings with available signal data; these automatic-input results are means across five random seeds.

TaskInitializationHypnogram inputAUCBACC
CAP CVFrom scratchExpert-scored0.9580.893
CAP CVSleepGPTExpert-scored0.9740.962
CAP CVSleepGPTYASA0.8950.846
CAP CVSleepGPTYASA + Markov10.8800.809
CAP CVSleepGPTYASA + SleepGPT0.9140.842

Bold marks the highest value within each task and input group (expert-scored or automatic). With expert-scored inputs, SleepGPT initialization raises AUC from 0.958 to 0.974 and BACC from 0.893 to 0.962 relative to the same hierarchical classifier trained from scratch on CAP. With initialization held fixed, SleepGPT-fused automatic inputs give an AUC of 0.914, compared with 0.895 for raw YASA and 0.880 for Markov1-fused inputs. This automatic-input advantage concerns AUC: BACC is higher than with Markov1 (0.842 versus 0.809), but slightly below raw YASA (0.846).

Together, the staging comparisons and downstream classification results support the practical value of learning reusable stage-sequence representations. The table provides descriptive comparisons with the tested transition and duration controls; whether a smaller model or a smaller pretraining corpus can achieve the same benefit remains an open question.

AuthorsOpenReview ↗

Response to Comment 3

Comment 3. Error propagation and automatically generated hypnograms

Thank you for raising the mismatch between expert-scored pretraining sequences and predicted histories at inference. We added controlled history and noise experiments to measure its impact, and evaluated disease classifiers trained and tested on automatically generated hypnograms on both CAP and MNC.

(a) Error propagation from predicted histories

During staging inference, SleepGPT uses previous fused predictions without access to expert stage labels. We compare three history sources while holding the current signal-classifier scores and prior weight fixed: preceding expert stages (an oracle diagnostic), preceding raw base predictions, and preceding fused predictions (the deployed closed loop). We also replace 10% or 20% of the tokens fed into the closed-loop history with a different stage to test sensitivity to additional history errors. The reference labels and current signal scores remain unchanged.

All entries below are ACC (%) at fixed λ=0.15\lambda=0.15. Noise columns are means across three random seeds; the other columns evaluate fixed models without added random corruption.

SettingBase ACC (%)Expert history (%)Raw-base history (%)Closed loop (%)Closed loop +10% noise (%)Closed loop +20% noise (%)
MASS / TinySleepNet, non-sequential81.6984.3282.3882.5482.2782.11
MASS / XSleepNet, non-sequential80.4483.4881.2981.4781.2581.08
MASS / TinySleepNet, sequential84.6185.8384.4784.4584.5084.50
MASS / SleepFM76.2280.4576.1277.2976.5176.42

The comparisons reveal three effects. First, replacing expert histories with closed-loop predictions reduces ACC by 1.38–3.16 percentage points across these settings, quantifying the performance gap associated with predicted histories. Second, closed-loop feedback outperforms raw-base histories in three of four settings, with the largest difference on SleepFM (77.29% versus 76.12%). Sequential TinySleepNet is the exception: closed-loop ACC is 84.45%, slightly below both raw-base history (84.47%) and the base classifier (84.61%). Third, additional history corruption reduces the benefit in the two non-sequential settings and SleepFM, but their ACC remains above the corresponding base classifiers even at 20% corruption.

Closed-loop fusion therefore retains an accuracy advantage over the base classifier in three of four settings even with 20% additional history corruption. This supports robustness to noisy histories under the tested conditions; the experiments measure overall performance rather than the temporal structure of error propagation.

(b) Disease classification from automatic hypnograms

We now train and test disease classifiers on automatically generated hypnograms, keeping the hierarchical classifier initialization fixed across the three input sources below. CAP evaluates abnormal versus normal sleep; MNC evaluates type-1 narcolepsy versus other hypersomnia cases and controls. Results are means across five random seeds for CAP and ten for MNC. Within each cohort, all input sources use the same recordings and ten-fold splits. CAP includes 184 recordings with matching signal data, and MNC includes 407. Staging fusion uses a fixed prior weight of 0.15.

TaskHypnogram inputAUCBACC
CAP CVYASA0.8950.846
CAP CVYASA + Markov10.8800.809
CAP CVYASA + SleepGPT0.9140.842
MNC CVYASA0.8730.772
MNC CVYASA + Markov10.8640.765
MNC CVYASA + SleepGPT0.8720.783

Bold marks the highest mean for each metric within each cohort. On CAP, SleepGPT-fused inputs increase AUC by 0.019 over raw YASA and 0.034 over Markov1-fused inputs. BACC is higher than with Markov1 but slightly below raw YASA. On MNC, fusion gives higher mean BACC but essentially unchanged AUC relative to YASA; paired analyses do not show a significant improvement over YASA.

These experiments directly address the previously unverified use of automatic hypnograms: the SleepGPT-initialized classifier achieves an AUC of 0.914 on CAP and 0.872 on MNC with SleepGPT-fused staging inputs, without expert-scored hypnograms as classifier inputs during either training or testing. They demonstrate downstream classification utility with predicted stages across two disease tasks, with an additional AUC benefit from staging fusion on CAP and nearly unchanged mean AUC relative to raw YASA on MNC.

Reviewer_pPniOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes SleepGPT, a small GPT-2 model pretrained on approximately 5.86 million sleep stage tokens using next-token prediction to learn temporal patterns in sleep stages. The pretrained model is then used as a sequence prior and combined with existing EEG classifiers through probability fusion to improve sleep staging. It is also used as a pretrained encoder for sleep disorder classification. The main contribution is demonstrating the transferability of learned sleep stage dynamics across models, datasets, and tasks.

Critical strengths
  1. The experiments are comprehensive, including cross-dataset generalization, comparisons across different EEG classifiers, and ablation studies.
  2. The proposed method shows good transferability across different EEG classifiers and datasets.
Critical weaknesses
  1. Limited method novelty. The method mainly relies on standard GPT-2 pretraining, probability fusion, and transfer learning, without significant algorithmic innovation.
  2. The paper lacks an ablation study on the important hyperparameter α and does not report the final values used.
Questions for authors
  1. What is the next-stage prediction accuracy of the pretrained SleepGPT on an independent test set?
  2. What α values were used for different classifiers and datasets? Is it possible to use a single α across all settings?
3 comments
AuthorsOpenReview ↗

Response to comments

C1. Methodological novelty

The contribution is learning a reusable representation of sleep-stage dynamics separately from physiological signals, then testing its transfer through two uses: a causal prior for staging and an initialization for whole-night disease classification. The transformer architecture, score fusion and initialization transfer are established components; the research question is whether stage-only pretraining provides useful information across signal models, cohorts and tasks.

The added experiments address that question in three ways. First, SleepGPT improves ACC in all 12 evaluated OSF settings (+0.18–3.26 percentage points), extending the staging results to recent pretrained signal representations. With the released temporal SleepFM classifier, MASS ACC/MF1 increases from 76.22%/70.27% to 77.29%/71.27%, although effects vary across cohorts. Second, the independent SHHS evaluation below compares the learned sequence prior directly with first-order Markov models. Third, on expert-input CAP cross-validation, initializing the same hierarchical classifier with SleepGPT increases AUC/BACC from 0.958/0.893 to 0.974/0.962 relative to training from scratch.

Together, these results support stage-sequence pretraining as a reusable component for both staging and downstream classification. We will make this contribution explicit in the revision, with the emphasis on demonstrated transfer and the settings in which it helps.

C2. Ablation of α and reporting of its values

The manuscript's α is the signal-classifier weight. Writing λ=1−α for the prior weight, fusion is

st(k)=αlog⁡pbase,t(k)+(1−α)log⁡pprior(k∣y^<t).s_t(k)=\alpha\log p_{\mathrm{base},t}(k)+(1-\alpha)\log p_{\mathrm{prior}}(k\mid\hat y_{<t}).

We normalize the fused scores when probabilities are needed. For YASA, probabilities are floored at 10−610^{-6} before taking logs. The first five stages use the base classifier; subsequent predictions use preceding fused stages causally.

The added sensitivity evaluation uses λ∈{0,0.05,…,1}, equivalently α∈{0,0.05,…,1}. CV weights maximize validation ACC within each training fold; transfer weights use source-cohort validation. Test labels are not used for weight selection.

The sensitivity results below compare a fixed α with validation selection using identical signal scores. TSN=TinySleepNet; XSN=XSleepNet; EDF=SleepEDF.

A shared α=0.85 (λ=0.15) is usable, but validation selection adapts to the signal model. The table compares ACC (%) for representative settings using the same cached scores:

SettingBaseFixed α=0.85Validation-selected α
CV EDF TSN nonseq80.7682.1382.68
CV MASS TSN nonseq81.6982.5482.80
CV MASS XSN seq85.3085.1485.30

The fixed weight retains gains in the two non-sequential examples. For sequential MASS XSN, validation selects α=1 (no prior), avoiding the small loss from fixed α=0.85. This demonstrates why a shared weight is convenient while validation selection can account for temporal information already captured by the signal classifier.

The setting-specific α values are listed under Question 2. We will include the selection protocol and sensitivity results in the revision.

AuthorsOpenReview ↗

Response to Questions For Authors

Q1. What is the next-stage prediction accuracy on an independent test set?

SleepGPT achieves 88.59% next-stage accuracy on a held-out set of 579 SHHS-1 recordings, with pretraining restricted to the other 5,214 recordings. All Markov controls are fitted on the same training recordings. Each prediction uses only preceding expert-scored stages; target stages are not included in the input. This evaluates the sequence prior directly, separately from EEG-based staging.

PredictorACC (%)MF1 (%)NLLTransition ACC (%)
Copy preceding stage87.9681.79—0.00
Markov187.9681.790.44550.00
Markov287.9681.770.39220.05
Markov388.2881.830.375412.76
SleepGPT88.5982.190.345518.03

The evaluation includes 586,108 target epochs and 70,565 transitions, defined by a change from the preceding expert stage. Copying the preceding stage already reaches 87.96% because stages persist across epochs. SleepGPT improves ACC by 0.63 points over copying and 0.31 over Markov3. Its transition ACC is 18.03%, versus 12.76% for Markov3, and NLL decreases from 0.3754 to 0.3455. Thus, the benefit extends beyond predicting persistence to better transition prediction and lower log loss. NLL is omitted for the copy rule because it requires an additional probability-smoothing convention.

This is a teacher-forced evaluation with expert histories. The separate predicted-history tests measure how the prior behaves when preceding stages must also be inferred.

Q2. What α values were used, and can one α be used across settings?

The table gives all distinct validation-selected α values across folds in the added neural-classifier evaluation. α weights the signal classifier; 1−α weights the prior. CV uses within-training-fold validation; transfer uses source-cohort validation. TSN=TinySleepNet; XSN=XSleepNet; EDF=SleepEDF; P18=Physio2018. Transfer and BOAS classifiers retain their temporal modules.

SettingClassifierSelected α
CV EDFTSN nonseq0.70, 0.75
CV EDFTSN seq0.75, 0.85, 0.90, 0.95, 1.00
CV EDFXSN nonseq0.65, 0.70
CV EDFXSN seq0.85, 0.90, 0.95, 1.00
CV MASSTSN nonseq0.65, 0.70, 0.75, 0.80
CV MASSTSN seq0.95, 1.00
CV MASSXSN nonseq0.60, 0.70, 0.75, 0.80
CV MASSXSN seq1.00
CV BOAS headbandTSN0.55, 0.80, 0.90, 0.95, 1.00
CV BOAS headbandXSN0.70, 0.80, 0.85, 0.90, 0.95, 1.00
CV BOAS PSGTSN0.40, 0.45, 0.55, 0.65, 0.70, 0.75, 0.80, 1.00
CV BOAS PSGXSN0.80, 0.90, 0.95, 1.00
MASS→P18TSN0.95
MASS→P18XSN1.00
P18→MASSTSN1.00
P18→MASSXSN0.95

Multiple values therefore represent different folds, not multiple weights applied within a fold. Within each fold, we select a single α; if validation ACC ties exactly, we choose the largest α (smallest prior weight λ), using unrounded validation ACC.

YASA and released SleepFM use a frozen α=0.85, derived from existing neural-classifier validation settings. This avoids selecting weights on their test results. We will report the α convention, selection protocol, and setting-specific values explicitly in the revision.

A shared α=0.85 retains the gains in the two non-sequential examples above, while validation selection chooses α=1 for sequential MASS XSN. Thus, one α is a practical shared setting, but setting-specific validation can avoid adding an unhelpful prior.

Thank you for the clarification and additional experiments. My main concerns have been addressed, and I will maintain my original rating of 3 (Weak Accept).

Reviewer_EBUQOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

SleepGPT is an autoregressive transformer pretrained on 5.86 million expert-scored sleep-stage annotations from 5,793 overnight recordings. The pretrained model is used as a sequence prior to refine independently trained sleep-staging models and as an initialization for a hierarchical sleep-disorder classifier. The evaluation includes external staging comparisons across datasets and classifiers, simultaneous PSG and wearable EEG, and disorder classification on external cohorts.

Critical strengths
  1. Separating stage dynamics from physiological signal representations is a clear and useful research question, and the resulting prior can be reused with existing classifiers.
  2. The evaluation spans multiple classifiers, datasets, and modalities, including classifiers that already use temporal context. This makes the transfer claim more convincing than a single-model evaluation.
  3. The disorder-classification experiments compare pretrained initialization with the same hierarchical architecture trained from scratch, as well as feature-based baselines.
Critical weaknesses
  1. The methodological novelty is moderate: autoregressive sequence pretraining and prior fusion are established ideas. Comparisons with carefully tuned Markov or semi-Markov priors and parameter- or data-matched sequence models would better establish the need for the transformer.
  2. A population-level prior could suppress rare but clinically meaningful transitions. Per-stage and subgroup results, calibration, uncertainty intervals, and analysis of unusual sleep architectures would clarify where the prior helps or hurts.
  3. The fusion procedure should be evaluated for sensitivity to its weighting and to temporal information already captured by the base classifier. The validation data used to select fusion parameters should be explicit.
  4. For disorder classification, the paper should carefully distinguish results based on expert-scored hypnograms from performance using automatically predicted stages, and clarify subject-level separation and cohort differences.
Questions for authors

See weakness.

3 comments
AuthorsOpenReview ↗

Response to Comment 1

Comment 1. Novelty and simpler sequence priors

Our contribution is learning sleep-stage dynamics separately from physiological signals and reusing that representation across signal classifiers, cohorts and tasks: as a causal staging prior and as an initialization for disease classification. The novelty lies in this reusable representation and its demonstrated utility, rather than a new transformer architecture. The comparisons below test its value beyond simple transition and duration priors.

We added a first-order Markov prior and a causal duration-conditioned prior that estimates the next-stage distribution from the current stage and its elapsed run length, capped at 120 epochs. Both controls are fitted on the same 5,214 SHHS-1 training recordings and evaluated using identical base scores and fusion-weight selection protocols. YASA uses a fixed prior weight of 0.15; the other settings use validation-selected weights.

The table below covers the primary external-staging and BOAS settings. Each result cell reports ACC (%) / MF1 (%) / κ. Bold marks the highest displayed value for each metric within a row.

Primary settingBaselineMarkov1Duration priorSleepGPT
YASA → SleepEDF72.70 / 63.30 / 0.611077.06 / 66.71 / 0.682377.33 / 67.01 / 0.685776.90 / 68.20 / 0.6770
YASA → MASS78.60 / 71.20 / 0.700077.78 / 69.54 / 0.687477.84 / 69.69 / 0.688080.20 / 73.60 / 0.7240
YASA → Physio201871.40 / 66.20 / 0.613072.49 / 66.38 / 0.625472.59 / 66.58 / 0.626573.50 / 68.60 / 0.6400
TinySleepNet, MASS → Physio201870.50 / 66.30 / 0.600071.11 / 66.96 / 0.607571.27 / 66.88 / 0.608872.10 / 68.00 / 0.6210
TinySleepNet, Physio2018 → MASS74.10 / 67.10 / 0.617073.62 / 67.30 / 0.609173.62 / 67.30 / 0.609177.00 / 71.30 / 0.6650
XSleepNet, MASS → Physio201870.30 / 65.80 / 0.594070.29 / 65.82 / 0.593670.29 / 65.82 / 0.593672.20 / 67.40 / 0.6170
XSleepNet, Physio2018 → MASS74.40 / 68.50 / 0.626071.53 / 63.99 / 0.576371.57 / 63.99 / 0.576676.70 / 71.30 / 0.6610
BOAS headband, TinySleepNet80.70 / 62.50 / 0.652080.59 / 62.76 / 0.652480.56 / 62.17 / 0.651081.30 / 64.50 / 0.6670
BOAS headband, XSleepNet83.00 / 65.70 / 0.694082.53 / 66.85 / 0.689682.40 / 66.49 / 0.687483.90 / 67.80 / 0.7120
BOAS PSG, TinySleepNet80.90 / 64.10 / 0.660079.48 / 63.19 / 0.639079.50 / 63.60 / 0.640281.60 / 65.60 / 0.6730
BOAS PSG, XSleepNet83.40 / 68.80 / 0.715083.22 / 68.56 / 0.711383.23 / 68.54 / 0.710984.30 / 70.40 / 0.7280

The reported SleepGPT values are higher in ACC, MF1 and κ than both controls in ten of the eleven settings. On SleepEDF with YASA, the duration prior gives higher ACC and κ, while SleepGPT gives the highest MF1 (68.20% versus 67.01%). The neural classifiers in the cross-dataset and BOAS rows retain their temporal modules, so these comparisons also evaluate the additional value of the prior when the signal model already incorporates temporal context.

The downstream classification experiment tests a second use of SleepGPT: initializing the hierarchical classifier with its pretrained representation. The following table reports augmented CAP cross-validation results for abnormal versus normal sleep. For CAP automatic inputs, classifiers are trained and tested on the corresponding hypnogram source using 184 recordings with available signal data; these automatic-input results are means across five random seeds.

TaskInitializationHypnogram inputAUCBACC
CAP CVFrom scratchExpert-scored0.9580.893
CAP CVSleepGPTExpert-scored0.9740.962
CAP CVSleepGPTYASA0.8950.846
CAP CVSleepGPTYASA + Markov10.8800.809
CAP CVSleepGPTYASA + SleepGPT0.9140.842

Bold marks the highest value within each task and input group (expert-scored or automatic). With expert-scored inputs, SleepGPT initialization raises AUC from 0.958 to 0.974 and BACC from 0.893 to 0.962 relative to the same hierarchical classifier trained from scratch on CAP. With initialization held fixed, SleepGPT-fused automatic inputs give an AUC of 0.914, compared with 0.895 for raw YASA and 0.880 for Markov1-fused inputs. This automatic-input advantage concerns AUC: BACC is higher than with Markov1 (0.842 versus 0.809), but slightly below raw YASA (0.846).

Together, the staging comparisons and downstream classification results support the practical value of learning reusable stage-sequence representations. The table provides descriptive comparisons with the tested transition and duration controls; whether a smaller model or a smaller pretraining corpus can achieve the same benefit remains an open question.

AuthorsOpenReview ↗

Response to Comment 2

Comment 2. Rare transitions, stage-wise effects, subgroups and calibration

We added stage-wise, transition-specific, sleep-architecture and calibration evaluations to test whether pooled accuracy conceals adverse effects. Transitions are changes in expert stages; rare types are the bottom quartile of off-diagonal transition counts in SHHS training data. This definition uses training frequencies, not target-test frequencies or clinical labels.

The MASS non-sequential results compare all three priors on identical signal scores and labels. Markov1, Duration and SleepGPT use the same 5,214 SHHS training recordings and the same validation-weight selection criterion. Entries are ACC (%); rare-transition results cover 150 epochs.

ClassifierPriorOverallTransitionRare transition
TinySleepNetBase81.6963.4937.33
TinySleepNetMarkov182.4057.5920.67
TinySleepNetDuration82.5658.3920.00
TinySleepNetSleepGPT82.8058.4917.33
XSleepNetBase80.4461.4432.00
XSleepNetMarkov181.7055.4716.67
XSleepNetDuration81.6357.0317.33
XSleepNetSleepGPT82.0056.0718.00

All three priors increase overall ACC while reducing transition ACC. SleepGPT has the highest overall ACC in both settings. For transition ACC, SleepGPT is highest among priors with TinySleepNet, while Duration is highest with XSleepNet. Rare-transition preservation is also mixed: SleepGPT is below both simple priors with TinySleepNet but above them with XSleepNet. Thus, the smoothing tradeoff is shared, and the preferred prior depends on the target metric.

Stage recall for MASS non-sequential TinySleepNet is:

StageBase (%)Markov1 (%)Duration (%)SleepGPT (%)
W86.2688.0587.8487.95
N133.9124.5926.3727.03
N288.9191.3891.5091.55
N379.8477.5477.2777.64
REM83.1885.4485.5186.06

SleepGPT retains more N1 recall than both simple priors in both classifiers (27.03% versus 24.59%/26.37% for TinySleepNet; 30.05% versus 27.51%/29.92% for XSleepNet), although all remain below the base classifiers.

We also evaluate fixed architecture strata: ≥20 transitions/hour, ≥40% wake, ≥20% N1, and <5% N3. These descriptive architecture groups use expert labels only for post hoc evaluation, not for inference or weight selection; they are not demographic or disease subgroups. For TinySleepNet, ACC increases from 79.70% to 80.38% in the 94 high-transition recordings, but decreases from 74.73% to 73.86% in the five high-N1 recordings. The latter group is small; both results identify conditions obscured by pooled averages.

Calibration uses normalized fused log scores. Each cell reports NLL / multiclass Brier / ECE (15 equal-width bins); lower is better, and bold marks the lowest displayed value per metric. All priors use the same SHHS training subset and validation-weight selection rules, and are compared on identical signal outputs.

SettingBaseMarkov1DurationSleepGPT
MASS / TinySleepNet0.5500 / 0.2686 / 0.04540.5229 / 0.2611 / 0.05020.5252 / 0.2593 / 0.05140.5196 / 0.2574 / 0.0527
MASS / XSleepNet0.5387 / 0.2825 / 0.04690.5070 / 0.2689 / 0.04890.5071 / 0.2689 / 0.04980.5021 / 0.2666 / 0.0542
BOAS headband / XSleepNet0.5892 / 0.2661 / 0.06760.5811 / 0.2689 / 0.07430.5848 / 0.2699 / 0.07690.5856 / 0.2704 / 0.0800
BOAS PSG / TinySleepNet0.8993 / 0.3333 / 0.12390.7845 / 0.3336 / 0.12440.8320 / 0.3357 / 0.12900.7938 / 0.3339 / 0.1293

On MASS, SleepGPT has the lowest NLL and Brier score, but ECE is higher than for either simple prior. On both BOAS settings, Markov1 has lower NLL, Brier and ECE than SleepGPT. All three priors increase ECE relative to the base models in these four settings. For headband XSleepNet, the SleepGPT ECE increase is 0.0124 (95% CI 0.0087–0.0163). These results distinguish improved predictive log loss from confidence calibration and show where a simpler prior is preferable.

Intervals use 2,000 paired recording/participant-cluster bootstrap resamples, without multiple-comparison adjustment. These diagnostics show both where SleepGPT improves on simple priors and where smoothing harms transition recognition or calibration. We will report these outcomes alongside overall performance to support choosing a prior for the intended metric; demographic subgroup effects remain untested.

AuthorsOpenReview ↗

Response to Comment 3 and 4

Comment 3. Fusion sensitivity and temporal context

We make the weighting convention explicit: α weights the signal classifier and λ=1−α weights the sequence prior.

st(k)=αlog⁡pbase,t(k)+(1−α)log⁡pprior(k∣y^<t).s_t(k)=\alpha\log p_{\mathrm{base},t}(k)+(1-\alpha)\log p_{\mathrm{prior}}(k\mid\hat y_{<t}).

The first five stages use the base classifier; subsequent predictions use previous fused stages causally. For YASA, probabilities are floored at 10−610^{-6} before taking logs. Fused log scores are normalized when probabilities are required.

The added sensitivity analysis searches λ=0,0.05,…,1. Within-dataset CV selects a weight on validation recordings inside each training partition. Transfer experiments use source-cohort validation recordings. Selection uses validation ACC, with exact ties resolved by the smallest λ (largest α), using unrounded ACC. Test labels are not used. YASA and released SleepFM use fixed λ=0.15, derived from existing neural-classifier validation settings.

MASS results quantify the role of temporal context. Entries are ACC (%).

ClassifierBaseValidation-selectedFixed λ=0.15Selected λ across folds
TinySleepNet non-sequential81.6982.8082.540.20, 0.25, 0.30, 0.35
TinySleepNet sequential84.6184.6384.450.00, 0.05
XSleepNet non-sequential80.4482.0081.470.20, 0.25, 0.30, 0.40
XSleepNet sequential85.3085.3085.140.00

Multiple selected weights represent different folds; each fold uses one weight. Non-sequential classifiers receive larger gains, while sequential TinySleepNet changes only from 84.61% to 84.63%. Sequential XSleepNet selects λ=0, preserving its 85.30% baseline; a fixed λ=0.15 reduces ACC to 85.14%. Validation therefore allows the method to reduce or disable the prior when the signal classifier already captures useful temporal information.

We will state the validation sources, weighting convention and selected weights explicitly and include the sensitivity results. A fixed weight offers a shared setting, while validation selection accounts for differences among classifiers and cohorts.

Comment 4. Automatic-stage inputs, subject separation and cohort differences

We added disease classifiers trained and tested on automatically generated hypnograms. The manuscript's expert-input results evaluate representation transfer; the new experiments test performance when stage sequences must first be predicted from signals.

For each cohort, we hold classifier initialization, included recordings and ten-fold splits fixed across YASA, YASA+Markov1 and YASA+SleepGPT inputs. The training and test inputs use the same hypnogram-generation method. CAP includes 184 recordings with matching signals, with results averaged over five random seeds; MNC includes 407 recordings, averaged over ten. Staging fusion uses λ=0.15. CAP evaluates abnormal versus normal sleep, whereas MNC evaluates type-1 narcolepsy versus other hypersomnia cases and controls.

TaskHypnogram inputAUCBACC
CAP CVYASA0.8950.846
CAP CVYASA + Markov10.8800.809
CAP CVYASA + SleepGPT0.9140.842
MNC CVYASA0.8730.772
MNC CVYASA + Markov10.8640.765
MNC CVYASA + SleepGPT0.8720.783

Bold marks the highest mean within each task. On CAP, SleepGPT-fused inputs increase AUC by 0.019 over YASA and 0.034 over Markov1; BACC improves over Markov1 but is slightly below YASA. On MNC, fusion has higher mean BACC but nearly unchanged AUC relative to YASA, without a significant paired improvement. These results verify classification with predicted stages during both training and testing; fusion benefits depend on the cohort and metric.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.