acceptodds
Under review as a conference paper at ICLR 2027

Temporal Structure in Contrastive EEG Representation Learning: A Controlled Study for Obstructive Sleep Apnea Detection

Abstract

Contrastive self-supervised learning for physiological signals typically treats windows as independent apart from augmentation-defined positives, ignoring that windows from the same patient are temporally related. We introduce Temporal NT-Xent, which retains augmentation-defined positives and full-strength cross-patient negatives while continuously down-weighting the repulsive contribution of same-patient negatives according to their temporal distance. We evaluate this objective for EEG-based obstructive sleep apnea (OSA) detection along three complementary axes. First, on a canonical patient-held-out split—with patients unseen during pretraining, validation, or test selection—Temporal NT-Xent produces a substantial downstream change relative to vanilla NT-Xent, raising AUROC from 0.5417 to 0.7667 and AUPRC from 0.4013 to 0.6673 (paired permutation p = 0.0455 for AUROC). Second, the same frozen 131K-parameter encoder transfers zero-shot to the independent SHHS-1 cohort without any apnea labels or target-domain fine-tuning, reaching AUROC = 0.724 (95% CI [0.641, 0.801]), compared with 0.718 for a single-channel source-to-target control. This probes cross-cohort transfer separately from the in-domain question addressed by the canonical split. Third, a five-seed robustness analysis on the source cohort shows that the magnitude of the in-domain downstream gain is not seed-robust (mean ΔAUROC = +0.0350 ± 0.1600, with A1 exceeding A0 in 2 of 5 seeds). We report this stochastic sensitivity as an empirical characterization of small-cohort EEG self-supervised learning rather than as a caveat to be minimized. Controlled comparisons against an explicit temporal-continuity objective, a SoftCLT-compatible soft instance-wise formulation, and three temporal kernels further show that the size and direction of the in-domain effect depend on how temporal information enters the objective. A representation-level diagnostic indicates that the canonical gain is not explained by increased effective rank. Finally, the compact encoder retains 0.9991 ± 0.0004 cosine similarity between FP32 and INT8 encoder outputs while achieving a 71% reduction in model size, demonstrating a practical consequence of the compact representation for resource-constrained physiological monitoring. Together, these results characterize patient-level temporal structure as a meaningful inductive bias whose in-domain downstream benefit is stochastic in small cohorts, while demonstrating measurable cross-cohort transfer of the learned representation.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Self-Supervised Learning, Contrastive Learning, Temporal Contrastive Learning, EEG Representation Learning, Physiological Time Series, Obstructive Sleep Apnea, Cross-Cohort Transfer, Model Quantization, Edge Deployment

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_NcXzOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes Temporal NT-Xent, a contrastive loss for windowed EEG. Negatives from the same patient are down-weighted in the softmax denominator by w = 1 − exp(−λd), where d is the distance between window indices. Cross-patient negatives keep weight 1 and the augmentation positive is unchanged. A ~131K-parameter STFT-CNN encoder is pretrained for 5 epochs on the training subjects of the OpenNeuro ds008108 sleep cohort (20-channel EEG, 30 s windows). It is then frozen and probed with an MLP for subject-level OSA vs. control classification, with window probabilities averaged per patient. On one patient-level split with 23 test patients, AUROC rises from 0.5417 (vanilla NT-Xent) to 0.7667 (p = 0.0455). Across five seeds, the mean gain is +0.035 ± 0.160 and A1 is better in 2 of 5 seeds. The paper also compares against a continuity regularizer, a PhysioCLR-style loss, an instance-wise SoftCLT adaptation and three weighting kernels. It further reports an effective-rank diagnostic, zero-shot transfer of the frozen encoder to SHHS-1 (AUROC 0.724 vs. 0.718 for a single-channel control), and INT8 quantization (71% smaller, encoder cosine similarity 0.9991).

Critical strengths

The question is well posed and the experimental design is the right kind for it. The paper asks where temporal information should enter a contrastive objective, and compares several answers under one shared encoder, augmentation, budget and probing protocol. Splitting is at the patient level, and the audit documents state that SSL pretraining uses only training subjects, which avoids the transductive leakage common in small EEG studies. This matters because a controlled comparison of this type would be useful to the field. The strength is conditional, though: I could not verify the split code, and some confounds between conditions (below) weaken the “controlled” label.

The authors are unusually open about negative results, and the archive includes per-patient prediction files with patient IDs. The paper reports that the in-domain gain is not seed-robust and that effective rank does not improve, rather than hiding these outcomes. The saved predictions and checkpoints let a reader recompute the reported metrics, which is how I found several of the problems below. This transparency is worth acknowledging even though it ends up exposing that the main claims are not supported.

Critical weaknesses

The paper's contribution is a pretraining objective, but the method is never used the way pretraining is used. The introduction motivates SSL by the "large volumes of unlabeled recordings but comparatively scarce clinical labels" in EEG. Yet the encoder is pretrained only on the 66 training subjects of a single cohort. These are the same labeled subjects later used to train the probe. That is 66 nights of EEG, roughly 450 hours if the training nights are as long as the test nights, and the encoder sees them for five epochs, with the loss still decreasing at the end (fig3b_ssl_training_curves.png). No unlabeled data beyond the labeled training set is used at any point. For comparison, SleepFM was pretrained on more than 100,000 hours of PSG (Thapa et al., ICML 2024). Even the remaining 31 subjects of ds008108, or SHHS itself, which the authors say they had access to, were not used for pretraining.

In this setup, the SSL step works more like a regularized feature extractor fitted to the training set than like pretraining. Results obtained this way say little about how the objective behaves at the scale where pretraining objectives matter. If Temporal NT-Xent is meant as a pretraining scheme, I would expect it to be pretrained on a large, multi-subject corpus and then transferred. Its comparison with vanilla NT-Xent should be made in that setting.

The small data regime also weakens the motivation itself. The premise is that windows recorded close together in the same night are partial false negatives and should repel each other less. The paper never measures this. The value λ = 0.1 (weight 0.63 at five minutes, 0.95 at fifteen) is not linked to sleep-stage bouts or to the 10 to 60 s duration of respiratory events.

The evaluation side is just as small. All in-domain conclusions come from one split with 23 test patients, 8 of them with OSA. With 120 positive-negative pairs, moving a single OSA patient from the top to the bottom of the ranking changes AUROC by 0.125. The headline gain of +0.225 has a paired interval of [0.05, 0.43] and shrinks to +0.035 ± 0.160 across seeds. None of the comparisons between temporal mechanisms in Sections 5.3 and 5.4 are significant across seeds, yet the paper draws general conclusions from them. The task is also narrow: subject-level OSA vs. control classification on one dataset, with no event detection or AHI estimation. That is far from the general claims in the title or the cross-cohort and deployment sections.

A pretraining objective should be compared with those used today. NeuroAtlas (Kontras et al., arXiv 2605.14698, 2026) benchmarks EEG and time-series foundation models with the same frozen-probe protocol used here. It covers 42 datasets, including sleep medicine and brain-age estimation, and finds that model rankings change considerably across datasets within a domain. That finding is the reason one 23-patient split is not enough. Evaluating A0 and A1 on its sleep tasks would show whether the weighting generalizes. Given how recent NeuroAtlas is, I see this as a comparison that should be considered rather than one the authors were required to run. Independently of it, the paper has no supervised baseline, no spectral-feature baseline, and no pretrained encoder such as BIOT, LaBraM (ICLR 2024), CBraMod (ICLR 2025), or SleepFM.

The novelty claim also needs to be discussed against prior work. StatioCL (Wu et al., CIKM 2024) already reweights negatives by temporal distance from the anchor, and Dependent TS2Vec (Marusov et al., arXiv 2025) builds temporal closeness into the contrastive loss. For biosignals, Cheng et al. (2020) use subject-specific negatives, CLOCS (Kiyasseh et al., ICML 2021) uses same-patient positives, and Banville et al. (J. Neural Eng. 2021) use temporal distance on sleep EEG as the training signal.

Beyond the scale problem, the released artifacts do not support the headline results. The archive contains no implementation of the loss, training, evaluation, SHHS, or quantization, only a dataset stub (sleep_apnea_dataset.py). The points below come from the checkpoints, prediction files, and the authors' own audits.

The five-seed analysis does not vary pretraining. The five A0 encoders in results/multiseed/seed_*/ are byte-identical (md5 eb67cec4…). So are the five A1 encoders (d09cc927…) and the five corrected A3 encoders in results/A3_multiseed/ (d1e499c4…). The audit traces this to a hard-coded set_seed(42) in pretrain_ssl() (REPRESENTATION_EFFECTIVE_RANK_AUDIT.md, lines 22 to 26). Tables 2, 7 and 8 therefore measure only downstream MLP variability, contrary to Appendix D and Section 7. A0 was never re-seeded, so there is no valid multi-seed A0 vs. A1 comparison.

The canonical result is also the most favorable of three seed-42 runs in the archive. Those runs give A0 vs. A1 of 0.600 vs. 0.550 (MEGA_EXPERIMENTAL_REPORT.md, §11), 0.600 vs. 0.725 (results/loss_ablation/), and 0.5417 vs. 0.7667 (results/final_primary/, the run used in the paper). Table 1's A3 value of 0.7250 comes from an implementation the authors' fix report says to discard (A3_IMPLEMENTATION_FIX.md, lines 84 to 87); the corrected value is 0.5917. In addition, according to the audit, A1 removes the positive from its denominator while A0 keeps it (A1_IMPLEMENTATION_AUDIT.md, line 251). The A0 vs. A1 comparison therefore changes two things at once.

I could not find evidence for the SHHS-1 result. The only cross-cohort file (cross_cohort_results.npz) is a Sleep-EDF wake/sleep run with AUROC 0.50. P0_FINAL_EXPERIMENTAL_STATUS.md (line 60) states that SHHS transfer was not evaluated. The paper also does not say how the 20-channel encoder was applied to SHHS, how OSA was defined there, or what the control is. Even taken at face value, 0.724 vs. 0.718 says nothing specific about Temporal NT-Xent.

The labels are strongly confounded with demographics. In participants.tsv, controls are younger (median age 24 vs. 34.5) and less often male (49% vs. 84%). On the 23 test subjects, which I recovered from the saved patient IDs, age alone gives AUROC ≈ 0.83 and age plus sex ≈ 0.84. Both are above every SSL result in the paper.

The quantization claims do not match the archive either. Appendix E describes static calibrated W8A8. CODEBASE_AUDIT.md (lines 70 to 76) describes dynamic weight-only quantization that was never run on patient data. The reported 0.7611 is not a possible AUROC with 8 positive and 15 negative patients, since attainable values are multiples of 1/240.

A few smaller points. The 66/22/23 split adds up to 111 subjects, not 142, and the exclusions are not explained. The effective rank of the same A0 encoder is 7.30, 8.08, or 10.51, depending on the file. Preprocessing lacks anti-aliasing. λ was never tuned. Finally, the A1 seed-42 probe assigns every test patient a probability above 0.59.

Questions for authors
  1. Why was pretraining restricted to the 66 labeled training subjects, with no additional unlabeled data? A consistent advantage for A1 over A0 when both are pretrained on a substantially larger corpus would change my assessment the most. The corpus could be the remaining ds008108 subjects plus a public PSG cohort, with the frozen encoders evaluated on several downstream datasets, for example the sleep, epilepsy and brain-age tasks of NeuroAtlas.

  2. The A0 and A1 encoders in results/multiseed/ are byte-identical across seeds. Were Tables 2, 7 and 8 produced with the hard-coded set_seed(42) in pretrain_ssl()? The archive also contains three seed-42 runs, and they disagree on the sign of the gain; why is results/final_primary/ the canonical one? Re-running A0 and A1 with the SSL seed actually varied, ideally over repeated patient-level splits with pretraining inside each fold, would address both points.

  3. Does the A1 used for all reported results exclude the augmentation positive from its denominator, as the audit indicates, while A0 keeps it? If so, could you report decoupled NT-Xent without temporal weights? That would show how much of the A0 to A1 difference comes from the temporal weighting itself.

  4. Please provide the code and prediction files for the SHHS-1 result, and describe the protocol in full. That means how the 20-channel encoder was applied to the SHHS EEG derivations, how OSA was defined, how many participants were included, which head produced the scores, and what the single-channel control is. How does this result relate to cross_cohort_results.npz, which is a Sleep-EDF wake/sleep run at chance, and to the statement in P0_FINAL_EXPERIMENTAL_STATUS.md that SHHS was not evaluated?

  5. On the 23 test subjects, age alone reaches AUROC ≈ 0.83, above every SSL result in the paper. Could you report age and age-plus-sex baselines on the same split, along with an age- and sex-matched or covariate-adjusted evaluation of all objectives? Without this, it is unclear whether the encoders capture OSA-related EEG features at all.

Reviewer_iMXxOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes Temporal NT-Xent a modification of NT-Xent that down-weighs signals close by in the same subject. Cross patient and agumentation positives are unchanged.

The method is evaluated on OSA (though if its classification, AHI regression, or event detection is not stated clearly).

Critical strengths

None

Critical weaknesses
  1. The task and data cannot answer the research question, and the evidence does not seem to support the claim.
  • The prediction target is never defined. "AHI-based labels" with patient-level averaging suggests OSA vs control, but no AHI threshold is given.
  • EEG-only OSA classification is probably a proxy for arousals, since apnea is scored from airflow, effort and SpO2. Staging, outcomes and other labels are not evaluated.
  • Only 142 subjects, although NSRR has tens of thousands of PSGs readily available.
  • The headline gain (0.54 → 0.77) is not significant though this is also not entirely clear
  • SHHS-1 transfer has no vanilla baseline and no task definition, and it equals the single-channel control (0.724 vs 0.718).
  • The tables are inconsistent (A3: Table 1 vs Table 8; exponential kernel vs A1)
  1. Limited novelty and no comparison to the state of the art.
  • Down-weighting same-subject negatives is a small change to NT-Xent.
  • Only 8 references, and no sleep foundation model or apnea baselines (e.g., SleepFM, OSF, ABED, CAISR).
  • The listed contributions seem to be results, and the quantization study is unrelated to the research question.
Questions for authors

Please clarify the task the model is solving.

Please compare to existing models encompassing sleep apnea.

Please state the contribution clearly and do not mix in results.

What is the point of the quanitzation in relation to the proposed method?

Reviewer_PfYLOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper tries to tackle the problem of detecting obstructive sleep apnea from sleep EEG readings where expert labels are scarce. They use self-supervised learning to do this. Current SSL methods for this task ignore the fact that the embeddings from windows of the same patient which are close in time might be related and instead teach the model to learn these as a negative pair. For example, the baseline method NT-Xent learns by pulling an augmented copy of an episode close to the original episode while pushing all other episodes in the batch away from the original episode with equal force. In this paper, the authors propose Temporal NT-Xent which retains the positive pairs and negative cross-patient pairs of NT-Xent but weaken the push-apart force for same-patient negative pairs that are close in time using the weight w = 1 − e^(−λd). The authors claim that their method gives higher AUROC on their test data when compared to the baseline method.

Critical strengths

The application is practical.

Critical weaknesses
  • Writing is unnecessarily verbose, padded, and overly defensive at many places. There are sentences and phrases everywhere which add little meaning to the message that is trying to be delivered and are characteristic to the output of LLMs. A few examples, although there are many more : o L100-101: “two results that answer different questions and must be read together, not as contradictory evidence”- There is no explanation of what the authors mean by this, and what alternate interpretation we should draw o L150-151: “a hypothesis we test empirically rather than assume.”- Since this is the premise of the paper and the whole method is developed around it, it is obvious that the authors will test it empirically rather than assume it. The authors also provide no further explanation on what they mean by this/point to specific areas which the reader can refer to. o Other examples are L376-377, L364-365, L340-341
  • Again, the writing is very scattered. The same results are reiterated in the abstract, Introduction (L72-91) and contributions section (L98-114). By the time someone finishes reading the introduction, they would have been bombarded by the results thrice, while there is minimal discussion of the methodology the authors are trying to propose in these sections.
  • It is unclear what the message/exact contribution of the paper is. There are a lot of results which are unrelated to the possible core method the authors are proposing, and there are places where the authors give experimental results and then go on to say they don’t claim superiority of their method (Example: L398, L340-341)
  • The headline result that is quoted is from the best seed. On an average over all seeds, there is negligible performance gain
  • Section 6, Representation Analysis: Very unclear what the authors are trying to convey. L346-348:“An earlier internal pipeline inadvertently fixed the SSL random seed during checkpoint generation, duplicating encoder checkpoints across nominal seeds; we discard those checkpoints for representation-level claims and use only correctly regenerated, independent checkpoints below.” What was the exact issue? What results were regenerated? What results, if any, were reused, and why are they still valid?
Questions for authors

I believe the paper requires substantial rewriting from scratch to improve its clarity, readability, soundness, and correctness. Even if there is any merit to the method that the authors are trying to propose, their evidence and especially the presentation of the evidence is very poor. I am of the opinion that the paper does not meet the quality standards of the conference, and I don’t have any questions that will change my mind until the paper is substantially rewritten.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.