acceptodds
Under review as a conference paper at ICLR 2027

SemPSG: A Semantic Channel-Aware Foundation Model for Polysomnography Analysis

Abstract

Polysomnography (PSG) integrates multiple physiological signals to provide a comprehensive characterization of human sleep, yet its heterogeneous channel configurations across centers pose substantial challenges for transferable representation learning. Existing foundation models mainly focus on physiological modeling or temporal learning, while channel identity is often treated as a fixed structural index, overlooking the physiological semantics encoded by signal modality and reference configuration. To this end, we propose **SemPSG**, a *Sem*antic channel-aware foundation model for heterogeneous *PSG* analysis. SemPSG explicitly represents the physiological semantics of channel identity and incorporates them into both signal representation learning and channel aggregation, enabling flexible modeling across diverse data configurations. Specifically, a semantic-conditioned time-series encoder captures signal-specific temporal dynamics and cross-signal interactions, while a multi-view image encoder extracts complementary time-frequency and morphological patterns from the same physiological recordings. We evaluate SemPSG on sleep and health-related tasks, including sleep staging, sleep-disorder breathing analysis, disease prediction, cognition and emotion recognition, and demographic estimation. Extensive experiments demonstrate consistent improvements over both general-purpose time series foundation models and PSG-specific foundation models, together with generalization across heterogeneous datasets across diverse channel configurations. The code is available at https://anonymous.4open.science/r/SemPSG.

Primary area:
learning on time series and dynamical systems
Keywords:
Time Series Analysis, Polysomonography, Foundation Models

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_oBfhOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes SemPSG, a sleep foundation model that uses channel semantics to guide the integration of signals within each modality. It combines raw physiological signals with three image views: STFT spectrograms, CWT scalograms, and morphology maps. The model performs strongly across several downstream tasks, including a reported 9.6 percentage-point improvement in macro-F1 over the second-best baseline for sleep-disordered breathing classification. It also presents a novel task - cognition and affective assessment.

Critical strengths
  • The evaluation covers a broad range of downstream tasks and shows strong performance relative to general-purpose time-series and sleep foundation models.
  • Combining raw signals with image views offers a plausible way to capture complementary waveform, spectral, and temporal information.
Critical weaknesses
  • The contribution of the main components remains difficult to assess. The channel-name shuffling visualization suggests that semantics affect the learned gating weights, but quantitative ablations are needed to establish how much channel semantics and each image view improve downstream performance. Comparisons with versions that omit semantic conditioning, omit image views, or use individual image views would help justify the added complexity.
  • Several masked reconstruction objectives use intermediate embeddings as targets. Stop-gradient prevents gradients from flowing through a target within a training step, but the shared components producing that target may change in subsequent steps. The paper does not provide enough information to assess whether these moving targets affect training stability or risk representation collapse. Training diagnostics or an explanation of the stabilization strategy would strengthen this part of the method. For example, JEPA [1] or DINO [2] training uses several tricks to stabilize the training.
  • The paper reports results for a single model size, so it is unclear whether performance improves as the model grows. Experiments across multiple model sizes, with training compute reported, would help establish whether SemPSG shows the scaling behavior observed in recent sleep foundation models [3, 4].
  • The channel semantics tokenizer is a central contribution, but the paper does not show how it handles names absent from its pretraining data. An evaluation that holds out channel names or naming conventions would help determine whether it learns transferable physiological semantics rather than relying on familiar labels.
Questions for authors
  1. Image-view aggregation: The image views aggregate channels using fixed weights derived from window-level signal statistics. How exactly are these weights calculated? Why was this approach chosen instead of using the learned channel semantics for aggregation?
  2. Channel-semantics pretraining: How is (LsemL_{\mathrm{sem}}) computed from the different semantic attributes, including missing or inapplicable labels? How is the direction loss handled for channels without an ordered derivation? Please also clarify how negative examples are selected for the alias and direction objectives - greater number of negative examples is important for contrastive learning to work.
  3. Semantic modulation: Equation (3) defines the modulated representation (z~\tilde{z}), but the subsequent reconstruction and channel-aggregation equations use zz. Is (z~\tilde{z}) used only as the input to self-attention, with zz denoting the resulting contextualized representation? Please make this distinction explicit.
  4. Masking and information leakage: For the time-series reconstruction objective, are tokens masked before or after the convolutional tokenizer? If convolutional receptive fields cross patch boundaries, information from a masked patch might enter neighboring visible tokens. Please clarify the tokenizer’s receptive field and how the implementation prevents this leakage.
  5. Final representation: Why does the final representation retain separate time-series embeddings for each modality but aggregate the image views into a single embedding? What motivates this asymmetry, and how does it affect downstream performance?

Minor issues

  • Please check the index and subscript formatting around lines 200 and 280. In Equation (12), the summation index also appears inconsistent with the indices in the loss term.
  • The inline notation around line 237 is difficult to parse and would benefit from clearer formatting.

[1] Assran, Mahmoud, et al. "Self-supervised learning from images with a joint-embedding predictive architecture." 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2023.

[2] Caron, Mathilde, et al. "Emerging properties in self-supervised vision transformers." 2021 IEEE/CVF international conference on computer vision (ICCV). IEEE, 2021.

[3] Shuai, Zitao, et al. "OSF: On pre-training and scaling of sleep foundation models." arXiv preprint arXiv:2603.00190 (2026).

[4] Carter, Jonathan F., and Lionel Tarassenko. "Next-Token Prediction Learns Generalisable Representations of Sleep Physiology." arXiv preprint arXiv:2606.09605 (2026).

6 comments
AuthorsOpenReview ↗

Author Response (W1-W4)

Many thanks to Reviewer oBfh for providing detailed and in-depth reviews. We address your concerns by providing more experiments and clarifications as follows.

W1: Ablations of individual components.

Table 1 presents ablations of channel semantics, image views, and pretraining objectives. Dropping any component lowers all three metrics, with inter modality reconstruction having the largest impact, confirming that all components complementarily contribute to the full model.

Table 1: Ablation study performance(%) for downstream sleep staging on the SHHS-1 dataset.

VariantAccuracyKappaMacro-F1
w/o Channel Semantics Tokenizer82.174.874.2
w/o Intra-Modality Reconstruction81.373.673.5
w/o Inter-Modality Reconstruction80.872.973.0
w/o CWT Scalogram81.173.673.7
w/o STFT Spectrogram81.974.274.0
w/o Morphology Envelope82.174.774.3
w/o Multi-Views Contrastive Learning81.473.873.8
w/o Masked Image Modeling81.874.273.9
w/o Align82.174.774.3
SemPSG82.475.274.6

W2: Training stability of masked latent reconstruction.

SemPSG computes reconstruction targets online using shared encoders, with stop-gradient applied to the target branch and no EMA teacher. Masked reconstruction is jointly optimized with cross-view and waveform-image InfoNCE losses from the start of training. These losses align representations of the same window and separate negatives, discouraging constant representations. We also use a 2,000-step learning-rate warmup, gradient clipping at norm 1, and a gradual increase in the cross-modal reconstruction weight to ease early optimization.

Pretraining completed full steps updates without reported NaNs. Final validation InfoNCE losses were 0.0130 (cross-view) and 0.00395 (waveform-image), supporting window-level discrimination and providing evidence against complete collapse at this level. Our assessment of training stability therefore rests on the joint objectives, optimization settings, and training records. We will add the figure of training and valiadation losses in the revised paper soon.

W3: Model scaling and training compute.

We have added Small/Large variants with 2.64/11.70M parameters. On downstream SHHS1 sleep staging, they achieve AUROC of 95.56% and 95.72%, respectively. The model in the original submission corresponds to the Base variant with 4.45M parameters, which achieves 95.69% AUROC. The figure below summarizes this comparison. We will add this comparison results to the revised paper soon.

Table 2: Architecture configurations and parameters across SemPSG scales.

ModelIntra layersInter layersImage layersHidden DHeadsParamsNPU-Hours
SemPSG-Small11112842.64M147.2 NPU-h
SemPSG-Base22212844.45M159.3 NPU-h
SemPSG-Large666128411.70M185.7 NPU-h

W4: Generalization to unseen channel names and naming conventions.

To test whether the channel semantics tokenizer learns transferable physiological semantics rather than relying on familiar labels, we evaluated CFS while holding out all original channel names. We replaced all CFS channel names in all splits with equivalent physiological descriptions absent from the CNE pretraining corpus, such as changing the abbreviated label THOR EFFORT to Thoracic Respiratory Effort, keeping signals and derivations fixed. Retraining the linear classifier on the frozen encoder with identical settings yields 82.1% Accuracy, 74.9% Kappa, and 73.7% Macro F1 , as shown in Table 3, matching the original familiar labels (81.9% Accuracy, 74.8% Kappa, and 73.6% Macro F1) and confirming transferable semantic understanding over label memorization.

Table 3: Sleep staging performance (%) on CFS under original and unseen channel naming conventions.

NameAccuracyKappaMacro-F1
Original Channel Name (SemPSG)81.974.873.6
Unseen Channel Name82.174.973.7
AuthorsOpenReview ↗

Author Response (Q1-Q5)

Q1: Channel aggregation in the image branch.

For each 30-second window, we normalize the standard deviations of valid channels within each physiological modality to obtain the channel weights:

, \epsilon = 10^{-6}.$$ Here, $\sigma_c$ is the channel standard deviation, $\mathcal{C}_m$ is the set of valid channel in modality $m$ and $\epsilon$ prevents division by zero. We chose this approach to generate image and reuse them during pretraining, avoiding repeated signal transformations. Semantic aggregation weights that change during training would require reaggregating channel-level feature maps after weights updates, preventing direct reuse of cached images. Thus, we use learned aggregation guided by channel semantics in the waveform branch. The image branch uses deterministic statistical aggregation to provide complementary time-frequency and morphological information. > **Q2: Channel semantics training objectives and negative sampling.** The pretraining objective is $L_{\mathrm{sem}} = L_{\mathrm{attr}} + L_{\mathrm{alias}} + \lambda_{\mathrm{dir}} L_{\mathrm{dir}}$, where $\lambda_{\mathrm{dir}}$ weights the direction loss. $L_{\mathrm{attr}}$ sums weighted prediction losses over four attributes (signal type, derivation, location, and sensor), computed strictly on known applicable labels while masking out missing or inapplicable ones. Channels without an explicit ordered derivation are similarly masked out from $L_{\mathrm{dir}}$. For alias contrastive learning, $L_{\mathrm{alias}}$ uses all eligible candidate names within the training split, providing about 261 negative examples per valid anchor on average. Verified aliases form positive pairs, while names with known semantic differences form negative pairs weighted by their semantic relationships, excluding uncertain pairs to reduce false negatives. $L_{\mathrm{dir}}$ uses triplets comprising an anchor name, an equivalent alias, and a name with reversed electrode order. Reversed negatives are taken from existing training names when available or generated by swapping electrode order, excluding any candidates that overlap with validation or test names. > **Q3: Use of semantically modulated representations.** In the following Equation, $\mathbf{\tilde{z}}$ is used to compute self-attention on the modulated representations. The $\mathbf{z}$ in the reconstruction and channel aggregation equations denotes the contextualized output after self-attention, residual connections, and feed-forward layers. Both operations use representations conditioned on channel semantics. The aggregation module also uses the semantic embeddings directly to compute its weights. $\widetilde{\mathbf{z}}^{(t)} _ {m,c,p} = \left(1 + \gamma^{(t)} _ {m,c}\right) \odot \operatorname{LN}\left(\mathbf{z}^{(t)} _ {m,c,p}\right) + \beta^{(t)} _ {m,c}.$ We will update the reconstruction and aggregation equations to distinguish it from the modulated attention input. > **Q4: Masking strategy and potential information leakage.** The mask strategy is applied after CNN tokenization and before context encoding. We first divide the waveform into 60 non-overlapping 0.5-second patches, then perform convolution and normalization on each patch independently, ensuring that information does not propagate to adjacent patches. The masked tokens are subsequently replaced with learnable mask tokens, while the unmasked tokens serve as reconstruction targets under stop-gradient. > **Q5: Asymmetric aggregation of time-series and image embeddings.** The asymmetry aggregation reflects their distinct scopes, since each temporal token represents a single modality (EEG, EOG, EMG, ECG, or respiration) whereas each image view already covers all modalities in the PSG window. Retaining separate temporal tokens preserves modality specific dynamics for flexible downstream weighting, while fusing CWT, STFT, and morphological views via learned weights provides a compact global summary without redundancy, jointly improving downstream performance. > **Minor issues: Mathematical notation and formatting.** Thank you for pointing this out. We will fix all the mathematical notation and formatting issues in the updated manuscript soon.

I thank the authors for the additional experiments and clarifications. The new ablations help understand each component's contribution, and the explanations address several of my concerns regarding the implementation.

My main remaining concern is whether the benefits of using channel-semantic conditioning, one of the main contributions, appears marginal relative to the added complexity. Removing the channel-name encoder reduces accuracy by only 0.3 percentage points in Table 1. For context, changing the channel-naming condition in Table 3 produces a 0.2-percentage-point difference in accuracy.

I also appreciate the model-size comparison. Scaling is not the paper’s primary contribution, and proportional performance improvements are not expected. Nevertheless, the AUROC gain from Base to Large is small, and reporting the original metrics would make the comparison easier to interpret.

Overall, the rebuttal improves the clarity and supporting evidence, but it does not sufficiently resolve my concern about the demonstrated value of the paper’s central contribution. I therefore maintain my current rating.

Reviewer_8otiOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes SemPSG, a channel-semantic-aware foundation model for heterogeneous polysomnography (PSG) analysis. It introduces a channel semantics tokenizer, hierarchically models intra- and inter-modality dependencies, and integrates time-series and multi-view image representations through self-supervised learning. Pretrained on nine PSG cohorts, SemPSG is evaluated on six downstream task groups. Experiments show improvements over general-purpose and sleep-specific foundation models, including a 9.6 percentage-point Macro-F1 gain in sleep-disordered breathing classification, as well as promising cross-dataset generalization.

Critical strengths
  1. Meaningful channel-semantic modeling: The channel semantics tokenizer explicitly incorporates physiological information from channel identities, providing a promising approach to handling heterogeneous PSG channel configurations.
  2. Comprehensive representation learning framework: The hierarchical intra- and inter-modality encoders, combined with multi-view time-frequency representations, provide a well-motivated framework for capturing complementary physiological information.
Critical weaknesses
  1. Unclear architectural presentation: Figure 2 lacks clear annotations of module inputs/outputs, intermediate representations, and signal flows. Inconsistent mathematical notations further reduce readability.
  2. Insufficient implementation details: Several important components are not adequately specified, particularly the construction of structured channel annotations, the pretraining procedure of the channel semantics tokenizer, and the generation of morphology representations.
  3. Lack of essential ablation studies: The paper does not provide systematic ablations of the channel semantics tokenizer, intra- and inter-modality encoders, multi-view image representations, or pretraining objectives. This makes it difficult to determine which components contribute to the reported improvements and whether the added complexity is justified.
  4. Insufficient validation of core claims: The paper lacks controlled experiments on unseen channels and varying channel configurations. Moreover, processing independent 30-second windows raises questions about the claimed long-range temporal modeling capability.
  5. Limited and inconsistent performance gains: Improvements over competitive baselines are modest on several tasks. For example, accuracy and Macro-F1 on SHHS1 sleep staging improve by only 0.5 percentage points over SleepMaMi. In disease prediction, SemPSG achieves the best results on four of six outcomes, but some gains are marginal while performance on the remaining outcomes is worse. These results do not consistently justify the additional architectural complexity.
Questions for authors
  1. How are the structured channel-semantic annotations constructed, and what supervision is used to train the channel semantics tokenizer?
  2. Can the authors provide ablation studies to quantify the individual contributions of channel semantics, hierarchical encoders, multi-view representations, and pretraining objectives?
  3. Can the authors provide controlled experiments to demonstrate robustness to unseen channel identities, missing channels, and different channel configurations?
  4. Given that the model processes independent 30-second windows, how does it capture the claimed long-range temporal dependencies?
  5. How do the authors ensure fair comparisons with baselines that differ in pretraining data, model capacity, and input configurations?
  6. Could the authors clarify the input/output relationships and signal flows in Figure 2, particularly how different PSG modalities are processed?
4 comments
AuthorsOpenReview ↗

Author Response (W1-W3, Q1-Q2, Q5-Q6)

Thank you very much for the invaluable and in-depth comments. We address your problems by providing more experiments and clarifications as follows.

W1 & Q6: Clarity of Figure 2 and mathematical notation.

We will revise Figure 2 to annotate all intermediate representations and unify mathematical notations with the main text, using distinct arrows for channel name flow (blue), signal and representation flow (black), and pretraining supervision (dashed gray). In the waveform branch, EEG, EOG, EMG, ECG, and RESP signals undergo intra-modality encoding and semantic-gated channel aggregation conditioned on channel name embeddings, followed by inter-modality encoding and temporal pooling to yield five 128D tokens. In the image branch, three views (CWT, STFT, and morphology) are encoded and aggregated into one 128D token. The final output concatenates these six unprojected tokens as Z=[u1,…,u5,zimg]∈R6×128Z = [u_1, \ldots, u_5, z_{\text{img}}] \in \mathbb{R}^{6 \times 128}, where umu_m denotes a physiological modality token and zimgz_{\text{img}} denotes the image token. Mean pooling and projection heads belong only to the pretraining alignment path. We will upload the final revised paper soon.

W2 & Q1: Implementation details of the channel semantics tokenizer and morphology representations.

We have clarified the implementation details of the channel semantics tokenizer and morphology representations as follows:

  • Annotations. The resource contains 882 supervised channel names from cohort inventories and alias resources. We derive four attributes from naming rules and dataset metadata, covering signal type, derivation (ordered active/reference), anatomical site, and sensor. Unknown or inapplicable labels are masked.
  • Tokenizer pretraining. The UTF-8 tokenizer uses CNN and Transformer layers and is pretrained through attribute prediction, verified alias contrast, and direction loss using reversed electrode order. During PSG pretraining, the tokenizer is frozen and its adapter is trainable.
  • Morphology generation. Maps are computed offline without learned parameters. Each 30-s channel window is divided into 80 bins. The four statistics per bin are RMS amplitude, mean absolute and mean squared first differences, and mean-centered zero-crossing rate. Channel maps are aggregated within each modality, normalized, and resized to form a 160×160 image with five modality strips.

W3 & Q2: Ablations of individual components.

Table 1 presents ablation studies of channel semantics, intra-modality, inter-modality encoding, image views, and pretraining objectives. Dropping any component lowers all three metrics, with inter modality reconstruction having the largest impact, confirming that all components complementarily contribute to the full model.

Table 1: Ablation study performance(%) for downstream sleep staging on the SHHS-1 dataset.

MetricsAccuracyKappaMacro-F1
w/o Channel Name Encoder82.174.874.2
w/o Intra-Modality Reconstruction81.373.673.5
w/o Inter-Modality Reconstruction80.872.973.0
w/o CWT Scalogram81.173.673.7
w/o STFT Spectrogram81.974.274.0
w/o Morphology Envelope82.174.774.3
w/o Multi-Views Contrastive Learning81.473.873.8
w/o Masked Image Modeling81.874.273.9
w/o Align82.174.774.3
SemPSG82.475.274.6

Q5: Fair comparisons with baselines in pretraining data, model capacity, and input configuration.

We have explained the fair comparisons with baselines in pretraining data, model capacity, and input configuration below.

  • Pretraining data. Fully matching pretraining corpora across released foundation models is infeasible because some rely on proprietary clinical data, such as the private BioSerenity cohort used by SleepFM. We therefore compare the released pretrained models under aligned downstream evaluation protocols.
  • Model capacity. SemPSG has 4.45M parameters, which is comparable to SleepFM (4.83M) and substantially smaller than MOMENT-Base (125M). Despite this compact size, SemPSG outperforms both models on the SHHS sleep staging benchmark.
  • Downstream protocol and inputs. We follow the SHHS and CFS data partitioning protocol of SleepMaMi, using identical training, validation, and test splits along with the same evaluation metrics across all baselines. Each model retains its required input format.
AuthorsOpenReview ↗

Author Response (W4-W5, Q3-Q4)

W4 (a) & Q3: Generalization to unseen channels and configurations.

We have conducted two controlled experiments on the unseen CFS cohort to directly evaluate model robustness to unseen channel identities, missing channels, and varying channel configurations.

  • Unseen Identities. We replaced CFS channel names in all splits with equivalent aliases absent from the channel name corpus, such as THOR EFFORT to Thoracic Respiratory Effort, keeping the time series signals and derivations unchanged. As illustrated in Table 2, retraining the linear classifier of the frozen channel name encoder with unseen names gives a negligible increase of 0.1 percentage points in Macro-F1, indicating strong trasnferbility of SemPSG on unseen channel names.
  • Missing Channels. With the channel name encoder and classifier frozen, we dropped channels during testing and regenerated image inputs on all 201 test records. As show in Table 2, without the channel C3 or LOC, the Macro-F1 only decreases by only 0.3 and 1.1 percentage points, respectively, demonstrating robustness to single channel missing. As shown in Table 3, randomly dropping 25% of channels across three random seeds, reduces accuracy by 2.6 percentage points, indicating that SemPSG could avoid performance collapse under heavy channel missing.

Table 2: Sleep staging performance (%) on CFS under original, unseen channel naming conventions and missing channels.

NameAccuracyKappaMacro-F1
Original Channel Name (SemPSG)81.974.873.6
Unseen Channel Name82.174.973.7
Remove C3 Channel81.774.573.3
Remove LOC Channel81.373.872.5

Table 3: Random removal of 25% of channels on CFS across three random seeds.

SeedAccuracyKappaMacro-F1
4278.870.769.9
4379.471.370.4
4479.771.971.1
mean ± std79.3 ± 0.571.3 ± 0.670.5 ± 0.6

W4 (b) & Q4: Temporal modeling and the long-range 30-second window setting.

"Long-range" is defined relative to the local patch scale and refers to temporal dependencies across patches with a 30-second window [1]. For example, the EEG sampled at 256 Hz, each 30-second window contains 256 Hz * 30s = 7,680 time-series points, which already consititutes a long sequence in raw signal modeling, therefore it could be defind as a long-range signal in existing studies [1, 2]. Moreover, the 30-second window follows the standard clinical scoring epoch in AASM guidelines [3] and represents a practical trade-off between temporal coverage and computational cost. We will clarify this temporal scope in the revised version to explicitly distinguish window-level dependencies from cross-window or whole-night dependencies.

[1] Wang, et al. Cbramod: A criss-cross brain foundation model for eeg decoding. ICLR, 2025.

[2] Thapa et al. A multimodal sleep foundation model for disease prediction. Nature Medicine, 2026.

[3] Malhotra R K. AASM Scoring Manual 3: a step forward for advancing sleep care for patients with obstructive sleep apnea. Journal of Clinical Sleep Medicine, 2024.

W5: Limited and inconsistent performance gains.

Table 4: Disease prediction C-Index (rank), Avg, and Mean Rank on SHHS1.

ModelCHD DeathCVD DeathStrokeCHFMIAnginaAvgMean Rank
SleepFM0.781 (2)0.791 (2)0.729 (2)0.764 (3)0.636 (3)0.632 (4)0.7222.67
SleepMaMi0.776 (3)0.788 (3)0.718 (3)0.793 (2)0.662 (1)0.778 (1)0.7532.17
SemPSG0.837 (1)0.836 (1)0.751 (1)0.798 (1)0.660 (2)0.727 (2)0.7681.33
Δ\Delta vs. Best+0.056+0.045+0.022+0.005-0.002-0.051+0.0161st

As shown in Table 4, baselines exhibit sharp trade-offs across tasks. SleepMaMi ranks 1st on Angina and MI, whereas it drops to 3rd on CHD Death, CVD Death, and Stroke. Conversely, SleepFM ranks 2nd on those three severe outcomes, yet it drops to 4th on Angina. SemPSG is the only model ranking 1st or 2nd across all six endpoints, achieving the best Mean Rank (1.33) and Avg C-Index (0.768).

SemPSG virtually ties on MI (-0.002) while delivering large gains on severe fatal and cerebrovascular outcomes (+0.056, +0.045, and +0.022 over the best baselines, or +0.061, +0.048, and +0.033 over SleepMaMi). Moreover, on SHHS1 sleep staging, although Macro-F1 improves by +0.5%, SemPSG boosts Kappa from 70.0% to 75.2%, outperforming SleepMaMi by +5.2 percentage points. These substantial gains in expert-level staging agreement and mortality prediction justify the architecture of SemPSG.

I Appreciate your efforts for the additional experiments and clarifications. The new experiments help clarify the contributions of individual components.

However, the improvements still seem relatively modest given the additional model complexity. While the rebuttal addresses some of my concerns, I am still not fully convinced of the value of the paper's core contributions. I will therefore maintain my current rating.

Dear Reviewer 8oti,

Thank you very much for your follow-up and valuable comments. We further clarify the methdological and experimental contributions of our work below for your reference.

1. Methodological Contributions.

SemPSG brings together three complementary components for learning from heterogeneous PSG recordings. The channel semantics tokenizer encodes the physiological information in channel identities and uses it to guide waveform encoding and channel aggregation across naming conventions. The semantic-aware time-series branch models relationships among channels within each modality and interactions across modalities. The time-frequency and morphology branch captures spectral and local waveform characteristics that complement the temporal representation. Cross-modal alignment links these views to the waveform representation, allowing the same encoder to serve six groups of sleep and health-related tasks.

2. Experimental Contributions.

Overall, SemPSG demonstrates strong and balanced performance across six downstream task groups, with efficient model capacity and promising cross-dataset generalization.

2.1 Sleep and Health-Related Task Contributions.

SemPSG achieves competitive performance across diverse sleep and health-related tasks. On SHHS1, SemPSG improves sleep staging Kappa by 5.2 percentage points over SleepMaMi and sleep-disordered breathing classification Macro-F1 by 9.6 percentage points. It also reduces AHI estimation MAE by 9.4% relative to SleepFM. On APPLES, SemPSG achieves the lowest prediction errors for both cognitive and affective assessments.

Table 4: Disease prediction C-Index (rank), Avg, and Mean Rank on SHHS1.

ModelCHD DeathCVD DeathStrokeCHFMIAnginaAvgMean Rank
SleepFM0.781 (2)0.791 (2)0.729 (2)0.764 (3)0.636 (3)0.632 (4)0.7222.67
SleepMaMi0.776 (3)0.788 (3)0.718 (3)0.793 (2)0.662 (1)0.778 (1)0.7532.17
SemPSG0.837 (1)0.836 (1)0.751 (1)0.798 (1)0.660 (2)0.727 (2)0.7681.33
Δ\Delta vs. Best+0.056+0.045+0.022+0.005-0.002-0.051+0.0161st

For disease prediction on SHHS1, it ranks first or second across all six outcomes, achieving the best mean rank (1.33) and highest mean C-index (0.768). These results demonstrate that SemPSG learns representations applicable to diverse tasks. For the complexity concern, SemPSG improves performance on downstream sleep analysis tasks with 4.45M parameters, which is about 3.4% of the parameter count of SleepMaMi [1]'s parameter count.

Reference

[1] Keondo Park, et al. SleepMaMi: A universal sleep foundation model for integrating macro- and micro-structures. ICML, 2026.

2.2 Cross-dataset Generalization Contributions.

SemPSG demonstrates promising cross-dataset generalization. On CFS, a cohort entirely excluded from PSG pretraining, SemPSG maintains competitive sleep staging performance. Additional experiments demonstrate robustness to unseen channel-name aliases and missing channels.

Table 5: Ablation studies of channel-semantic conditioning on the unseen CFS cohort.

ModelAccuracyKappaMacro-F1
w/o Channel Semantics Tokenizer79.571.270.8
SemPSG81.974.873.6

Moreover, as shown in Table 5, removing channel-semantic conditioning on CFS reduces Accuracy by Kappa by 3.6 percentage points, highlighting its contribution to representation transfer across datasets.

We hope the new responses and the updated paper could address your problems regarding paper contributions. If there are further questions, please feel free to let us know. We appreciate your feedback and would welcome any further discussions.

Best regards,

Authors

Reviewer_mQDWOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes SemPSG, a foundation model for learning transferable representations from heterogeneous polysomnography recordings. The model encodes channel names into physiological semantic embeddings and uses them to condition waveform processing and channel aggregation. It models dependencies within and across physiological modalities through hierarchical encoders, while a parallel image branch captures time-frequency and waveform patterns using CWT, STFT, and morphology views. Pretraining combines masked latent reconstruction with contrastive learning across image views and between waveform and image representations. The authors pretrain SemPSG on nine cohorts and evaluate its frozen representations on multiple sleep and health-related tasks, reporting improvements over general-purpose and sleep-specific foundation models, including on an external cohort excluded from pretraining.

Critical strengths
  1. The paper tackles a practical challenge in PSG modeling: heterogeneous channel identities and configurations across datasets.
  2. The hierarchical architecture provides a reasonable way to model within-modality channel relationships and cross-modality interactions.
  3. Evaluation covers diverse sleep and health-related tasks with a frozen encoder, including an external cohort excluded from pretraining. The improvement in sleep-disordered breathing classification is particularly notable.
Critical weaknesses
  1. The framework combines many modules and training objectives, but the experiments do not show why this level of complexity is needed. I would like to see ablations that isolate the contributions of channel semantics, hierarchical intra-/inter-modality encoding, and each image view. Comparisons with simpler alternatives would also help determine whether the added complexity pays off.

  2. The evaluation on CFS is useful, but it does not directly test generalization to unseen channel configurations. Did the channel tokenizer encounter CFS channel names or their semantic signatures during training? Testing new derivations, alternative naming conventions, and missing channels would provide stronger evidence for this claim.

  3. The paper gives too little detail about the data used to train the channel-semantics tokenizer. How did the authors collect the attribute annotations, construct alias pairs, and generate directional negatives? Reporting the data sources, sizes, and splits would make the method easier to reproduce and clarify what counts as an unseen channel.

  4. Some improvements are small, and the paper does not report confidence intervals or variation across runs. For example, sleep-staging Macro-F1 on SHHS1 increases from 74.1 to 74.6. Without uncertainty estimates, it is difficult to tell whether this gain is reliable.

Questions for authors
  1. What evidence shows that channel-semantic conditioning improves downstream performance over a model without it?
  2. Which channel names or configurations in CFS, if any, were unseen during tokenizer training?
4 comments
AuthorsOpenReview ↗

Author Response (W1 & Q1, W2 & Q2)

Many thanks for your detailed and constructive feedback. We have been trying our best to address your commnets. Please see the response as follows.

W1 & Q1: Ablations of individual components.

We have conducted ablation studies of without channel semantics, intra-modality, inter-modality encoding, and each image view. The results are shown in Table 1, channel-semantic conditioning provides gains across all three metrics, directly supporting its downstream value. Reconstruction objectives contribute most, particularly inter-modality reconstruction, while image views and auxiliary objectives provide additional gains. These components offer complementary benefits, with the full model achieving the best results across all three metrics.

Table 1: Ablation study performance(%) for sleep staging on SHHS-1 dataset.

VariantAccuracyKappaMacro-F1
w/o Channel Name Encoder82.174.874.2
w/o Intra-Modality Reconstruction81.373.673.5
w/o Inter-Modality Reconstruction80.872.973.0
w/o CWT Scalogram81.173.673.7
w/o STFT Spectrogram81.974.274.0
w/o Morphology Envelope82.174.774.3
w/o Multi-Views Contrastive Learning81.473.873.8
w/o Masked Image Modeling81.874.273.9
SemPSG82.475.274.6

W2 & Q2: Channel generalization on CFS evaluation.

The PSG recordings of CFS dataset were entirely unseen during pretraining, serving as an unseen signal cohort. While the original CFS channel names and configurations appeared during training process of the channel name encoder. To achieve both unseen channel sematics and unseen PSG recordings, we have conducted two new controlled experiments on CFS to directly evaluate generalization to unseen naming conventions and unseen channel configurations.

  • Unseen Name. We replaced CFS channel names in all splits with equivalent aliases absent from the channel name corpus, such as THOR EFFORT to Thoracic Respiratory Effort, keeping the time series signals and derivations unchanged. As illustrated in Table 2, retraining the linear classifier of the frozen channel name encoder with unseen names gives a negligible increase of 0.1 percentage points in Macro-F1, indicating strong trasnferbility of SemPSG on unseen channel names.
  • Missing Channels. With the channel name encoder and classifier frozen, we dropped channels during testing and regenerated image inputs on all 201 test records. As show in Table 2, without the channel C3 or LOC, the Macro-F1 only decreases by only 0.3 and 1.1 percentage points, respectively, demonstrating robustness to single channel missing. As shown in Table 3, randomly dropping 25% of channels across three random seeds, reduces accuracy by 2.6 percentage points, indicating that SemPSG could avoid performance collapse under heavy channel missing.

Table 2: Sleep staging performance (%) on CFS under original, unseen channel naming conventions and missing channels.

NameAccuracyKappaMacro-F1
Original Channel Name (SemPSG)81.974.873.6
Unseen Channel Name82.174.973.7
Remove C3 Channel81.774.573.3
Remove LOC Channel81.373.872.5

Table 3: Random removal of 25% of channels on CFS across three random seeds.

SeedAccuracyKappaMacro-F1
4278.870.769.9
4379.471.370.4
4479.771.971.1
mean ± std79.3 ± 0.571.3 ± 0.670.5 ± 0.6
AuthorsOpenReview ↗

Author Response (W3, W4)

W3: Details about training data for the channel semantics tokenizer.

We have clarified dataset scale, attributes, alias and direction supervision below.

  • Dataset scale and attributes. We collect 882 unique channel strings from multi-cohort channel inventories and alias resources (740 train, 71 validation, and 71 test), covering EEG (354), EOG (55), EMG (191), ECG (33), and respiratory/other (249). Based on the overview pages of public PSG datasets, each string is parsed into 4 structured attributes covering signal type, lead configuration, anatomical site, and sensor. Active and reference electrodes are logged separately to retain lead polarity, while ambiguous attributes are labeled unknown and masked in the supervised loss.
  • Alias and direction supervision. We construct alias positive examples by pairing names that share the same physiological meaning and lead configuration, such as E1-M2 and E1M2 where both specify E1 as the recording electrode and M2 as the reference. Directional negative examples are built by reversing the recording and reference electrodes, such as pairing E1-M2 with M2-E1. We prioritize reversed names already in the naming resource and generate missing ones via naming rules for training only. Channels with ambiguous direction are excluded from the directional loss.

W4: Statistical reliability of performance gains.

To assess statistical reliability in downstream training, we repeated the sleep-staging evaluation on SHHS1 with five continues random seeds in Table 4, we can see that all three metrics shows limited variation across seeds. Moreover, Kappa consistently exceeds the strongest reported baseline Zeus by 4.3 percentage points, indicating that the gain of SemPSG is reasonably reliable.

Table 4: Sleep staging results (%) on SHHS1 across five random seeds, and mean ± standard deviation.

SeedAccuracyKappaMacro-F1
42 (SemPSG)82.475.274.6
4382.374.874.0
4482.074.774.5
4581.774.274.1
4682.375.074.6
mean ± std82.1 ± 0.374.8 ± 0.474.4 ± 0.3

I appreciate the authors’ efforts in providing additional experiments. While the rebuttal addresses some of my concerns, I am not convinced by the methodological contribution of this work, so I will maintain my original rating.

AuthorsOpenReview ↗

Response to Reviewer mQDW

Dear Reviewer mQDW,

Thank you for your follow-up and valuable comments. We further clarify the methodological contributions of our work below for your reference.

1. SemPSG components address three distinct issues.

SemPSG integrates three complementary components to address key challenges in heterogeneous PSG representation learning. First, the channel semantics tokenizer encodes structured physiological information from channel identities, enabling generalization across different channel naming conventions and acquisition systems. Second, the semantic-aware time series component models intra- and inter-modality dependencies, separating channel-level relationships within each physiological modality from interactions across different modalities. Third, the time-frequency and morphology component captures spectral and local waveform characteristics to balance multi-scale temporal dynamics. These components are jointly integrated through semantic conditioning, hierarchical physiological modeling, and cross-modal alignment. Together, they enable SemPSG to capture channel semantics, temporal characteristics, and physiological interactions within a unified representation framework to understand PSG signals that benefit clinician workflows.

2. SemPSG generalizes to unseen channel names and remains robust to missing channel secnarios.

To evaluate generalization to unseen channel names, we replaced the original channel names in the CFS dataset with unseen aliases. SemPSG achieved comparable performance to that obtained with the original names (Table 2 of our first response). To assess robustness to missing channels, we kept both the encoder and classifier fixed. Removing either the C3 or LOC channel had minimal impact on sleep staging performance (Table 2 of our first response). Even when randomly removing 25% of the channels, the mean accuracy across three random seeds decreased by only 2.6 percentage points compared with the full-channel setting (Table 3 of our first response). These results demonstrate SemPSG's ability to generalize to unseen channel-name aliases while maintaining strong downstream performance under missing-channel conditions.

3. The channel semantics guide waveform learning and time series channel agregation

Consider three channel names E1-M2, E1M2, and M2-E1 collected by different devices. The first two represent the same measurement, whereasM2-E1 uses the same electrodes with reverses roles. Despite sharing the same modality label, these channels encode different physiological semantics. The Channel Semantics Tokenizer learns these relationships by aligning equivalent channel names in the embedding space while distinguishing reversed derivations. The resulting semantic embeddings modulate waveform patches before attention and guide channel weighting during aggregation. As shown in the right panel of Figure 5 (in our revised manuscript), shuffling channel names within the same recording changes the aggregation weights despite identical waveform inputs. This demonstrates that SemPSG incorporates channel identity into modality-level representation learning rather than relying solely on waveform features.

We hope the new responses and the updated paper could address your problems regarding method novelty. If there are further questions, please feel free to let us know. We would welcome a closer discussion of any aspect of the methodolofical design.

Best regards,

Authors

Comments on the paper

1 comment
AuthorsOpenReview ↗

Author Response to All Reviewers

We thank all three reviewers for their constructive and insightful comments. The table below summarizes all revisions made in response to their Weaknesses and Questions. Please note that the page and line numbers refer to the revised version of our submission, with all modifications highlighted in blue.

Summary of Revisions in Response to Reviewer Comments

ReviewerPageLineW&QModification
mQDW10, 25–26486–494, 1315–1354W1&Q1Added a summary of ablation studies in Section 3.7 and detailed settings and results in Appendix A.12.
mQDW20, 23–251062–1068, 1220–1241, 1294–1314W2&Q2Clarified the pretraining coverage of CFS dataset in Appendix A.7 and added unseen-name and missing-channel experiments in Appendix A.11.
mQDW4, 16–18175–180, 858–880, 909-932W3Added the channel-name dataset split and annotation procedure in Section 2.1 with alias and direction supervision in Appendix A.5.
8oti3108–128W1&Q6Revised Figure 2 of overall framework, and its caption to label intermediate representations and signal flows in Section 2.
8oti4, 16–18, 5175–180, 858–880, 909–932, 263–267W2&Q1Expanded tokenizer data and supervision details in Section 2.1 and Appendix A.5. Clarified the construction of morphology features in Section 2.2.
8oti10, 25–26486–494, 1315–1354W3&Q2Added component ablations in Section 3.7 and Appendix A.12, covering channel semantics, reconstruction, image views, and auxiliary objectives.
8oti23–251220–1241, 1294–1314W4(a)&Q3Added sleep staging evaluations with unseen aliases, individual channel removal, and random removal of 25% of channels in Appendix A.11.
8oti298–101W4(b)&Q4Clarified that long-range dependencies refer to interactions across patches within each 30-second window in Section 1.
oBfh10, 25–26486–494, 1315–1354W1Added component ablations and their analysis in Section 3.7 and Appendix A.12.
oBfh10496–512W2Added Figure 6 showing training and validation losses in Section 3.8.
oBfh10513–530W3Added the model scaling experiments including Table 5 presenting model scaling configurations, and the Figure 7 of sleep-staging across model sizes, and a discussion of results in Section 3.9.
oBfh20, 23–251062–1068, 1220–1241, 1294–1314W4Clarified CFS channel-name exposure and added evaluation with unseen physiological aliases in Appendix A.11.
oBfh16–18858–880, 909–932Q2Added the tokenizer objective, label masking, alias negative sampling, and direction supervision in Appendix A.5.
oBfh4196–201Q3Distinguished the Transformer input, modulated attention input, and contextualized output used for reconstruction and channel aggregation in Section 2.1.
oBfh4203–208Q4Clarified patch-independent convolution and normalization, masking placement, and reconstruction targets in Section 2.1.
oBfh4207–208Minor IssueCorrected the reconstruction-head subscript formatting before Equation (4).
oBfh5243–244Minor IssueCorrected the temporal sequence notation before Equation (9).
oBfh6288–289Minor IssueCorrected the target representation subscripts before Equation (12).
oBfh6290–293Minor IssueChanged the summation index in Equation (12) to match the view and patch indices in the loss term.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.