- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The authors have developed PPG-LM, a photoplethysmography-language model which links clinical descriptions of segments, events, or visits to the physiological waveforms recorded during those periods. PPG-LM uses an automated captioning pipeline based on signal-derived features and structured electronic health record data. The training process for PPG-LM is divided into two phases. The first phase uses contrastive alignment to link waveform segments to their corresponding descriptions, together with waveform-conditioned captioning to generate segment-level descriptions. In the second phase, PPG-LM extends alignment to event- and visit-level clinical descriptions. In addition to these two phases, PPG-LM uses time-aware aggregation to combine individual waveform segment features into representations of events or visits. Finally, PPG-LM matches temporal statements describing visits with the corresponding visit representations. PPG-LM is pretrained using approximately 73,700 hours of PPG recordings from MC-MED. PPG-LM is evaluated on MC-MED, MIMIC-III, and VitalDB. Evaluation of PPG-LM indicates that it outperforms several existing models in terms of language-based recognition of physiology, cross-modal retrieval, caption generation, and frozen-representation clinical prediction.
Critical strengths
A meaningful formulation of multi-level physiological–clinical alignment.
The authors address a key challenge in physiological foundation models: how to relate characteristics of individual waveforms to broader clinical data. Their approach of organizing clinical supervision at three levels (segment, event, visit) is well reasoned; also, the combination of automated caption generation with temporal aggregation provides a logical framework for using diverse sources of clinical data. As such, this represents a valuable extension of current approaches to signal-language representation learning.
Substantial experimental coverage and encouraging transfer performance.
The study evaluated the performance of the model on three clinical datasets, which have different data collection environments and involve recognition, retrieval, captioning, and clinical prediction tasks. It was promising to observe performance improvements on both MIMIC-III and VitalDB, as the model had been pretrained on MC-MED alone. Furthermore, the evaluation involved the use of independent ECG reference signals to assess heart rate, frozen-representation probes, and component analyses; this helped to support the empirical findings.
Critical weaknesses
The methodological contribution made by multi-level alignment has yet to be adequately demonstrated.
The proposed architecture combines multiple existing methods, such as contrastive learning, CoCa-style captioning, attention-based temporal aggregation, and late-interaction matching. While the integration of these methods across clinical temporal levels provides value, the experimental design does not clearly demonstrate how much each method contributes to the overall performance. Specifically, the Stage 1 vs. Stage 2 experiment alters three factors (supervision, training duration, and temporal aggregation architecture) at once. Additionally, the component ablation studies in Table 8 utilize a CLIP-based configuration instead of the primary CoCa model. The lack of direct comparison with CAP, which addresses PPG-EHR clinical alignment, further restricts the ability to evaluate the improvements over similar prior work. These limitations are significant as multi-level clinical alignment represents the core methodological contribution of the manuscript.
The evaluation does not fully demonstrate the generalizability of the learned clinical language representations beyond structured descriptions.
A large amount of the training supervision is created through automated descriptions that come from both physiological measurements and structured clinical data. Several evaluation tasks involve attributes and annotation methods closely related to those used to create the pretraining data. Therefore, the zero-shot results reported here primarily demonstrate how well the model can apply its knowledge of the relationship between physiology and language without task-specific model fine-tuning, although some visit-level tasks use threshold calibration. However, there is less evidence that the model can understand clinical information expressed independently. Because of the use of automatically generated reference descriptions as well as the sensitivity shown to different prompt formulations, there remain unanswered questions about whether the model will be able to generalize beyond the structured description format.
Some qualifications to the baseline comparisons should be stated. The Qwen3-4B supervised fine-tuning baseline uses significantly fewer training windows than PPG-LM, and general-purpose LLMs use downsampled waveforms (represented numerically as text), while PPG-LM uses a dedicated waveform encoder. Although these qualifications do not invalidate the reported findings, they do restrict our ability to make conclusions regarding the comparative performance of the proposed architecture.
Questions for authors
Can the authors perform controlled studies where they hold constant the primary CoCa architecture, the training budget, and the temporal aggregation method and systematically remove event-level and visit-level clinical supervision? This will allow us to assess whether improved performance is due to multi-level alignment or simply additional training and/or increased aggregation capacity.
Can the authors provide a quantitative comparison with CAP using a common evaluation protocol, or explain why they cannot do so due to methodological constraints? Also, the use of a more closely matched signal-language baseline would allow us to see if the disparity in pretraining exposure has an impact.
When PPG-LM is tested against independently written clinical descriptions or clinical attributes that have not been included in the pretraining captions, can we determine whether the model has learned more general clinical concepts beyond those represented in the training data? This testing will allow us to assess whether PPG-LM is capable of understanding clinical semantics at a higher level than just what it was trained on.
Since the waveform encoder standardizes each window of data, and perfusion strength is based on an AC/DC amplitude measurement, could the authors please provide more explanation of how they preserve the necessary information (or indirectly infer it) when they remove the absolute offset and scale information from the data during normalization?
10 comments
Response [2/8]
- Disentangling Stage 1 vs. Stage 2 Downstream Representations: Probing frozen features without text prompts across 8 MC-MED, 3 MIMIC-III, and 4 VitalDB classification tasks on official splits (Table R12) shows that extra Stage-2 duration and capacity with segment supervision alone yield no gain over Stage 1 on dynamic events (0.613 vs. 0.640 Stage-1 window mean) or external transfer (0.731 vs. 0.733 on MIMIC-III). In contrast, multi-level supervision lifts Event Dynamic Probe AUROC to 0.712 and improves cross-center transfer on MIMIC-III (0.731 → 0.742) and VitalDB (0.702 → 0.708), while under matched 4-window pooling, MC-MED static probes saturate in Stage 1 (0.785 vs. 0.784).
Table R12: Non-Text Frozen Representation & Cross-Center Transfer Probes (AUROC ↑; Bold = Notable Degradation)
| Model / Variant | Event Dynamic Probe ↑ (MC-MED) | MC-MED Static Probe ↑ | MIMIC-III Transfer ↑ | VitalDB Transfer ↑ |
|---|---|---|---|---|
| PPG-LM | 0.712 | 0.785 | 0.742 | 0.708 |
| w/o higher-level losses (segment only) | 0.613 | 0.784 | 0.731 | 0.702 |
| w/o event loss | 0.615 | 0.784 | 0.735 | 0.703 |
| w/o visit & temporal loss | 0.714 | 0.784 | 0.740 | 0.706 |
| w/o Stage-2 (Stage-1 init) | 0.640 | 0.784 | 0.733 | 0.701 |
Response [3/8]
W1-2: The lack of direct comparison with CAP, which addresses PPG-EHR clinical alignment, further restricts the ability to evaluate the improvements over similar prior work. These limitations are significant as multi-level clinical alignment represents the core methodological contribution of the manuscript.
Q2: Can the authors provide a quantitative comparison with CAP using a common evaluation protocol, or explain why they cannot do so due to methodological constraints? Also, the use of a more closely matched signal-language baseline would allow us to see if the disparity in pretraining exposure has an impact.
Thank you for raising this important comparison. Because CAP's pretrained weights and processed data are not publicly released, we retrained CAP on MC-MED following its official architecture and training objective (pairing 10 consecutive 30s windows with a single visit summary; 5,000-step pre-divergence checkpoint). CAP's downstream protocol probes only its single-scale PPG encoder without a generative decoder or event/visit temporal aggregators, precluding caption generation and multi-scale event or trajectory evaluation. Under a common evaluation protocol using matched 4-window linear probing across all 32 clinical prediction tasks (17 MC-MED, 6 MIMIC-III, and 9 VitalDB) on patient-disjoint splits (Table R13, matching Tables 7 and 22) and 50-shot adaptation on MC-MED (Table R14), PPG-LM achieves state-of-the-art performance, outperforming CAP across all six dataset AUROC and R² group means and all 32 of 32 clinical prediction tasks (with Stage 1 alone leading CAP on 29 of 32, and Stage 2 improving over Stage 1 on 30 of 32)—as well as across all 50-shot Macro-F1 tasks—while uniquely supporting multi-level clinical language interfaces (for the signal–language pretraining-exposure comparison, please see our response to W3 immediately below).
Table R13: Full Per-Task Clinical Prediction Comparison Between CAP and PPG-LM on Patient-Disjoint Splits (4-Window Protocol; Bold = Best)
| Dataset / Task | CAP | PPG-LM (Stage 1) | PPG-LM (Stage 2) |
|---|---|---|---|
| MC-MED (ED) — AUROC ↑ | |||
| Sex | 0.769 | 0.813 | 0.856 |
| ED admission | 0.729 | 0.765 | 0.787 |
| Atrial fibrillation | 0.825 | 0.857 | 0.870 |
| Heart failure | 0.781 | 0.836 | 0.860 |
| Cardiac device | 0.771 | 0.836 | 0.857 |
| Chronic kidney disease | 0.737 | 0.774 | 0.792 |
| Diabetes | 0.702 | 0.728 | 0.740 |
| Anemia | 0.669 | 0.686 | 0.709 |
| Group Mean (n = 8) | 0.748 | 0.787 | 0.809 |
| MC-MED (ED) — R² ↑ | |||
| Age | 0.588 | 0.691 | 0.782 |
| eGFR | 0.276 | 0.356 | 0.404 |
| Hemoglobin | 0.173 | 0.238 | 0.274 |
| Albumin | 0.197 | 0.239 | 0.286 |
| BUN | 0.129 | 0.178 | 0.213 |
| Potassium | 0.038 | 0.067 | 0.086 |
| Calcium | 0.035 | 0.051 | 0.076 |
| Sodium | 0.050 | 0.068 | 0.091 |
| Glucose | 0.079 | 0.071 | 0.083 |
| Group Mean (n = 9) | 0.174 | 0.218 | 0.255 |
| MIMIC-III (ICU) — AUROC ↑ | |||
| Sex | 0.706 | 0.730 | 0.733 |
| In-hospital mortality | 0.743 | 0.786 | 0.793 |
| Circulatory diagnosis | 0.686 | 0.720 | 0.726 |
| Group Mean (n = 3) | 0.712 | 0.745 | 0.751 |
| MIMIC-III (ICU) — R² ↑ | |||
| Age | 0.394 | 0.502 | 0.507 |
| BUN | 0.083 | 0.081 | 0.086 |
| Hemoglobin | 0.040 | 0.092 | 0.095 |
| Group Mean (n = 3) | 0.172 | 0.225 | 0.229 |
| VitalDB (OR) — AUROC ↑ | |||
| Sex | 0.635 | 0.684 | 0.691 |
| ASA ≥ 3 | 0.690 | 0.699 | 0.704 |
| Hypertension | 0.630 | 0.656 | 0.655 |
| ICU admission | 0.701 | 0.733 | 0.751 |
| Group Mean (n = 4) | 0.664 | 0.693 | 0.700 |
| VitalDB (OR) — R² ↑ | |||
| Age | 0.365 | 0.418 | 0.435 |
| BMI | 0.042 | 0.041 | 0.043 |
| Albumin | 0.095 | 0.134 | 0.138 |
| EtCO₂ (window) | 0.123 | 0.153 | 0.152 |
| BIS (window) | 0.380 | 0.407 | 0.408 |
| Group Mean (n = 5) | 0.201 | 0.231 | 0.235 |
Table R14: Few-Shot Adaptation Comparison on MC-MED (k = 50 Support Examples per Class, Macro-F1 ↑; Bold = Best)
| Model | Rhythm ↑ | HR Class ↑ | Sex ↑ |
|---|---|---|---|
| CAP | 0.507 | 0.873 | 0.602 |
| PPG-LM | 0.597 | 0.921 | 0.710 |
Response [1/8]
Thank you for highlighting this experimental confound. To strictly disentangle clinical supervision, training duration, and temporal aggregation on our primary CoCa model (rather than CLIP), we ran one-factor-at-a-time ablations on held-out MC-MED patients from the same Stage-1 CoCa checkpoint and fixed Stage-2 training budget—showing that extra duration and aggregator capacity alone yield no cross-scale gains without both multi-level supervision and physiological time modeling.
Table R10: Primary CoCa Supervision Ablations (Fixed Architecture, Aggregators & Budget, MC-MED Test; Bold = Notable Degradation)
Table R11: Primary CoCa Temporal Aggregation Ablations (Fixed 3-Level Supervision & Budget, MC-MED Test; Bold = Notable Degradation)