- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposed a framework to predict a patient’s lab value changes between blood draws using bedside PPG and ECG waveforms. The proposed method uses the patient’s previously measured value and reads the change from their current and past waveforms. The authors also introduce a new multimodal waveform foundation model. On held-out patients, the method modestly improves over strong non-waveform baselines, partly because it uses the patient’s own past waveforms. The new encoder ranks first on average among ten encoders, slightly ahead of the best public model.
Critical strengths
-
Most waveform foundation-model work reads a value from a single window of seconds to minutes. This paper instead asks how waveforms can track a patient’s physiology across the gaps between blood draws, from hours in the ICU to months between ED visits. This long-horizon setting is clinically relevant and under-explored.
-
The paper is transparent about pretraining overlap and reproducibility limitations.
Critical weaknesses
-
The donor control overstates the patient-specific gain. From Table 4, the donor arm gains less over the floor than the network that sees only the current window. A donor’s past windows are probably acting as noise. Please add a donor − current column with CIs, report own − current as the primary contrast, cap donor reuse.
-
The stage-1 checkpoint lacks details. The authors mentioned Stage 1 is “a checkpoint … that our group pretrained earlier”. Section K states it “has no identifiers that join to MIMIC-III or MC-MED”, so its overlap with the evaluation patients cannot be determined. This is a reproducibility gap and a risk of label-free transductive leakage that would directly favour the own-past contrast. Please properly cite the previous work in third person so that its details and pretraining corpus can be determined.
-
The comparison with public encoders may not be fair. First of all, the public encoders were pre-trained on some of the evaluation datasets used. The pretraining exposure is uneven. Secondly, 518 of 528 ED test patients (98%) may appear in the stage-1 corpus of the proposed model. In the ED task, the four encoders that may have seen MC-MED have the four largest own − donor contrasts (+0.025 to +0.038). Every unexposed encoder is at most +0.012. Comparing against models without this access is not fair, and a Limitations note is not sufficient. Please retrain stage 1 on a split or dataset that excludes evaluation patients, or restrict the claims to the ICU task, which has no overlap.
-
The encoder’s main contribution is joint ECG–PPG pretraining, yet every comparator is a single-modality or general time-series model. Please compare against, or discuss, a few ECG–PPG multimodal foundation models.
-
The distribution of gap lengths and their units are not reported. Only the ED median is reported, but not for ICU tasks. Please add a histogram or table of gap lengths per task and split, with sample sizes per bin. This would support the “long-horizon” framing and the no-decay claim. In addition, it is unclear what the unit for delta t is and whether it's the same for ED and ICU. It matters for log(1 + delta t) since ICU gaps are hours and ED gaps are months.
Questions for authors
-
The hemoglobin-only floor term needs justification. Only the ED hemoglobin floor includes the time from arrival to the start of waveform monitoring. Why only hemoglobin? Was the term added after seeing, in validation patients, that monitoring timing correlates with hemoglobin?
-
The preprocessing pipeline for public encoders needs clarification. Were all signals first downsampled to 40Hz for PPG and 120 Hz for ECG, then later resampled to each public encoder's native required sampling rate?
2 comments
Response to Reviewer zzlr Part 2/2
This comment answers weaknesses 4 and 5 and the two questions. Table, figure and section numbers refer to the revised PDF.
W4. ECG–PPG multimodal foundation models
The reviewer asked for this comparison because the encoder was presented as a contribution. It no longer is. Native calibration is the paper's single contribution, and the encoder we pretrain is one of 9 frozen encoders it is applied to (Section 4.1). We now discuss ECG–PPG multimodal foundation models in Section 2, but we did not run more of them. CSFM, xMAE and the wearable encoders of Abbaspourazad et al. (2024) have no openly available weights. NormWear's weights are public, and we did not evaluate it. Two of the encoders we use are pretrained on both modalities. AnyPPG aligns its PPG encoder to a frozen ECG encoder on paired segments. The encoder we pretrain trains first on PPG with ECG as an auxiliary view and then on paired ECG–PPG segments. In the ICU task we also apply native calibration to ECGFounder. Section 5 lists the missing comparison as a limitation.
W5. Gap lengths and their units
Table 8 now gives, for each task and split, the number of rows, the median and the 10th to 90th percentiles of the gap, and the number of rows in each of five gap bins. Its second part counts ICU draws by their number of usable earlier draws. The unit differs between the two tasks, which are fitted separately. Between ED visits the gap is in days (test median 64 d). Within an ICU stay it is in hours (test median 10.9 h to the nearest usable earlier draw). The ED cross-visit floor, NC-lin, NC-attn and the estimator baselines all take the gap as log(1 + Δt), in the unit of their task (Section 3.3, Appendix C). Figure 7b gives own − donor by tertile of the ED gap.
Q1. Monitoring timing in the hemoglobin floor
On validation visits, the timing terms changed the hemoglobin floor's R² by +0.0002 [−0.0028, +0.0031]. Monitoring timing therefore does not predict hemoglobin beyond the case-mix floor. The hemoglobin floor went through extra checks on validation patients, and this was one of them. To check that the waveform's gain does not reflect when monitoring began, we added to this floor two offsets of the start of waveform monitoring, the arrival hour and the weekday. We kept them for hemoglobin. The floors of the other labs do not have them. Arrival hour and weekday carry no clock information, because timestamps are shifted per patient (Appendix C).
Q2. Resampling for the public encoders
Yes. Every encoder receives the same stored windows, PPG at 40 Hz and ECG lead II at 120 Hz. Each public encoder's preprocessing then resamples them to the rate or length that encoder expects, for example 125 Hz for AnyPPG and PaPaGei-S and 500 Hz for ECGFounder (Appendix E). The windows of the standard PPG tasks are likewise resampled to 40 Hz first (Appendix B).
Response to Reviewer zzlr Part 1/2
Thank you for a detailed and specific review. The reviewer's points on pretraining exposure led us to retrain the encoder on training patients only and to recompute every result that uses it. We also reorganised the paper. Native calibration is now its single contribution, and the encoder is one of 9 encoders it is applied to. Its two checkpoints are now called the stage-1 and stage-2 encoders (Ours (initial) and Ours in the submission). This comment answers weaknesses 1 to 3; the next answers weaknesses 4 and 5 and the two questions. Table, figure and section numbers refer to the revised PDF.
W1. Donor − current, own − current first, capped donors
We added donor − current with its 95% interval for every encoder to Table 3. Own − current now leads: the introduction states it in words, and Figure 2 and Table 3 report it. Own − donor for every encoder is in Figure 3 and Table 24. The reviewer was right that the donor control can overstate the patient-specific gain. Donor − current is below zero for 3 of the 8 encoders in the ED (Stage 1, AnyPPG, Chronos-Bolt-small) and for 3 of the 5 in the ICU (Stage 2, Stage 1, MOMENT-base). For those rows own − donor slightly overstates what the own past adds, and own − current is the conservative figure. With the stage-2 encoder, donor − current is −0.0004 [−0.0031, +0.0020] in the ED, which is what a valid control should show, and −0.0023 [−0.0045, −0.0004] in the ICU. Own − current is above zero on both tasks: +0.039 [+0.023, +0.056] for ED hemoglobin and +0.010 [+0.0050, +0.015] for ICU potassium. Own − donor was the contrast fixed in advance, so it keeps the Bonferroni-adjusted interval. Table 25 gives all contrasts for the stage-1 and stage-2 encoders. We also capped donor reuse, so that each row donates at most once per draw (Appendix H.6). On validation patients, capped minus standard donor is −0.0006 [−0.0029, +0.0016] for the stage-2 encoder with NC-attn, and its interval includes zero for both checkpoints and both estimators (Table 29).
W2. The stage-1 checkpoint
The reviewer was right about the risk. Mapped to patients, 37.6% of the windows of the submitted stage-1 training set came from validation or test patients of the evaluation tasks. We retrained both stages from scratch on training patients only: a pretraining window is kept only if its patient is identified, belongs to an evaluation cohort of its data source, and is a training patient in every such cohort. The retrained stage-1 training set contains 0 such windows. Stage 1 is now trained in this work, without labels, on 3M windows, 89.2% of them from MC-MED and the rest from the MIMIC-III and VitalDB parts of PulseDB. Its checkpoint is the stage-1 encoder. Appendix D.1 and Table 11 give how each window was linked to its patient, the windows kept and dropped per source, and the checks. Appendix D, Appendix E and Appendix K give the corpus, objective, hyperparameters and compute of both stages.
W3. Fairness against the public encoders
No validation or test patient of the lab and haemodynamics tasks is now in the pretraining data, and native calibration holds. Own − donor is +0.040 [+0.023, +0.058] in the ED and +0.012 [+0.0070, +0.018] in the ICU. In the ED the stage-2 encoder has the largest own − donor contrast, above AnyPPG's by +0.015 [+0.0045, +0.026]. The three encoders with MC-MED in their pretraining, the stage-1 and stage-2 encoders and AnyPPG, still have the three largest ED contrasts. Excluding the evaluation patients rules out leakage, but not an advantage of pretraining on the task's data source. We state this in Section 4.4 and Section 5. In the ICU, the contrasts of the stage-1 and stage-2 encoders do not differ, and the contrast is above zero for all 5 encoders used on ECG, including ECGFounder. The public encoders were not retrained. AnyPPG's pretraining lists MC-MED and PulseDB, and PaPaGei's lists VitalDB and MIMIC-III, so any overlap with our evaluation patients would favour them (Appendix J). On the standard PPG tasks the ranking changed. Recomputed among the eight encoders that remain, the submitted mean ranks were 2.54 [1.88, 3.25] for ours and 2.71 [2.08, 3.42] for AnyPPG. After retraining, AnyPPG ranks first (2.62 [1.96, 3.33]) and the stage-2 encoder second (3.08 [2.33, 3.92]). AnyPPG's scores did not change; only the two retrained rows did. The two are statistically on par. We withdraw the claim of ranking first there, and the encoder is no longer presented as a contribution. Three of these tasks lack subject identifiers, so the exclusion cannot be verified for them; Table 22 reports the other 21.