- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3(+1)
Read review
Paper summary
The paper proposes BEATFORMER, a 73.8M-parameter foundation model for synchronized ECG and PPG that represents a window as a sequence of paired beats anchored on ECG R peaks, each beat resampled to 128 samples and encoded as eight phase tokens per modality. A delay-aware fusion module estimates a posterior over the ECG-to-PPG lag for each beat and uses its confidence to bias bidirectional cross-modal attention, followed by factorized within-beat and across-beat attention with physical-time biases. Pretraining on about 84.8M beats, almost all from ICU and operating-room recordings, combines masked reconstruction with PAT, PPG morphology and HRV regression. Evaluation covers 15 frozen-backbone classification and regression tasks on nine datasets against six foundation models (five of them PPG-only), PAT on four datasets, and fine-tuned reconstruction and denoising on VTaC. BEATFORMER is best on 9 of 15 tasks, with a PAT MAE of 11.48 ms on Li ECG–PPG.
Critical strengths
- The main strength, to me, is a design that follows the physiology: beats anchored on R peaks with phase tokens, an explicit ECG-PPG lag, and positional biases in physical time with RR information, instead of arbitrary fixed patches.
- The evaluation is broad and honest: 15 tasks on nine datasets plus reconstruction and denoising, subject-disjoint splits, PAT on four datasets including the ones where it loses, an explicit statement that recovery assumes clean beat segmentation, and code released.
Critical weaknesses
- Novelty for ICLR is incremental, and the scope stays cardiac... (i) In my opinion the methodological novelty is incremental for ICLR. Each block has a known counterpart: beat-level tokenization anchored on R peaks has already been done for ECG, the within-beat and across-beat attention is axial (factorized) attention, the delay posterior is a soft cross-correlation over eight lags turned into an attention bias, and the pretraining is multi-task regression of handcrafted physiological targets (PAT, PPG morphology, HRV). The combination is well thought out, but to me it is careful engineering for one pair of signals. Also, the claim of being the first to put PAT supervision into pretraining needs softening, since App A itself describes a model with a PAT-anchored delay aligner. (ii) The scope is still one physiological system: synchronized ECG and PPG, cardiac only, and the model cannot run without both signals (no unimodal variant, App. h), while most wearables record PPG continuously and ECG only on demand, if at all. If the general idea is event-anchored tokens with an uncertain lag between modalities, it would need to be shown on other paired signals with physiological delays, e.g. ECG-PCG (already in SensSmartTech, which the paper uses), ECG-arterial pressure, or beyond cardiology. As it stands, it may fit better more applied venues.
- The delay-aware part is not isolated (i) The module in the title, delay-aware fusion, is never ablated: App. G.6 removes six components, not this one, and evaluates only HR and BP, not PAT or recovery, with one run per variant. I also wonder what "beat-specific delay" can resolve when the lag posterior moves in steps of 1/8 beat, around 100 ms at 75 bpm, while the beat-to-beat changes in PAT that motivate the paper are much smaller. This needs to be done thoroughly imo. (ii) Five of the six baselines see only PPG, while BEATFORMER sees ECG and PPG. For HR, PAT and VT false-alarm detection the ECG is decisive, and the RR intervals are given to the model as input features (the beat-context descriptor), so the HR results are close to by construction. Against the only paired baseline, CSFM, it loses on 3 of 15 tasks, including VT false alarms. A late-fusion baseline, e.g. an ECGFounder or ECG-FM embedding concatenated with a PPG foundation model embedding, is needed to show that the delay-aware fusion adds something beyond having both signals. In the end, this is your selling point. (iii) The PAT evidence is weaker than it reads. The reference is computed by an algorithm from the same two signals (R peak to PPG foot), close to the pretraining target, so the probe mainly shows that the embedding kept what it was trained to predict, and the PPG-only models cannot see the R peak at all. On UQ and ErgoBP the PAT errors are around 90 ms with window-level SDs above 120 ms, and a PPG-only model is better on ErgoBP. Could device-dependent ECG–PPG offsets in monitor data explain this? Am I missing something? (iv) The statistics do not support most of the margins. There is a single pretraining run, and the authors' own sensitivity analysis moves SBP MAE between 10.42 and 11.02 mmHg, more than the gap to the best PPG-only baseline (10.75 vs 10.89). The fold stds in Table 3 are also large compared with most of the gaps. (v) The recovery results lean on the setup. Beats are segmented on the clean signals before masking or corruption (stated in App. E.2), the noise is defined in per-beat coordinates, and BEATFORMER is fine-tuned with 11 to 22 times more optimizer updates than CSFM, VCT-M2AE and MOMENT at the same learning rates; CSFM ends with negative SNR improvements, which looks more like a broken adaptation than a fair comparison. In denoising, the from-scratch Small Transformer on the same beat packing is within about 2 dB and wins two conditions, so imo most of the gain over the other baselines comes from the beat packing with clean segmentation, not from pretraining.
Questions for authors
My main concerns are the two weaknesses above, so I copy them here. I'd be open to raising the score if the delay-aware fusion is ablated, the comparisons are made like for like and the scope goes beyond ECG and PPG, and the below points are defended
- Novelty for ICLR is incremental, and the scope stays cardiac... (i) In my opinion the methodological novelty is incremental for ICLR. Each block has a known counterpart: beat-level tokenization anchored on R peaks has already been done for ECG, the within-beat and across-beat attention is axial (factorized) attention, the delay posterior is a soft cross-correlation over eight lags turned into an attention bias, and the pretraining is multi-task regression of handcrafted physiological targets (PAT, PPG morphology, HRV). The combination is well thought out, but to me it is careful engineering for one pair of signals. Also, the claim of being the first to put PAT supervision into pretraining needs softening, since App A itself describes a model with a PAT-anchored delay aligner. (ii) The scope is still one physiological system: synchronized ECG and PPG, cardiac only, and the model cannot run without both signals (no unimodal variant, App. h), while most wearables record PPG continuously and ECG only on demand, if at all. If the general idea is event-anchored tokens with an uncertain lag between modalities, it would need to be shown on other paired signals with physiological delays, e.g. ECG-PCG (already in SensSmartTech, which the paper uses), ECG-arterial pressure, or beyond cardiology. As it stands, it may fit better more applied venues.
- The delay-aware part is not isolated (i) The module in the title, delay-aware fusion, is never ablated: App. G.6 removes six components, not this one, and evaluates only HR and BP, not PAT or recovery, with one run per variant. I also wonder what "beat-specific delay" can resolve when the lag posterior moves in steps of 1/8 beat, around 100 ms at 75 bpm, while the beat-to-beat changes in PAT that motivate the paper are much smaller. This needs to be done thoroughly imo. (ii) Five of the six baselines see only PPG, while BEATFORMER sees ECG and PPG. For HR, PAT and VT false-alarm detection the ECG is decisive, and the RR intervals are given to the model as input features (the beat-context descriptor), so the HR results are close to by construction. Against the only paired baseline, CSFM, it loses on 3 of 15 tasks, including VT false alarms. A late-fusion baseline, e.g. an ECGFounder or ECG-FM embedding concatenated with a PPG foundation model embedding, is needed to show that the delay-aware fusion adds something beyond having both signals. In the end, this is your selling point. (iii) The PAT evidence is weaker than it reads. The reference is computed by an algorithm from the same two signals (R peak to PPG foot), close to the pretraining target, so the probe mainly shows that the embedding kept what it was trained to predict, and the PPG-only models cannot see the R peak at all. On UQ and ErgoBP the PAT errors are around 90 ms with window-level SDs above 120 ms, and a PPG-only model is better on ErgoBP. Could device-dependent ECG–PPG offsets in monitor data explain this? Am I missing something? (iv) The statistics do not support most of the margins. There is a single pretraining run, and the authors' own sensitivity analysis moves SBP MAE between 10.42 and 11.02 mmHg, more than the gap to the best PPG-only baseline (10.75 vs 10.89). The fold stds in Table 3 are also large compared with most of the gaps. (v) The recovery results lean on the setup. Beats are segmented on the clean signals before masking or corruption (stated in App. E.2), the noise is defined in per-beat coordinates, and BEATFORMER is fine-tuned with 11 to 22 times more optimizer updates than CSFM, VCT-M2AE and MOMENT at the same learning rates; CSFM ends with negative SNR improvements, which looks more like a broken adaptation than a fair comparison. In denoising, the from-scratch Small Transformer on the same beat packing is within about 2 dB and wins two conditions, so imo most of the gain over the other baselines comes from the beat packing with clean segmentation, not from pretraining.
12 comments
Response to Reviewer dUR8 (2/6): Delay-aware fusion and lag resolution
Ablation of delay-aware fusion
We thank the reviewer for pointing out that the original ablations did not directly examine the fusion module. We have added comparisons covering HR, PAT, blood pressure, and classification tasks. Specifically, we compare the full model with two variants: one removes the delay bias while retaining cross-modal attention, and the other retains the delay bias but fixes the posterior confidence to . All three variants were trained for five epochs using the same training data and optimization settings.
\text{Dataset / task / metric}&\begin{array}{c} \text{Without cross-modal} \\\\ \text{attention} \end{array}&\begin{array}{c} \text{Without delay bias} \\\\ \text{(} B_k=0 \text{)} \end{array}&\begin{array}{c} \text{Fixed confidence} \\\\ \text{(} \gamma_k=1 \text{)} \end{array}&\text{Full model} \\\\ \hline \text{Simultaneous / HR MAE (bpm)} \downarrow&15.305\pm7.148&14.634\pm5.901&13.553\pm4.492&\mathbf{11.354\pm4.511} \\\\ \text{Li ECG--PPG / PAT MAE (ms)} \downarrow&13.568\pm2.381&13.377\pm1.766&13.621\pm1.728&\mathbf{12.227\pm2.602} \\\\ \text{PPG-DaLiA / Activity accuracy (\\%)} \uparrow&43.84\pm10.92&43.72\pm9.80&44.54\pm8.12&\mathbf{47.57\pm8.40} \\\\ \text{Simultaneous / Load accuracy (\\%)} \uparrow&54.23\pm10.64&54.44\pm11.73&56.08\pm10.32&\mathbf{57.64\pm11.61} \\\\ \text{Li ECG--PPG / SBP MAE (mmHg)} \downarrow&10.730\pm1.937&10.597\pm1.805&10.759\pm1.242&\mathbf{10.411\pm1.386} \\\\ \text{Li ECG--PPG / DBP MAE (mmHg)} \downarrow&7.657\pm0.918&7.504\pm0.597&7.397\pm0.673&\mathbf{7.367\pm0.576} \\\\ \end{array}$$ Removing the delay bias or fixing the confidence to $\gamma_k=1$ degrades mean performance across all six tasks. These comparisons support the downstream utility of the delay bias and its adaptive modulation by posterior confidence. We alos thank the reviewer for raising the concern about temporal resolution. We conducted an ablation comparing 4, 8, and 16 lag candidates. $$\begin{array}{lrrr} \text{Dataset / task / metric}&\text{4 candidates}&\text{8 candidates}&\text{16 candidates} \\\\ \hline \text{PTT PPG / HR MAE (bpm)} \downarrow&5.608&\mathbf{5.556}&5.570 \\\\ \text{Li ECG--PPG / PAT MAE (ms)} \downarrow&13.041&\mathbf{12.002}&12.233 \\\\ \text{Li ECG--PPG / SBP MAE (mmHg)} \downarrow&11.209&\mathbf{10.687}&10.960 \\\\ \text{Li ECG--PPG / DBP MAE (mmHg)} \downarrow&7.613&\mathbf{7.305}&7.460 \\\\ \end{array}$$ Increasing the number of lag candidates from 4 to 8 improves mean performance on all four tasks. Increasing it further to 16 yields similar performance without additional improvement. We therefore adopt 8 lag candidates for their lower computational cost relative to 16 lag candidates. However, we agree that this does not guarantee the accuracy of the module's internal PAT-related delay estimates. The preceding fusion ablations support the downstream utility of this module. These findings are consistent with our motivation to introduce PAT-related physiological timing information into paired ECG--PPG representation learning through delay guidance. This may suggest that even coarse-grained timing information can benefit downstream tasks.Response to Reviewer dUR8 (3/6): ECG-PPG late-fusion comparisons
Comparison with ECG--PPG late fusion We compared BEATFORMER with ECGFounder and AnyPPG as unimodal baselines, and with ECGFounder+AnyPPG and CLEF-Medium+AnyPPG using concatenation-based late fusion across all 15 tasks.
\text{Dataset / task / metric}&\text{BEATFORMER}&\text{ECGFounder}&\text{AnyPPG}&\text{Founder+AnyPPG}&\text{CLEF+AnyPPG} \\\\ \hline \text{PTT PPG / HR MAE (bpm)} \downarrow&\mathbf{4.025\pm3.177}&4.335\pm3.411&5.623\pm4.652&4.337\pm3.517&5.030\pm5.247 \\\\ \text{Simultaneous / HR MAE (bpm)} \downarrow&\mathbf{9.861\pm4.114}&16.122\pm8.679&15.711\pm5.848&13.461\pm8.920&10.197\pm5.113 \\\\ \text{Li ECG--PPG / PAT MAE (ms)} \downarrow&\mathbf{11.475\pm2.185}&24.161\pm3.888&16.418\pm2.503&16.709\pm1.944&16.957\pm2.240 \\\\ \text{BUT PPG / Quality AUROC} \uparrow&\mathbf{0.839\pm0.033}&0.622\pm0.070&0.801\pm0.040&0.751\pm0.047&0.751\pm0.030 \\\\ \text{WESAD / Stress AUROC} \uparrow&0.919\pm0.027&0.887\pm0.066&\mathbf{0.922\pm0.026}&0.921\pm0.035&0.899\pm0.023 \\\\ \text{PTT PPG / Activity accuracy} \uparrow&\mathbf{0.680\pm0.145}&0.591\pm0.128&0.550\pm0.129&0.660\pm0.140&0.600\pm0.117 \\\\ \text{PPG-DaLiA / Activity accuracy} \uparrow&\mathbf{0.517\pm0.108}&0.463\pm0.123&0.457\pm0.067&0.500\pm0.133&0.467\pm0.123 \\\\ \text{ScientISST MOVE / Activity accuracy} \uparrow&0.656\pm0.245&0.625\pm0.282&0.674\pm0.207&0.646\pm0.251&\mathbf{0.686\pm0.247} \\\\ \text{Simultaneous / Load accuracy} \uparrow&\mathbf{0.569\pm0.098}&0.500\pm0.094&0.452\pm0.082&\mathbf{0.569\pm0.087}&0.553\pm0.073 \\\\ \text{SensSmartTech / Before--after AUROC} \uparrow&0.871\pm0.049&\mathbf{0.913\pm0.050}&0.787\pm0.085&0.890\pm0.065&0.863\pm0.061 \\\\ \text{VTaC / False-alarm detection AUROC} \uparrow&0.8435\pm0.0040&0.9262\pm0.0040&0.780\pm0.043&\mathbf{0.9295\pm0.0026}&0.8358\pm0.0084 \\\\ \text{PPG-DaLiA / Age MAE (years)} \downarrow&\mathbf{6.449\pm3.096}&8.376\pm2.970&6.810\pm2.177&9.289\pm1.570&9.323\pm5.027 \\\\ \text{PPG-DaLiA / BMI MAE (kg/m} ^2 \text{)} \downarrow&1.459\pm0.523&1.497\pm0.513&\mathbf{1.309\pm0.522}&1.596\pm0.418&1.625\pm0.407 \\\\ \text{Li ECG--PPG / SBP MAE (mmHg)} \downarrow&\mathbf{10.752\pm1.474}&10.822\pm1.606&10.975\pm1.528&11.790\pm1.711&11.671\pm1.640 \\\\ \text{Li ECG--PPG / DBP MAE (mmHg)} \downarrow&\mathbf{7.114\pm0.505}&7.838\pm0.854&7.206\pm0.896&8.001\pm1.075&7.518\pm0.828 \\\\ \end{array}$$ Consistent with the original main-table results, BEATFORMER maintains its advantages on HR, PAT, and BP prediction even when compared with baselines that also use both ECG and PPG through late fusion. On VTaC, ECGFounder+AnyPPG performs better, while ECGFounder alone already achieves an AUROC of 0.9262. ECGFounder's large-scale supervised pretraining on ECG diagnostic labels, including rhythm-related information, may contribute to its strong performance on this task. Overall, these results support the effectiveness of BEATFORMER's pretrained representations relative to simple ECG--PPG late fusion.
Response to Reviewer dUR8 (1/6): Methodological contribution and ECG-PCG transfer
We thank the reviewer dUR8 for recognizing the physiological organization of the model. We address the concerns and questions below, including methodological novelty, ECG--PCG adaptation, delay-aware fusion, dual-input comparisons, PAT reference quality, and the recovery evaluation.
Methodological contribution and PAT supervision
We thank the reviewer for recognizing that the combination is well thought out. Our research is indeed motivated by practical problems. In the early stages, we also spent a lot of time collecting data ourselves in real-world settings to understand the challenges these signals present in practice and inform our model design.
We acknowledge that several components are closely related to existing methods. Our emphasis is on carefully designing and empirically evaluating their integration, guided by established physiological understanding, to address paired ECG--PPG correspondence, temporal offsets, and cardiac dynamics. Rather than claiming that every component is individually new, our primary aim is to address these problems effectively.
We also thank the reviewer for pointing out that our original priority claim was insufficiently precise. We narrow this claim as follows:
Our intended contribution is to model ECG and PPG as physiologically corresponding paired beats and use beat-specific PAT supervision to encourage these paired representations to preserve the timing relationship between the two signals, rather than broadly claim the first use of PAT as a pretraining target.
Extension to ECG--PCG
We agree with the reviewer that our model was designed to align physiologically corresponding events in ECG and PPG, using ECG R peaks as shared anchors for paired-beat segmentation. When only PPG is available, the absence of these ECG anchors makes beat segmentation more challenging, and our current architecture also performs less well when adapted to PPG-only inputs. However, the paired-event formulation can be explored with other physiologically related signal pairs. We thank the reviewer for suggesting ECG--PCG as such an extension. Following this suggestion, we evaluated the transfer of BEATFORMER, pretrained on ECG--PPG, to ECG--PCG under both frozen and fine-tuned settings.
\text{Model}&\begin{array}{c} \text{SensSmartTech} \\\\ \text{AUROC} \uparrow \end{array}&\begin{array}{c} \text{SensSmartTech} \\\\ \text{HR MAE (bpm)} \downarrow \end{array}&\begin{array}{c} \text{CinC 2016} \\\\ \text{AUROC} \uparrow \end{array} \\\\ \hline \text{CNN, from scratch}&0.8355\pm0.0682&9.7888\pm1.9244&0.7456\pm0.0080 \\\\ \text{Transformer, from scratch}&0.7764\pm0.0750&12.6297\pm2.7008&0.6905\pm0.0259 \\\\ \text{ECGFounder+PANNs, frozen}&0.9197\pm0.0677&4.3330\pm1.4896&\mathbf{0.8544\pm0.0007} \\\\ \text{BEATFORMER, frozen}&0.8733\pm0.0241&9.2565\pm0.7168&0.5768\pm0.0073 \\\\ \text{ECGFounder+PANNs, fine-tuned}&0.9362\pm0.0430&3.8191\pm0.6319&0.8373\pm0.0188 \\\\ \text{BEATFORMER, fine-tuned}&\mathbf{0.9586\pm0.0399}&\mathbf{3.2740\pm0.5481}&0.7745\pm0.0260 \\\\ \end{array}$$ Despite being pretrained on ECG--PPG rather than ECG--PCG, BEATFORMER adapts to the new signal pair through fine-tuning, achieving the best classification and HR results among the compared learned models on SensSmartTech. On CinC 2016, fine-tuning improves its AUROC from 0.5768 to 0.7745, exceeding the CNN and Transformer trained from scratch, although it remains below ECGFounder+PANNs. These results support the applicability of our paired-signal modeling framework beyond ECG--PPG, with the CinC 2016 experiment providing exploratory record-level evidence.