- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper systematically studies the effects of pretraining objectives, model architectures, and data scaling on ECG foundation models. The authors compare five self-supervised objectives across multiple architectures and investigate scaling with up to 11M samples. Experiments show that S4 and CPC generally achieve the strongest downstream performance, while CPC and JEPA exhibit the clearest pretraining loss scaling.
Critical strengths
The paper provides a systematic comparison of pretraining objectives and model architectures under controlled settings, addressing an important gap in existing ECG foundation model research. It provides useful empirical insights for ECG foundation model development.
Critical weaknesses
The abstract suggests that scaling data up to 11M samples leads to improved downstream performance, while Sec. 4 reports consistent improvements mainly on PTB-XL and mixed results on other datasets. The additional gains from scaling from 753K to 11M appear limited or inconsistent across tasks, making the benefits of further scaling less clear.
Questions for authors
Could the authors further clarify the downstream benefits of scaling from 753K to 11M samples, given the mixed improvements across tasks?
3 comments
Thank you for the detailed response. The proposed revisions would address my main concern by more clearly distinguishing pre-training loss scaling from downstream performance improvements. I will therefore maintain my current score.
We thank the reviewer for the positive response. We have uploaded the revised manuscript, in which all changes are highlighted in blue.
-
Abstract. We revised the scaling statement to make explicit that downstream gains under full finetuning diminish at large scale.
-
Section 4 ("Scaling with dataset size"). We added that downstream gains from 753K to 11M samples are modest and task-dependent, as expected for a power law with a residual error floor, where each additional order of magnitude of data yields smaller absolute improvements.
-
Appendix. The 753K vs. 11M comparison tables for all five objectives are included in the Tables 40-44.
We thank the reviewer for this helpful comments and for the positive assessment of our work. We address W1 and Q1 jointly, as both concern the downstream benefit of scaling from 753K to 11M samples.
1. W1 + Q1 (Downstream benefit of scaling from 753K to 11M)
To make this explicit, we report finetuning performance (macro-AUROC) for all five objectives at 753K and 11M samples, together with the gain, using identical training configurations (4-layer S4, 10 epochs).
CPC
JEPA
DinoSR
HuBERT++
data2vec
We agree that the gains from 753K to 11M under full finetuning are modest. We would, however, like to offer a complementary perspective. First, this is the behavior expected from a power law with a residual error floor. In this regime, each additional order of magnitude of data yields smaller absolute improvements, and the purpose of a scaling analysis is precisely to quantify these diminishing returns rather than to assume linear gains. Second, the clearest evidence for scaling lies in the pretraining loss, which decreases consistently with dataset size for CPC and JEPA, and in its strong rank correlation with downstream error for CPC, JEPA and DinoSR on all three investigated datasets. Lower pretraining loss therefore translates into better downstream performance, even though the absolute gains under full finetuning become small at large scale.
We will revise the abstract and Section 4 to state more precisely that scaling yields consistent improvements in pretraining loss and its correlation with downstream error, with diminishing downstream gains under full finetuning between 753K and 11M samples. The tables above will be added to the appendix.