- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper asks how much of the apparent unimodal–multimodal gap in ECG pretraining survives a controlled comparison. Existing methods are retrained under a common source-pool and preprocessing framework, then evaluated with a shared frozen linear-probe protocol on PTB-XL and SPH. A signal-only JEPA encoder is also compared with its report-anchored counterpart at different downstream label budgets.
The reported pattern is that latent prediction approaches the leading multimodal baseline with full labels, whereas reports provide clearer benefits at reduced label budgets.
Critical strengths
I recommend weak acceptance: the controlled empirical comparison is a valuable contribution. Two considerations are particularly important to this assessment.
(i) The paper tackles an important empirical question and appropriately treats controlled measurement as its central contribution. Re-training baselines, standardizing the probe protocol, and using an explicit equivalence test make the comparison more informative than juxtaposing published headline numbers.
(ii) The distinction between full-label performance and label efficiency is useful. The paired JEPA versus JEPA-plus-reports comparison, together with per-statement and representation analyses, gives readers a more nuanced understanding of when reports help. The paper also explicitly discusses the limits of frozen-probe evaluation and single pretraining runs.
Critical weaknesses
My positive assessment is conditional on interpreting the conclusions within the stated setting: the actual data exposure, the trained checkpoints, and frozen-probe evaluation. The manuscript substantially acknowledges these limitations. Clarifying exactly what is controlled in each comparison, and what equivalence means here, would make the result easier to interpret without detracting from its contribution.
Questions for authors
The following are requests to clarify the existing comparison:
-
Data exposure: Could you summarize the actual number of records and report-bearing records consumed by each method, alongside training exposure? A shared source pool does not necessarily imply identical consumed samples, and the distinction is important when interpreting the cross-method and paired JEPA comparisons.
-
Equivalence and uncertainty: Please explain the practical interpretation of the 0.02 reference equivalence margin alongside the smallest passing margins. It would help to state prominently that the intervals are conditional on the trained checkpoints and do not incorporate independent pretraining-seed variability.
-
Normalization and scope: How should the per-record, per-lead normalization be considered when interpreting the weaker results on voltage and ST–T-related statements? Please connect this preprocessing choice to the scope of the conclusions without extending them to fine-tuning, zero-shot tasks, or report generation.
5 comments
Response to Reviewer 3: Q1
Q1: Records and report-bearing records consumed
Thank you; we agree that a shared source pool does not imply identical consumed samples. Below we give, for each method, the records its released pipeline actually trains on per epoch after its own internal hold-out, how many of them carry a report, and the resulting exposure, with samples equal to records times passes. Validation records are not counted. Tracing each released pipeline at this level of detail refines two entries of Table 5, TolerantECG's records and MERL's passes, and we update Table 5 with these values.
Full-pool methods. JEPA, JEPA + reports, ST-MEM, CLOCS and ECG-FM all train on the full pool of 1,211,640 records, of which 800,034, or 66%, carry a report. The pool contains 800,035 MIMIC-IV-ECG recordings, the count given in the paper, and one of them has no machine-report text. Only JEPA + reports uses the report text. These methods differ only in passes: JEPA and JEPA + reports make 21.1 passes, 25.6M samples; ECG-FM 16.9 passes, 20.5M; CLOCS 8.5 passes, 10.3M; and ST-MEM 4.0 passes, 4.85M.
Report-anchored baselines. TolerantECG and MERL train only on report-bearing records. TolerantECG's released pipeline keeps 780,985 of the 800,035 MIMIC-IV-ECG records and holds out 20% for validation and test with a fixed seed, leaving 624,788 training records, all with reports; at 5 passes it sees 3.12M samples. MERL trains on 392,000 report-bearing records after holding out 8,000 for validation; its released loop runs one epoch beyond its configured 13, so it completes 14 passes, 5.49M samples.
HuBERT-ECG. HuBERT-ECG trains on a subsample of 196,000 records after holding out 3,999 for validation, of which 129,414 carry a report that it does not use; at 14 passes it sees 2.74M samples.
What this means for the comparisons. In the paired JEPA comparison both arms consume identical records and samples, and only the 800,034 report-bearing records contribute a report loss. For the cross-method comparison, our responses to Reviewer 1 (W1/Q1) and Reviewer 2 (W1/Q1) show that on TolerantECG's 624,788 records JEPA at its own schedule remains equivalent to TolerantECG, and that under matched epochs the gap closes steadily with training.
Response to Reviewer 3: Q3
Q3: Normalization and scope
Per-record, per-lead z-scoring removes absolute amplitude from the input, so voltage-defined statements are where this choice can matter. To measure its effect, we pretrained JEPA across 3 seeds with one population-level scale per lead instead, so that absolute amplitude is preserved.
Preserving amplitude helps voltage statements. The voltage/axis gap to TolerantECG roughly halves at 10% labels, from −0.079 to −0.043, a change whose interval excludes zero, and moves from −0.017 to +0.001 at full labels. The weaker voltage results in the paper are therefore largely a consequence of this preprocessing choice.
Preserving amplitude does not help ST-T statements. The ST-T gap barely moves, from −0.011 to −0.008 at full labels, and neither an amplitude-preserving auxiliary loss, beat-synchronous tokens nor morphology-aware masking reduces it, as our response to Reviewer 2 (Q3) reports. The ST-T gap is therefore a modest representational gap rather than a normalization effect.
Either way, the full-label equivalence with TolerantECG holds with amplitude-preserving input, at smallest passing margins of 0.011 on both benchmarks, in a post-submission variant of the recipe; our response to Reviewer 1 (Q3) gives the details.
Scope. Our conclusions therefore concern frozen-probe classification of 10-second, 12-lead ECGs, with the per-statement voltage findings conditional on amplitude-normalized input. We do not extend them to fine-tuning, zero-shot tasks or report generation.
Response to Reviewer 3: W1/Q2
We thank Reviewer 3 for the supportive assessment and for precise clarification requests. W1/Q2 is answered here; Q1 and Q3 follow in separate comments.
W1/Q2: What is controlled, and what equivalence means
We agree, and we now make the conditions explicit next to each result.
What each comparison controls.
What equivalence means. In our paper, "equivalent" means that the 90% interval of the macro-AUROC difference lies inside ±0.02, for frozen linear probes on PTB-XL and SPH at full labels. The interval now also includes pretraining-seed variance. We state this definition in Section 2 and repeat the condition wherever the claim appears.
Practical meaning of 0.02. A difference of 0.02 in macro-AUROC is of the same order as the variation from re-running one method: three JEPA pretraining seeds span 0.911–0.924 on PTB-XL, a range of 0.013, and retraining the released baselines in our pipeline shifted them by up to 0.026. Seed variance, not the paper's probe noise of 0.0008 on PTB-XL, is therefore the dominant uncertainty, and the multi-seed intervals below incorporate it. A gap inside ±0.02 is small but not negligible, which is why we also report the smallest passing margin, which states how tight the evidence is: the submitted single-checkpoint result passes at 0.011 on PTB-XL and 0.017 on SPH.
Pretraining-seed variance. We agree, and have now measured it. We trained 3 pretraining seeds each for JEPA, JEPA + reports and TolerantECG, and combined record-level bootstrap variance with between-seed variance in the interval. The submitted runs are seed 0 of each arm; per-seed values are in our response to Reviewer 1 (W1/Q1).
The equivalence therefore survives pretraining-seed variance, inside the margin on both benchmarks. The same analysis updates the label-efficiency result. Over 3 seeds and 20 patient-level label draws, the 10%-label report effect on PTB-XL is unchanged from the submission, +0.015 [+0.006, +0.024], while on SPH it falls from +0.045 [+0.035, +0.057] to +0.021 [+0.007, +0.035], both clearly positive. We will revise the abstract, Section 3.2, Figure 3 and Table 11 to these values. At full labels the report effect is +0.003 / −0.002, consistent with reports improving efficiency rather than the ceiling.