acceptodds
Under review as a conference paper at ICLR 2027

JOINT-EMBEDDING PREDICTIVE LEARNING CLOSES THE UNIMODAL–MULTIMODAL GAP IN ECG PRE- TRAINING

Abstract

In time series representation learning, pairing signals with free-text descriptions typically outperforms signal-only pretraining; in ECG, where most recordings carry a written interpretive report, the gap is read as evidence that language supplies information the waveform lacks. Whether it survives a controlled comparison is unknown: published comparisons vary the pretraining corpus, the evaluation protocol and whether baselines are retrained at all. We fix all three, retraining six published methods from their official code on 1.21 million recordings and probing every model with the same frozen linear probe on PTB-XL and SPH. Under these controls, joint-embedding predictive learning (JEPA) largely closes the gap: it comes within 0.005 macro-AUROC of the strongest multimodal model on PTB-XL and scores 0.007 above it on SPH, the two being equivalent under two one-sided tests with a 0.02 margin. Reconstruction, discrete-target and contrastive baselines trail behind. Adding reports gives no gain with all labels, but adds 0.015 and 0.045 with 10% of them. Language buys label efficiency rather than a higher ceiling: signal-only latent prediction reaches the same performance without ever seeing a report, and whether text earns its cost elsewhere in time series needs the same controls applied.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Self Supervised Representation, JEPA, Foundation Model, 12 LeadECG

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_DJowOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper asks how much of the apparent unimodal–multimodal gap in ECG pretraining survives a controlled comparison. Existing methods are retrained under a common source-pool and preprocessing framework, then evaluated with a shared frozen linear-probe protocol on PTB-XL and SPH. A signal-only JEPA encoder is also compared with its report-anchored counterpart at different downstream label budgets.

The reported pattern is that latent prediction approaches the leading multimodal baseline with full labels, whereas reports provide clearer benefits at reduced label budgets.

Critical strengths

I recommend weak acceptance: the controlled empirical comparison is a valuable contribution. Two considerations are particularly important to this assessment.

(i) The paper tackles an important empirical question and appropriately treats controlled measurement as its central contribution. Re-training baselines, standardizing the probe protocol, and using an explicit equivalence test make the comparison more informative than juxtaposing published headline numbers.

(ii) The distinction between full-label performance and label efficiency is useful. The paired JEPA versus JEPA-plus-reports comparison, together with per-statement and representation analyses, gives readers a more nuanced understanding of when reports help. The paper also explicitly discusses the limits of frozen-probe evaluation and single pretraining runs.

Critical weaknesses

My positive assessment is conditional on interpreting the conclusions within the stated setting: the actual data exposure, the trained checkpoints, and frozen-probe evaluation. The manuscript substantially acknowledges these limitations. Clarifying exactly what is controlled in each comparison, and what equivalence means here, would make the result easier to interpret without detracting from its contribution.

Questions for authors

The following are requests to clarify the existing comparison:

  • Data exposure: Could you summarize the actual number of records and report-bearing records consumed by each method, alongside training exposure? A shared source pool does not necessarily imply identical consumed samples, and the distinction is important when interpreting the cross-method and paired JEPA comparisons.

  • Equivalence and uncertainty: Please explain the practical interpretation of the 0.02 reference equivalence margin alongside the smallest passing margins. It would help to state prominently that the intervals are conditional on the trained checkpoints and do not incorporate independent pretraining-seed variability.

  • Normalization and scope: How should the per-record, per-lead normalization be considered when interpreting the weaker results on voltage and ST–T-related statements? Please connect this preprocessing choice to the scope of the conclusions without extending them to fine-tuning, zero-shot tasks, or report generation.

5 comments
AuthorsOpenReview ↗

Response to Reviewer 3: W1/Q2

We thank Reviewer 3 for the supportive assessment and for precise clarification requests. W1/Q2 is answered here; Q1 and Q3 follow in separate comments.

W1/Q2: What is controlled, and what equivalence means

We agree, and we now make the conditions explicit next to each result.

What each comparison controls.

ComparisonHeld fixedAllowed to differ
Cross-method, Table 1Source pool; downstream datasets, splits and labels; probe; statistics; final-checkpoint ruleRecords consumed, passes, input preprocessing, crop length and sampling rate, each as in the method's published configuration
JEPA vs. JEPA + reportsArchitecture, recipe, records, steps, seedReport anchoring only
New: identical recordsTraining records, TolerantECG's 624,788Schedule: each method's own, about 41 JEPA passes against 5, or matched epochs at 5, 10 and 21
New: matched passesAbout 21 passes for both methodsRecords consumed
New: matched samples3.12M samples for both methods, schedule completedRecipe: post-submission JEPA variant with an amplitude side channel and learning-rate warmup; records consumed
New: seed replicatesEverything in the cross-method rowPretraining seed, 3 per arm

What equivalence means. In our paper, "equivalent" means that the 90% interval of the macro-AUROC difference lies inside ±0.02, for frozen linear probes on PTB-XL and SPH at full labels. The interval now also includes pretraining-seed variance. We state this definition in Section 2 and repeat the condition wherever the claim appears.

Practical meaning of 0.02. A difference of 0.02 in macro-AUROC is of the same order as the variation from re-running one method: three JEPA pretraining seeds span 0.911–0.924 on PTB-XL, a range of 0.013, and retraining the released baselines in our pipeline shifted them by up to 0.026. Seed variance, not the paper's probe noise of 0.0008 on PTB-XL, is therefore the dominant uncertainty, and the multi-seed intervals below incorporate it. A gap inside ±0.02 is small but not negligible, which is why we also report the smallest passing margin, which states how tight the evidence is: the submitted single-checkpoint result passes at 0.011 on PTB-XL and 0.017 on SPH.

Pretraining-seed variance. We agree, and have now measured it. We trained 3 pretraining seeds each for JEPA, JEPA + reports and TolerantECG, and combined record-level bootstrap variance with between-seed variance in the interval. The submitted runs are seed 0 of each arm; per-seed values are in our response to Reviewer 1 (W1/Q1).

All training labels, 3 seeds per armPTB-XLSPH
JEPA minus TolerantECG−0.003, equivalent within 0.016+0.002, equivalent within 0.012
JEPA + reports minus TolerantECG−0.000, equivalent within 0.009+0.000, equivalent within 0.009
Seed pairs, JEPA × TolerantECG, inside ±0.029 of 99 of 9

The equivalence therefore survives pretraining-seed variance, inside the margin on both benchmarks. The same analysis updates the label-efficiency result. Over 3 seeds and 20 patient-level label draws, the 10%-label report effect on PTB-XL is unchanged from the submission, +0.015 [+0.006, +0.024], while on SPH it falls from +0.045 [+0.035, +0.057] to +0.021 [+0.007, +0.035], both clearly positive. We will revise the abstract, Section 3.2, Figure 3 and Table 11 to these values. At full labels the report effect is +0.003 / −0.002, consistent with reports improving efficiency rather than the ceiling.

AuthorsOpenReview ↗

Response to Reviewer 3: Q1

Q1: Records and report-bearing records consumed

Thank you; we agree that a shared source pool does not imply identical consumed samples. Below we give, for each method, the records its released pipeline actually trains on per epoch after its own internal hold-out, how many of them carry a report, and the resulting exposure, with samples equal to records times passes. Validation records are not counted. Tracing each released pipeline at this level of detail refines two entries of Table 5, TolerantECG's records and MERL's passes, and we update Table 5 with these values.

Full-pool methods. JEPA, JEPA + reports, ST-MEM, CLOCS and ECG-FM all train on the full pool of 1,211,640 records, of which 800,034, or 66%, carry a report. The pool contains 800,035 MIMIC-IV-ECG recordings, the count given in the paper, and one of them has no machine-report text. Only JEPA + reports uses the report text. These methods differ only in passes: JEPA and JEPA + reports make 21.1 passes, 25.6M samples; ECG-FM 16.9 passes, 20.5M; CLOCS 8.5 passes, 10.3M; and ST-MEM 4.0 passes, 4.85M.

Report-anchored baselines. TolerantECG and MERL train only on report-bearing records. TolerantECG's released pipeline keeps 780,985 of the 800,035 MIMIC-IV-ECG records and holds out 20% for validation and test with a fixed seed, leaving 624,788 training records, all with reports; at 5 passes it sees 3.12M samples. MERL trains on 392,000 report-bearing records after holding out 8,000 for validation; its released loop runs one epoch beyond its configured 13, so it completes 14 passes, 5.49M samples.

HuBERT-ECG. HuBERT-ECG trains on a subsample of 196,000 records after holding out 3,999 for validation, of which 129,414 carry a report that it does not use; at 14 passes it sees 2.74M samples.

What this means for the comparisons. In the paired JEPA comparison both arms consume identical records and samples, and only the 800,034 report-bearing records contribute a report loss. For the cross-method comparison, our responses to Reviewer 1 (W1/Q1) and Reviewer 2 (W1/Q1) show that on TolerantECG's 624,788 records JEPA at its own schedule remains equivalent to TolerantECG, and that under matched epochs the gap closes steadily with training.

AuthorsOpenReview ↗

Response to Reviewer 3: Q3

Q3: Normalization and scope

Per-record, per-lead z-scoring removes absolute amplitude from the input, so voltage-defined statements are where this choice can matter. To measure its effect, we pretrained JEPA across 3 seeds with one population-level scale per lead instead, so that absolute amplitude is preserved.

Preserving amplitude helps voltage statements. The voltage/axis gap to TolerantECG roughly halves at 10% labels, from −0.079 to −0.043, a change whose interval excludes zero, and moves from −0.017 to +0.001 at full labels. The weaker voltage results in the paper are therefore largely a consequence of this preprocessing choice.

Preserving amplitude does not help ST-T statements. The ST-T gap barely moves, from −0.011 to −0.008 at full labels, and neither an amplitude-preserving auxiliary loss, beat-synchronous tokens nor morphology-aware masking reduces it, as our response to Reviewer 2 (Q3) reports. The ST-T gap is therefore a modest representational gap rather than a normalization effect.

Either way, the full-label equivalence with TolerantECG holds with amplitude-preserving input, at smallest passing margins of 0.011 on both benchmarks, in a post-submission variant of the recipe; our response to Reviewer 1 (Q3) gives the details.

Scope. Our conclusions therefore concern frozen-probe classification of 10-second, 12-lead ECGs, with the per-statement voltage findings conditional on amplitude-normalized input. We do not extend them to fine-tuning, zero-shot tasks or report generation.

Reviewer_8A2VOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents a controlled empirical study of ECG pretraining that evaluates the multimodal (ECG+report) and unimodal (ECG‑only) objectives. The authors pretrain eight encoders (six prior methods retrained from official code plus two arms of a JEPA encoder with and without report anchoring) on a unified corpus of 1.21M ECGs, and evaluate all with the same frozen linear probe on two held‑out datasets (PTB‑XL and SPH) at 1%, 10%, and 100% label budgets, using paired bootstraps and two one‑sided tests (TOST) to assess equivalence. Their main finding is that a signal‑only latent‑prediction encoder (JEPA) is statistically equivalent to the strongest multimodal baseline with all labels, while reports provide gains specifically in low‑label regimes, suggesting language primarily improves label efficiency rather than the performance ceiling.

Critical strengths
  1. A rigorous and fair experimental protocol that fixes corpus, preprocessing, probing, splits, and checkpoint selection is valuable.
  2. The paper clearly states “full‑label” versus “reduced‑label” regimes and provides per‑statement analyses that reveal that text anchoring improves label efficiency, not the ceiling.
Critical weaknesses
  1. Multimodal baselines are pretrained with report‑bearing examples (i.e., 66% carry machine-generated report, MIMIC-IV-ECG subset) while JEPA leverages the full 1.21M pool. This data exposure asymmetry might disadvantage multimodal methods beyond objective design.
  2. The evaluation focuses on frozen linear probing; other settings where multimodality is often motivated—zero‑shot, retrieval, or text‑conditioned inference—are out of scope, limiting generality of “no multimodal advantage” claims.
Questions for authors
  1. How do you assess the fairness of comparing JEPA trained on the full 1.21M pool to multimodal methods that, by design, consume only report‑bearing rows? 
  2. How would your conclusions change under alternative evaluations (e.g., zero‑shot label phrases, report retrieval, or longer‑context signals) where multimodality is often expected to shine?
  3. Could the JEPA deficit on ST‑T/voltage be reduced by simple augmentations or auxiliary losses that preserve absolute amplitude (e.g., amplitude calibration, morphology‑aware masking, beat‑synchronous tokens)? Any initial experiments?
  4. Can you report compute budgets (GPU days) and training wall‑times for each retrained baseline and JEPA to better contextualize efficiency and reproducibility?
7 comments
AuthorsOpenReview ↗

Response to Reviewer 2: W1/Q1

We thank Reviewer 2 for recognizing the value of the fixed protocol and the distinction between full-label and reduced-label performance. W1/Q1 is answered here; W2/Q2, Q3 and Q4 follow in separate comments.

W1/Q1: Data-exposure asymmetry

Thank you for this important point. We assess fairness in two ways.

Design principle. Each method consumes the data its objective can use, which is the choice a practitioner faces: a signal-only objective can learn from every recording, while a report-anchored objective needs a report. To check that this principle is not what drives the result, we also ran the controlled comparison the reviewer has in mind, from both sides.

Removing JEPA's advantage. Our response to Reviewer 1 (W1/Q1) gives the full results. Trained only on the 624,788 report-bearing records that TolerantECG trains on, with its own 100k-step schedule, about 41 passes over them, JEPA remains equivalent to TolerantECG across 3 seeds: −0.005 within 0.018 on PTB-XL and −0.003 within 0.019 on SPH. Removing the non-report recordings changes it by a non-significant −0.001 / −0.006. With epochs matched as well, the gap closes steadily with training, from −0.049 / −0.030 at 5 epochs each to −0.015 / −0.005 at 21, against one TolerantECG run at 21 epochs.

Giving a multimodal method more data. We retrained MERL on the 602,464 of TolerantECG's training records that pass MERL's own report filter, 1.5× its released set, for 21 epochs instead of 14. (MERL's released training loop runs one epoch beyond its configured 13, so the released run completes 14 epochs; we update Table 5 with this value.) It changes by −0.001 [−0.055, +0.053] / +0.007 [−0.064, +0.078] relative to its released configuration; these are 95% intervals from 2 seeds against one released run, hence wide. At this exposure JEPA's point estimates remain higher, +0.022 / +0.031, comparing JEPA trained on TolerantECG's 624,788 records for 21 epochs with two seeds each, although both intervals include zero, so JEPA's lead over MERL is less certain here than in Table 1; with all four 21-epoch JEPA runs it is +0.015 [−0.001, +0.031] / +0.025 [+0.005, +0.045]. TolerantECG at 21 epochs is ahead of MERL by +0.030 / +0.030.

Pool access alone does not produce the result. Other signal-only methods trained on the same full pool, namely ST-MEM, ECG-FM and CLOCS, remain well below JEPA in Table 1. We therefore find no evidence that the exposure asymmetry disadvantages the multimodal methods at their own schedules. The paired JEPA versus JEPA + reports comparison was already controlled: both arms use the same records, steps and architecture, and differ only in report anchoring.

AuthorsOpenReview ↗

Response to Reviewer 2: W2/Q2

W2/Q2: Scope beyond frozen linear probing

We agree that the claim must be read within the frozen-probe setting, and we now state this more precisely. We chose frozen probing because it measures representation quality directly and is a setting where signal-only and report-anchored encoders can be compared on equal terms, without text at test time. Within that setting we go beyond a single linear probe: the paper includes an MLP probe and per-statement analyses, and we now add width-matched probes and amplitude-augmented probes, described in our response to Reviewer 1 (Q2 and Q3).

Alternative evaluations. Zero-shot classification from label phrases, report retrieval and text-conditioned inference require a text tower, which signal-only methods do not have by construction. For these tasks we would expect report-anchored methods to have the advantage, since they require a shared text–signal space that signal-only encoders do not learn, and we make no claim of "no multimodal advantage" there. Our evaluation also uses 10-second, 12-lead recordings, so we make no claim about longer-context signals such as Holter monitoring.

Where reports help. Our results are consistent with the expectation that multimodality helps where supervision is scarce. At 10% labels, over 3 pretraining seeds and 20 label draws, report anchoring improves JEPA by +0.015 [+0.006, +0.024] on PTB-XL, unchanged from the submission, and by +0.021 [+0.007, +0.035] on SPH, which replaces the submission's single-draw +0.045; our response to Reviewer 3 (Q2) gives details. It does not help at full labels, +0.003 / −0.002. Language therefore acts mainly on label efficiency, a positive and practically useful role for reports that the paper highlights.

AuthorsOpenReview ↗

Response to Reviewer 2: Q3

Q3: Reducing the ST-T / voltage deficit

Thank you for these suggestions; we have tested each of them. We tested amplitude calibration in two forms, ran an amplitude-preserving auxiliary loss and beat-synchronous tokens as new experiments for this response, and had tested two morphology-aware masking schemes during method development. All post-submission recipe changes here were selected using PTB-XL test-fold scores, so SPH is the only benchmark untouched by selection, and we treat it as the primary test.

  1. Amplitude-preserving input, with one population-level scale per lead instead of per-record z-scoring, all else equal. As our response to Reviewer 1 (Q3) details, the voltage/axis gap to TolerantECG roughly halves at 10% labels, a change of +0.036 [+0.009, +0.062], and moves from −0.017 to +0.001 at full labels, an interval that still includes zero; the ST-T gap did not change measurably.
  2. Amplitude as a side channel. The encoder receives per-lead z-scored input plus the per-lead amplitude scales through a separate learned embedding. This variant also adds a 2,000-step learning-rate warmup, so its result is not attributable to amplitude alone. To match exposure exactly, we trained it for 12,200 steps, 3.12M samples, the same sample exposure as TolerantECG's 5-epoch run, with its schedule completed at that point, on 3 seeds. Under full labels it is equivalent to TolerantECG on SPH, +0.002 with 90% CI [−0.008, +0.012], within ±0.012, and on PTB-XL, −0.010 with 90% CI [−0.019, −0.001], within ±0.019; on PTB-XL it is thus equivalent at the 0.02 margin but significantly lower. With 10% of labels it does not reach equivalence, −0.058 on SPH and −0.023 on PTB-XL, so we limit this result to the full-label setting. For the same recipe at 20k steps, the full-label change relative to its z-scored control is +0.019 [+0.000, +0.037] on voltage/axis and +0.005 [−0.007, +0.018] on ST-T.
  3. Amplitude-preserving auxiliary loss, new. On amplitude-preserving input, a small head on the pooled encoder features predicts, for each lead, the median beat's QRS peak-to-peak amplitude, ST level and T-wave amplitude relative to the PR baseline, 36 targets with loss weight 0.1, across 3 seeds against the same recipe without the head. At full labels it changes the voltage/axis group by +0.011 [−0.001, +0.022] and the ST-T group by +0.004 [−0.006, +0.014], neither significant. Macro-AUROC rises slightly, +0.005 on PTB-XL and +0.007 on SPH, and the variant remains equivalent to TolerantECG, within 0.013 on PTB-XL and 0.017 on SPH.
  4. Beat-synchronous tokens, new. Each token is one complete heartbeat, delimited from the R peaks and resampled to a fixed length, with the beat's RR interval as an extra input channel so that rhythm timing is preserved; masking then hides whole beats. Across 3 seeds against fixed-length patch tokens, beat tokens make the representation worse: the ST-T group drops by −0.039 [−0.056, −0.021], and macro-AUROC by −0.023 on PTB-XL and −0.010 on SPH. A plausible cause is that resampling every beat to a fixed length removes absolute durations within the beat, such as QRS width and QT interval, which fixed-length patches keep.
  5. Morphology-aware masking, development stage. We tested two beat-aware masking schemes on an earlier version of our recipe, with a single seed on PTB-XL. Beat-length masking, where each masked span covers one cardiac cycle, lowered macro-AUROC from 0.860 with fixed block masking to 0.822, mostly on rhythm statements. QRS-phase masking, which hides every QRS complex while leaving P and T waves visible, lowered it to 0.763 and specifically hurt ST depression, −0.089 [−0.130, −0.051]. We therefore kept fixed block masking.

For the two new experiments, the configuration and decision rule were fixed before training: a method counts as reducing a deficit only if its 95% interval for the voltage/axis or ST-T group, pooled over both benchmarks at full labels, excludes zero in its favor. Neither meets it. The voltage deficit shrinks substantially when amplitude is preserved, most clearly at 10% labels. The ST-T deficit is not removed by any of these approaches; with amplitude preserved it is small and not significant, −0.008 [−0.028, +0.012], or −0.004 with the auxiliary loss, and we report it as a limitation.

Reviewer_sEufOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper investigates whether signal-only ECG pretraining can match report-anchored multimodal pretraining under a standardized evaluation protocol. It compares a JEPA encoder and its report-anchored counterpart with six retrained published methods, using a pretraining source pool of 1.21 million ECGs and frozen-probe evaluation on PTB-XL and SPH. With all downstream training labels, JEPA achieves macro-AUROCs of 0.921 and 0.967, compared with 0.926 and 0.960 for TolerantECG, satisfying the stated equivalence criterion at a margin of 0.02. Reports improve the JEPA encoder at the 10% label budget. Additional analyses examine representation similarity, feature complementarity, diagnostic categories, and sensitivity to evaluation choices.

Critical strengths

1.The paper addresses an important empirical question: how much of the reported multimodal advantage survives standardized evaluation and baseline retraining?

2.The evaluation goes beyond aggregate benchmark scores. Explicit equivalence testing, patient-level resampling, an alternative MLP probe, and per-statement analyses help characterize the scope of the findings. The distinction between full-label performance and reduced-label performance provides useful evidence about when report supervision benefits the evaluated representations.

Critical weaknesses

The central comparison does not sufficiently control actual pretraining exposure and training budget. Although the source pool and downstream evaluation are standardized, Table 5 shows that JEPA sees 1,211,640 recordings for 21.1 passes, whereas TolerantECG sees 800,035 recordings for 5 passes and MERL sees 392,000 recordings for 13 passes. These differences leave substantial variation in data coverage and training exposure. Importantly, Table 17 reports that JEPA still trails TolerantECG by approximately 0.034 on PTB-XL at comparable numbers of passes. The authors do not extend TolerantECG’s training to determine whether its performance would also improve. Consequently, the final-checkpoint comparison does not establish that the unimodal–multimodal gap closes under matched pretraining conditions. The within JEPA report ablation remains informative, but it does not resolve this limitation of the cross-method comparison.

Questions for authors
  1. Please provide a comparison between JEPA and TolerantECG on the identical report-bearing MIMIC-IV-ECG subset under comparable training exposure. Can TolerantECG also be trained for a longer schedule to establish whether the full-label gap remains closed when both methods are adequately trained? Table 17 suggests that the PTB-XL conclusion depends materially on the training budget.

  2. Please repeat the feature-complementarity analysis with dimensionality-matched representations and regularization selected appropriately for the concatenated features. Does the finding that other encoders provide no additional benefit to JEPA persist under these controls? The directional asymmetry in Figure 5 alone does not exclude dimensionality or regularization effects.

  3. How much of the voltage/ST-T performance pattern is attributable to per-record, per-lead z-score normalization, which removes absolute amplitude scales? Please add an amplitude-preserving preprocessing control and examine whether the same category-level differences remain.

6 comments
AuthorsOpenReview ↗

Response to Reviewer 1: W1/Q1

We thank Reviewer 1 for a precise and constructive review. W1/Q1 is answered here; Q2 and Q3 follow in separate comments.

W1/Q1: Pretraining exposure and training budget

This is the most important check on our claim. We ran both requested experiments and one further control, and the results further strengthen the paper's claims.

TolerantECG trained longer. We changed only the number of epochs in TolerantECG's released configuration. Its released scheduler is a cosine with a 2,000-epoch period, stepped once per epoch, so the learning rate stays within 0.1% of its initial 3×10−53\times10^{-5} in every run; longer training continues the released schedule. The 21-epoch run started from scratch, stopped at 10 epochs and resumed unchanged; a second run, continuing 5-epoch seed 1, gives the second 10-epoch point.

TolerantECG trainingPTB-XLSPH
5 epochs, released, mean of 3 seeds0.9220.963
10 epochs, mean of 2 runs0.9240.966
21 epochs, 1 run0.9250.964
21 minus 5 epochs+0.003, equivalent within 0.014+0.001, equivalent within 0.012

Four times longer training changes TolerantECG by at most 0.003, equivalent to zero on both benchmarks; note that the 5-epoch row is a 3-seed mean and the 21-epoch row a single run. Table 1's runs, 0.926 / 0.960 for TolerantECG, are seed 0 of each arm. Per-seed macro-AUROC, seeds 0 / 1 / 2:

ArmPTB-XLSPH
JEPA0.921 / 0.924 / 0.9110.967 / 0.965 / 0.964
JEPA + reports0.923 / 0.917 / 0.9240.963 / 0.963 / 0.962
TolerantECG0.926 / 0.918 / 0.9220.960 / 0.965 / 0.964

At about 21 passes each, JEPA minus TolerantECG is −0.006 on PTB-XL and +0.001 on SPH: equivalent within 0.016 on SPH, and within 0.021 on PTB-XL, just outside our 0.02 margin.

Reading Table 17. Table 17 uses an intermediate checkpoint of one 100k-step JEPA run. Retraining JEPA with its schedule completed at 5 epochs gives a PTB-XL gap of −0.049, so Table 17 did not understate the matched-budget gap: JEPA needs more passes than TolerantECG. The plateau at 13 passes in Figure 10 refers to checkpoints within one schedule over the full pool; we will clarify this in Section 3.5.

The identical report-bearing records. TolerantECG's released code fixes its split seed, so we rebuilt exactly its 624,788 training records and trained JEPA on them with the paper's recipe unchanged. JEPA's 100,000 steps at batch 256 are about 41 passes over these records.

JEPA on TolerantECG's records, minus TolerantECGPTB-XLSPH
JEPA's own schedule, about 41 passes, 3 seeds; TolerantECG as released, 5 passes−0.005, equivalent within 0.018−0.003, equivalent within 0.019
5 epochs each−0.049−0.030
10 epochs each, 2 runs per method−0.028−0.006
21 epochs each, 4 JEPA runs, 1 TolerantECG run−0.015−0.005
JEPA on full pool minus JEPA on these records+0.001, n.s.+0.006, n.s.

The four 21-epoch JEPA runs are 2 seeds, each trained on two GPU types, which did not reproduce exactly, so we pool them; the original pair alone gives −0.008 / +0.000. The last row is computed from unrounded values.

Three conclusions follow. First, on identical records JEPA at its own schedule is equivalent to TolerantECG on both benchmarks, and removing the non-report recordings makes no significant difference, a comparison that also differs in passes. Second, under matched epochs the gap closes steadily with training: on SPH it is already −0.006 by 10 epochs, and on PTB-XL it narrows from −0.049 at 5 epochs to −0.015 at 21, approaching the margin. JEPA simply converges over more passes, each of which costs a fraction of TolerantECG's, and given its own schedule it reaches equivalence on both benchmarks. Third, longer TolerantECG training does not reopen the gap. A second 21-epoch TolerantECG run is training, and we will post its result when it completes.

Wall-clock time. On the same GPU type, JEPA's 21 passes take 2.9 GPU-hours and TolerantECG's 5 passes 25–30, about nine times as long; its 21-epoch run took 105.5 GPU-hours, about 36 times JEPA's.

AuthorsOpenReview ↗

Response to Reviewer 1: Q2

Q2: Feature-complementarity controls

Thank you; this made the analysis more rigorous. Every encoder is reduced by PCA, fitted on the training split, to JEPA's own width of 384, so all concatenated probes compare representations of equal dimension without compressing JEPA. At 384 components PCA retains at least 99.5% of each encoder's feature variance (TolerantECG 99.5–99.6%, ST-MEM 99.6%, MERL 99.8%, ECG-FM >99.99%), and the conclusions below also hold with every encoder at its full native width without PCA. Weight decay is tuned on validation over the same six-value grid, separately for every single and every concatenated probe. Gains involving JEPA and TolerantECG are paired over 3 pretraining seeds; MERL, ST-MEM and ECG-FM have one run each. Intervals are 95%.

Gain from adding X to JEPA, width 384PTB-XLSPH
TolerantECG+0.002 [−0.009, +0.012]+0.001 [−0.019, +0.022]
MERL−0.007 [−0.021, +0.006]−0.004 [−0.019, +0.011]
ST-MEM−0.013 [−0.026, −0.001]−0.014 [−0.035, +0.008]
ECG-FM−0.016 [−0.033, +0.001]−0.012 [−0.041, +0.017]
Reverse: adding JEPA to MERL+0.016 [+0.001, +0.031]+0.020 [+0.003, +0.036]
Reverse: adding JEPA to ST-MEM+0.024 [+0.008, +0.040]+0.015 [−0.011, +0.041]
Reverse: adding JEPA to ECG-FM+0.031 [+0.015, +0.046]+0.029 [−0.003, +0.061]
Reverse: adding JEPA to TolerantECG−0.003 [−0.020, +0.014]+0.005 [−0.011, +0.020]

At matched width no encoder adds significantly to JEPA on either benchmark. In the reverse direction JEPA adds to MERL on both benchmarks, and to ST-MEM and ECG-FM on PTB-XL, with SPH pointing the same way but intervals including zero; the asymmetry of Figure 5c therefore persists under both controls. JEPA and TolerantECG: neither adds significant linearly decodable task information to the other, and the SPH intervals are wide.

AuthorsOpenReview ↗

Response to Reviewer 1: Q3

Q3: Amplitude normalization and the voltage / ST-T pattern

Thank you for this question. We added amplitude-preserving controls at the probe level and at the pretraining level.

Probe level. Without amplitude information, PTB-XL equivalence is sensitive to probe width: with PCA widths of 32, 64 and 128 it fails, at smallest margins of 0.0203–0.031. Appending to each encoder's features the 24 amplitude scalars that per-record, per-lead z-scoring removes, 12 lead means and 12 log standard deviations, restores equivalence on both benchmarks at every probe width, native and PCA 32, 64 and 128, with smallest passing margins of 0.011–0.0195 on PTB-XL and 0.008–0.014 on SPH. JEPA's sensitivity to probe width on PTB-XL is therefore consistent with amplitude scale removed in preprocessing rather than with diagnostic information absent from the waveform. We will add this width analysis to the appendix.

Pretraining level. We pretrained JEPA with 3 seeds on amplitude-preserving input that uses one population-level scale per lead, so absolute voltage is retained. We compared it with an otherwise identical JEPA trained on per-record z-scored input, and recomputed the paper's statement-category analysis, splitting the amplitude statements into voltage/axis and ST-T as an exploratory split.

Gap to TolerantECG, pooled over both benchmarksPer-record z-scoreAmplitude-preservingChange
Voltage / axis, all labels−0.017+0.001+0.018 [−0.002, +0.038]
Voltage / axis, 10% labels−0.079−0.043+0.036 [+0.009, +0.062]
ST-T, all labels−0.011−0.008+0.004 [−0.010, +0.017]
ST-T, 10% labels−0.053−0.038+0.015 [−0.006, +0.036]
Timing / rhythm, negative control, all labels+0.010+0.010−0.000 [−0.006, +0.005]

What the controls show. The answer is category-specific, and the two categories the reviewer names behave differently.

  • Voltage: partly a preprocessing effect. With amplitude preserved, the voltage gap roughly halves at 10% labels, from −0.079 to −0.043, a change whose interval excludes zero. At full labels it moves the same way, from −0.017 to +0.001, although that interval still includes zero. The weaker voltage results in the paper are therefore substantially conditional on per-record, per-lead normalization, which removes absolute amplitude from JEPA's input.
  • ST-T: a modest gap that preserving amplitude did not measurably reduce. The ST-T deficit did not change measurably, +0.004 [−0.010, +0.017] at full labels; our data do not show that amplitude accounts for it, though the interval cannot exclude it. Our response to Reviewer 2 (Q3) reports the other approaches we tested: an amplitude side channel, an amplitude-preserving auxiliary loss, beat-synchronous tokens and two masking schemes. None of them reduces the ST-T deficit.

Timing statements carry no amplitude information and do not move, which confirms that the control is specific. The full-label equivalence with TolerantECG also holds with amplitude-preserving input, at smallest passing margins of 0.011 on both benchmarks across 3 seeds.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.