acceptodds
Under review as a conference paper at ICLR 2027

Contrastive Time-Invariant Learning for Transferable ECG Representations

Abstract

Learning transferable electrocardiogram (ECG) representations requires models to preserve diagnostic information and remain stable under changes that do not affect the diagnosis. Contrastive learning can encourage this stability through positive-pair construction. However, we observe that small temporal shifts still alter model predictions and internal representations. To address this issue, we propose Contrastive Time-Invariant Learning (CTI), which forms positive pairs from two independently circularly shifted views of the same ECG. CTI jointly optimizes supervised classification and cross-view contrastive objectives to explicitly encourage representation consistency across temporal positions, without changing the encoder architecture or test-time inference. In alignment analyses under fixed shifts, CTI improves deep-layer representation consistency at corresponding waveform locations. In frozen-encoder cross-dataset evaluation on three external ECG datasets, CTI improves classification accuracy by 4.9–14.2 percentage points over a two-view cross-entropy baseline using the same shifted views. These results show that explicitly encoding temporal shifts as a contrastive positive-pair relation improves the transferability of ECG representations beyond temporal-shift augmentation alone.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
ECG representation learning, contrastive learning, temporal invariance, cross-dataset transfer

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_hiYMOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes Contrastive Time-Invariant Learning (CTI): two independently circularly shifted views (up to about 5% of the window) of the same 12-lead ECG are classified with cross-entropy and pulled together with a symmetric InfoNCE loss, with the other recordings in the batch as negatives. The encoder is a 6-layer Transformer trained on the five PTB-XL superclasses at 100 Hz. Evaluation includes shift-sensitivity analyses of predictions and token representations, frozen linear transfer to Chapman, CPSC2018 and Georgia against supervised and self-supervised objectives, adding the CTI loss to existing ECG methods, ablations, and shift stability on HAR, Sleep-EDF and Epilepsy. CTI improves transfer accuracy by 4.9 to 14.2 points over the two-view cross-entropy control (CE-2V).

Critical strengths
  1. The main strengths is the control of the obvious confound: CE-2V uses the same shifted views with cross-entropy only, so the gain is not just shift augmentation. The paired tests with Holm correction, padding variants and encoder variants are also well done.
  2. The shift-sensitivity analysis is useful: a plain classifier flips about one in five predictions under small shifts, and the scale-normalized, position-aligned distances separate displacement from real changes in the representation. This sounds useful!
Critical weaknesses
  1. The related work is very limited and many key works are missing. On ECG self-supervision it discusses three methods, but not patient contrastive learning, the systematic self-supervision and augmentation studies on 12-lead ECG, the work showing that contrastive learning is not optimal for quasi-periodic signals, or the recent ECG foundation models (ECGFounder, ECG-FM, HuBERT-ECG, drnetMamba, MERL, HeartLang, ECGJepa,…). On view design for time-series contrastive learning it misses several ICLR papers. On shift invariance it misses architectural solutions. The JS-Consistency baseline is essentially the AugMix consistency loss, which is not cited. Most importantly, the work on why instance discrimination transfers better than supervised pretraining, and why better supervised losses give less transferable features, is missing, and it is directly relevant to the results (see 2).
  2. There is a major lack of comparison to state-of-the-art methods, of which there are many. The ECG-specific methods only appear as deltas after adding CTI, never as standalone rows in Table 1, so we cannot see whether CTI beats them. TS-TCC and TS2Vec are cited but not run, the other time-series contrastive methods above are absent, no released ECG foundation model is evaluated as a frozen encoder on the same targets, and phase-guided normalization is the only shift-invariance method compared. The absolute numbers are also low, so the gains are measured against a weak setup.
  3. The scope is very limited for ICLR: one source dataset, one small Transformer at 100 Hz, single-label tasks built by keeping one code per record, and random record-level splits on the targets. Beyond ECG, CTI is compared only with plain cross-entropy without augmentation, which is exactly the confound the ECG part controls with CE-2V. The idea itself is generic, a shift augmentation used as a contrastive positive pair next to cross-entropy, so for ICLR it would need standard time-series benchmarks with strong baselines. As it stands, it fits way better IEEE JBHI or MLHC.
  4. The method is very limited: cross-entropy on two circularly shifted views plus a SimCLR-style InfoNCE between them, with no change to the model. Shift augmentation for ECG and supervised-plus-contrastive objectives are both standard, so the contribution on the method side is small imo.
  5. the paper's own ablations point away from time invariance. Adding the contrastive term with shared offsets, i.e., with no temporal difference between the two views, already gives about half of CTI's gain over CE-2V on Chapman and CPSC2018. Pulling the shifted views together alone hurts transfer, the gain is flat across shift ranges, and CTI's prediction flip rate is slightly higher than CE-2V's, while JS-Consistency is much more stable and transfers wors. This is what one would expect if the gain comes from instance discrimination, with other recordings as negatives even within a class, which keeps the features from collapsing onto the five source classes, a known effect. The paper can't both call this time-invariant learning and show that the time-invariance part alone does not help. Am I missing something?
  6. The control that would settle this is missing: the same objective with non-temporal augmentations (noise, amplitude scaling, lead masking) at matched strength. Also, CTI's flip rate is 13.51% in Section 4.4 and 11.54% in Table 11. Maybe I am missing something, could you please clarify which checkpoint each number comes from?
Questions for authors

My main concerns are lack of awareness of related work, novelty on the method, and narrow scope. I pointed out at everything in detail above, so I copy paste here.

  1. The related work is very limited and many key works are missing. On ECG self-supervision it discusses three methods, but not patient contrastive learning, the systematic self-supervision and augmentation studies on 12-lead ECG, the work showing that contrastive learning is not optimal for quasi-periodic signals, or the recent ECG foundation models (ECGFounder, ECG-FM, HuBERT-ECG, drnetMamba, MERL, HeartLang, ECGJepa,…). On view design for time-series contrastive learning it misses several ICLR papers. On shift invariance it misses architectural solutions. The JS-Consistency baseline is essentially the AugMix consistency loss, which is not cited. Most importantly, the work on why instance discrimination transfers better than supervised pretraining, and why better supervised losses give less transferable features, is missing, and it is directly relevant to the results (see 2).
  2. There is a major lack of comparison to state-of-the-art methods, of which there are many. The ECG-specific methods only appear as deltas after adding CTI, never as standalone rows in Table 1, so we cannot see whether CTI beats them. TS-TCC and TS2Vec are cited but not run, the other time-series contrastive methods above are absent, no released ECG foundation model is evaluated as a frozen encoder on the same targets, and phase-guided normalization is the only shift-invariance method compared. The absolute numbers are also low, so the gains are measured against a weak setup.
  3. The scope is very limited for ICLR: one source dataset, one small Transformer at 100 Hz, single-label tasks built by keeping one code per record, and random record-level splits on the targets. Beyond ECG, CTI is compared only with plain cross-entropy without augmentation, which is exactly the confound the ECG part controls with CE-2V. The idea itself is generic, a shift augmentation used as a contrastive positive pair next to cross-entropy, so for ICLR it would need standard time-series benchmarks with strong baselines. As it stands, it fits way better IEEE JBHI or MLHC.
  4. The method is very limited: cross-entropy on two circularly shifted views plus a SimCLR-style InfoNCE between them, with no change to the model. Shift augmentation for ECG and supervised-plus-contrastive objectives are both standard, so the contribution on the method side is small imo.
  5. the paper's own ablations point away from time invariance. Adding the contrastive term with shared offsets, i.e., with no temporal difference between the two views, already gives about half of CTI's gain over CE-2V on Chapman and CPSC2018. Pulling the shifted views together alone hurts transfer, the gain is flat across shift ranges, and CTI's prediction flip rate is slightly higher than CE-2V's, while JS-Consistency is much more stable and transfers wors. This is what one would expect if the gain comes from instance discrimination, with other recordings as negatives even within a class, which keeps the features from collapsing onto the five source classes, a known effect. The paper can't both call this time-invariant learning and show that the time-invariance part alone does not help. Am I missing something?
  6. The control that would settle this is missing: the same objective with non-temporal augmentations (noise, amplitude scaling, lead masking) at matched strength. Also, CTI's flip rate is 13.51% in Section 4.4 and 11.54% in Table 11. Maybe I am missing something, could you please clarify which checkpoint each number comes from?
Reviewer_eRtBOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper claims that ECG representations transfer better when models are less sensitive to small temporal shifts. It proposes Contrastive Time-Invariant Learning (CTI), which treats two circularly shifted views of the same ECG, each with an independent offset, as a positive pair. CTI jointly optimizes a cross-entropy loss and a cross-view contrastive loss. Encoders are trained on PTB-XL and evaluated by linear probing on 3 external ECG datasets. Shift sensitivity is measured through prediction flip rates and representation distances, and the paper also reports ablations, fine-tuning, and non-ECG tasks.

Critical strengths

CTI's performance is reported over five seeds and across three target datasets, under both frozen-encoder and full fine-tuning evaluation, and on patient-disjoint multi-label transfer to CODE-15%. Extensive ablations against CE-2V, which uses the same shifted views, show that adding the contrastive term improves transfer across encoder architectures, shift magnitudes, boundary treatments, and loss-weight and temperature settings.

Critical weaknesses

W1. The paper does not establish that time-shift invariance drives transferability, which is the central claim.

The title and abstract attribute CTI's transferability to time-shift invariance, but the paper never links the two. Invariance is measured only on the source dataset (PTB-XL), and transfer only on the target datasets. No result shows that higher invariance comes with better transfer. Figure 3a suggests the opposite: JS-Consistency is more shift-stable than CTI but transfers worse and CTI and CE-2v have similar source FlipRate both different on accuracy. The appendix does not support the link either. The alignment term alone lowers transfer (Table 8), and when both views share the same offset, so that the contrastive term enforces no temporal invariance, it still yields most of CTI's gain on Chapman and CPSC2018 (Table 9).

W2. The comparison with existing ECG representation-learning methods is incomplete.

The ECG-specific methods (CLOCS, 3KG, ST-MEM, ACL-ECG, TF-C) appear only as changes after adding CTI (Table 2), never in absolute terms. The general self-supervised baselines (MoCo v2, BYOL, VICReg, Barlow Twins, SimSiam) are not configured as these methods are normally used. They use only shifted views instead of their standard augmentations, a fixed 60-epoch budget, and loss parameters fixed before training (Appendix A.2), although prior ECG work shows that these methods depend strongly on augmentation choice.

Questions for authors
  1. Link between invariance and transfer (W1). Is there direct evidence that the transfer gains are due to time-shift invariance? For example, how shift-stable are the representations on the target datasets, and does this stability relate to transfer performance across methods or settings?

  2. Conflicting results (W1). How do the authors reconcile the claimed mechanism with three observations? First, JS-Consistency is more shift-stable than CTI but transfers worse. Second, CTI and CE-2V have similar source FlipRates but differ substantially in transfer (Figure 3a). Third, when both views share the same offset, the contrastive term still yields most of CTI's gain on Chapman and CPSC2018 (Table 9).

  3. Generality of the motivating observation. The shift sensitivity is shown for a patch-based Transformer with absolute positional encoding. What is the FlipRate of the 1D CNN in Table 7? Is the sensitivity specific to the patch tokenization?

  4. Comparison with existing methods (W2). What is the absolute performance of CLOCS, 3KG, ST-MEM, ACL-ECG, and TF-C on the three targets? How do the general self-supervised baselines perform with the augmentations commonly used for ECG?

Reviewer_5KHGOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes Contrastive Time-Invariant Learning (CTI), a simple approach to learning transferable ECG representations by reducing unnecessary sensitivity to window-relative temporal positions. The method constructs positive pairs using independently circularly shifted views of the same ECG recording and combines supervised classification with a cross-view contrastive objective.

The authors evaluate CTI on multiple ECG datasets and several non-ECG time-series tasks. The results show substantial improvements in cross-dataset transfer performance compared with existing methods, including an augmentation-matched baseline. Additional analyses investigate representation consistency, prediction stability, and the contributions of different training objectives.

Critical strengths
  1. Although I'm not fully convinced by the significance of ECG representation learning, as it's already widely developed, I personally appreciate the simplicity of the design (simpler is stronger) and its applicability to existing representation-learning frameworks.

  2. The experimental results are impressive, and I like the cross-dataset transfer scenario. Domain shift and generalization is always a challenge in DL.

  3. The analysis of feature scale and positional alignment provides useful insights into how representations respond to temporal transformations. I also appreciate the authors' observation that better prediction stability does not necessarily imply better transfer performance.

Critical weaknesses
  1. The distinction between diagnostically relevant "relative timing" and non-diagnostic "window-relative temporal position" is not immediately clear. Both expressions seem to refer to relative temporal information. The intended contrast is hard to follow. I suggest providing explicit definitions or examples, or using figures to help illustrate. This would make the motivation more intuitive and clarify the intended invariance.

  2. The paper motivates CTI primarily to reduce dependence on window-relative temporal positions. But as far as I can tell from the results, the observed improvements may arise from both general contrastive representation learning and the specific temporal-shift consistency encouraged by the proposed positive pairs. The authors have partially investigated this issue through the shared-versus-independent shift comparison in Table 9 and the objective-component analysis in Table 8. But I found the shared-offset contrastive objective already provides substantial transfer improvements, suggesting that shift-specific invariance may not fully explain CTI's effectiveness. Additional controlled analysis would help and would strengthen the connection between the proposed motivation and the experimental findings.

  3. An important motivation of the paper is that ECG representations should preserve diagnostically meaningful temporal structures, such as waveform morphology, rhythm, and relative timing, while removing unnecessary dependence on window-relative temporal position. But most of the experiments report improved classification performance and representation consistency under temporal shifts. These can be viewed as indirect evidence. I think it would be better to provide a clearer illustration of whether the learned representations preserve the specific temporal relationships highlighted in the introduction. Alternatively, the authors could clarify the scope of the preservation claim.

Questions for authors
  1. Given that the shared-offset contrastive objective in Table 9 already improves transfer performance, could the authors further clarify or isolate how much of CTI's improvement is attributable specifically to temporal-shift invariance rather than general contrastive learning?

  2. Can the authors provide more direct evidence, or a more detailed discussion, demonstrating that CTI preserves diagnostically relevant temporal relationships while reducing sensitivity to window-relative temporal position?

  3. What is the key methodological distinction between CTI and existing contrastive learning approaches that already incorporate temporal shifts into their view construction?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.