- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes Contrastive Time-Invariant Learning (CTI): two independently circularly shifted views (up to about 5% of the window) of the same 12-lead ECG are classified with cross-entropy and pulled together with a symmetric InfoNCE loss, with the other recordings in the batch as negatives. The encoder is a 6-layer Transformer trained on the five PTB-XL superclasses at 100 Hz. Evaluation includes shift-sensitivity analyses of predictions and token representations, frozen linear transfer to Chapman, CPSC2018 and Georgia against supervised and self-supervised objectives, adding the CTI loss to existing ECG methods, ablations, and shift stability on HAR, Sleep-EDF and Epilepsy. CTI improves transfer accuracy by 4.9 to 14.2 points over the two-view cross-entropy control (CE-2V).
Critical strengths
- The main strengths is the control of the obvious confound: CE-2V uses the same shifted views with cross-entropy only, so the gain is not just shift augmentation. The paired tests with Holm correction, padding variants and encoder variants are also well done.
- The shift-sensitivity analysis is useful: a plain classifier flips about one in five predictions under small shifts, and the scale-normalized, position-aligned distances separate displacement from real changes in the representation. This sounds useful!
Critical weaknesses
- The related work is very limited and many key works are missing. On ECG self-supervision it discusses three methods, but not patient contrastive learning, the systematic self-supervision and augmentation studies on 12-lead ECG, the work showing that contrastive learning is not optimal for quasi-periodic signals, or the recent ECG foundation models (ECGFounder, ECG-FM, HuBERT-ECG, drnetMamba, MERL, HeartLang, ECGJepa,…). On view design for time-series contrastive learning it misses several ICLR papers. On shift invariance it misses architectural solutions. The JS-Consistency baseline is essentially the AugMix consistency loss, which is not cited. Most importantly, the work on why instance discrimination transfers better than supervised pretraining, and why better supervised losses give less transferable features, is missing, and it is directly relevant to the results (see 2).
- There is a major lack of comparison to state-of-the-art methods, of which there are many. The ECG-specific methods only appear as deltas after adding CTI, never as standalone rows in Table 1, so we cannot see whether CTI beats them. TS-TCC and TS2Vec are cited but not run, the other time-series contrastive methods above are absent, no released ECG foundation model is evaluated as a frozen encoder on the same targets, and phase-guided normalization is the only shift-invariance method compared. The absolute numbers are also low, so the gains are measured against a weak setup.
- The scope is very limited for ICLR: one source dataset, one small Transformer at 100 Hz, single-label tasks built by keeping one code per record, and random record-level splits on the targets. Beyond ECG, CTI is compared only with plain cross-entropy without augmentation, which is exactly the confound the ECG part controls with CE-2V. The idea itself is generic, a shift augmentation used as a contrastive positive pair next to cross-entropy, so for ICLR it would need standard time-series benchmarks with strong baselines. As it stands, it fits way better IEEE JBHI or MLHC.
- The method is very limited: cross-entropy on two circularly shifted views plus a SimCLR-style InfoNCE between them, with no change to the model. Shift augmentation for ECG and supervised-plus-contrastive objectives are both standard, so the contribution on the method side is small imo.
- the paper's own ablations point away from time invariance. Adding the contrastive term with shared offsets, i.e., with no temporal difference between the two views, already gives about half of CTI's gain over CE-2V on Chapman and CPSC2018. Pulling the shifted views together alone hurts transfer, the gain is flat across shift ranges, and CTI's prediction flip rate is slightly higher than CE-2V's, while JS-Consistency is much more stable and transfers wors. This is what one would expect if the gain comes from instance discrimination, with other recordings as negatives even within a class, which keeps the features from collapsing onto the five source classes, a known effect. The paper can't both call this time-invariant learning and show that the time-invariance part alone does not help. Am I missing something?
- The control that would settle this is missing: the same objective with non-temporal augmentations (noise, amplitude scaling, lead masking) at matched strength. Also, CTI's flip rate is 13.51% in Section 4.4 and 11.54% in Table 11. Maybe I am missing something, could you please clarify which checkpoint each number comes from?
Questions for authors
My main concerns are lack of awareness of related work, novelty on the method, and narrow scope. I pointed out at everything in detail above, so I copy paste here.
- The related work is very limited and many key works are missing. On ECG self-supervision it discusses three methods, but not patient contrastive learning, the systematic self-supervision and augmentation studies on 12-lead ECG, the work showing that contrastive learning is not optimal for quasi-periodic signals, or the recent ECG foundation models (ECGFounder, ECG-FM, HuBERT-ECG, drnetMamba, MERL, HeartLang, ECGJepa,…). On view design for time-series contrastive learning it misses several ICLR papers. On shift invariance it misses architectural solutions. The JS-Consistency baseline is essentially the AugMix consistency loss, which is not cited. Most importantly, the work on why instance discrimination transfers better than supervised pretraining, and why better supervised losses give less transferable features, is missing, and it is directly relevant to the results (see 2).
- There is a major lack of comparison to state-of-the-art methods, of which there are many. The ECG-specific methods only appear as deltas after adding CTI, never as standalone rows in Table 1, so we cannot see whether CTI beats them. TS-TCC and TS2Vec are cited but not run, the other time-series contrastive methods above are absent, no released ECG foundation model is evaluated as a frozen encoder on the same targets, and phase-guided normalization is the only shift-invariance method compared. The absolute numbers are also low, so the gains are measured against a weak setup.
- The scope is very limited for ICLR: one source dataset, one small Transformer at 100 Hz, single-label tasks built by keeping one code per record, and random record-level splits on the targets. Beyond ECG, CTI is compared only with plain cross-entropy without augmentation, which is exactly the confound the ECG part controls with CE-2V. The idea itself is generic, a shift augmentation used as a contrastive positive pair next to cross-entropy, so for ICLR it would need standard time-series benchmarks with strong baselines. As it stands, it fits way better IEEE JBHI or MLHC.
- The method is very limited: cross-entropy on two circularly shifted views plus a SimCLR-style InfoNCE between them, with no change to the model. Shift augmentation for ECG and supervised-plus-contrastive objectives are both standard, so the contribution on the method side is small imo.
- the paper's own ablations point away from time invariance. Adding the contrastive term with shared offsets, i.e., with no temporal difference between the two views, already gives about half of CTI's gain over CE-2V on Chapman and CPSC2018. Pulling the shifted views together alone hurts transfer, the gain is flat across shift ranges, and CTI's prediction flip rate is slightly higher than CE-2V's, while JS-Consistency is much more stable and transfers wors. This is what one would expect if the gain comes from instance discrimination, with other recordings as negatives even within a class, which keeps the features from collapsing onto the five source classes, a known effect. The paper can't both call this time-invariant learning and show that the time-invariance part alone does not help. Am I missing something?
- The control that would settle this is missing: the same objective with non-temporal augmentations (noise, amplitude scaling, lead masking) at matched strength. Also, CTI's flip rate is 13.51% in Section 4.4 and 11.54% in Table 11. Maybe I am missing something, could you please clarify which checkpoint each number comes from?