acceptodds
Under review as a conference paper at ICLR 2027

Large Language Models as Tutors of Clinical Semantics for Learning Cardiac Representations

Abstract

ECG–language pretraining derives its primary advantage over signal-only self-supervised learning (SSL) by incorporating clinical reports, thereby grounding ECG representation learning in explicit clinical semantics. However, existing methods typically treat text as an auxiliary modality jointly learned with ECG signals, pursuing performance gains through increasingly elaborate cross-modal alignment mechanisms. This paper is driven by a fundamental yet largely overlooked question: does the quality of text representations itself constitute a bottleneck for ECG representation learning? To address this, we fundamentally redesign the ECG–language framework. Instead of relying on conventional text encoders, we employ a frozen large language model (LLM) to directly encode clinical reports, leveraging its extensive knowledge priors to generate superior text representations. Furthermore, we introduce targeted architectural and optimization mechanisms that enable effective ECG representation learning under these fixed textual supervisory signals. Extensive experiments across three widely used public datasets demonstrate that our proposed method outperforms existing approaches, despite relying solely on a simple global alignment mechanism.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
ECG representation learning, ECG-language pretraining, large language models, clinical reports, cross-domain transfer

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_pSUTOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper revisits the ECG–language pretraining paradigm and asks whether the quality of text representations themselves — rather than the sophistication of cross-modal alignment mechanisms — constitutes a bottleneck for ECG representation learning. The authors propose to replace conventional jointly trained text encoders with a frozen large language model (Meta Llama 3.1 8B, with Qwen2-7B also compared) that directly encodes clinical reports, using its representations as fixed supervisory signals for ECG representation learning via a simple global contrastive alignment. To make this work, they introduce three targeted mechanisms: (1) Text-Space Rectification (TSR), which removes the top-4 dominant common components of the LLM text space to mitigate anisotropy; (2) a scale-aware ECG encoder that uses a lightweight FiLM branch conditioned on pre-standardization lead-level statistics to recover scale cues lost by per-lead standardization; and (3) a memory-efficient negative expansion strategy combining offline text caching/deduplication into a text bank with an online ECG feature queue to support out-of-batch negatives. The method is pretrained on MIMIC-IV-ECG (~761k ECG–report pairs) and validated on three public datasets (PTB-XL, CSN, CPSC2018) across linear probing (1%/10%/100% labels), cross-domain transfer, and zero-shot evaluation against 18 baselines (e.g., MERL, MELP, ECG2TOK, CLEAR). The authors report state-of-the-art results — best in 17 of 18 linear probing settings (average AUROC 88.49%, +2.71 over the runner-up MELP) and +2.55/+3.20 over MELP/MERL in zero-shot — and attribute these gains to the quality of the frozen LLM text supervision, with ablations showing TSR is the most critical component (−6.30 AUROC when removed), followed by FiLM (−0.47) and out-of-batch negatives.

Critical strengths
  1. A well-motivated reframing of the ECG–language bottleneck, validated by unusually clean gains. The paper shifts the community's attention from ever more elaborate alignment mechanisms to text representation quality, and the empirical evidence supports the claim: the method outperforms 18 baselines across 18 linear probing settings, cross-domain transfer, and zero-shot evaluation, despite using only a simple global alignment loss. The strong low-data results (e.g., 91.28% on PTB-XL-Super with 10% labels, beating all baselines at 100%) directly demonstrate the value of high-quality textual supervision, which is the paper's central thesis. This reframing is consequential because it offers the field a new, orthogonal axis of improvement rather than incremental alignment-design gains.
  2. The TSR analysis is thorough and convincingly explained. Beyond reporting a large performance drop when TSR is removed, the authors provide converging evidence — training loss curves (plateau ~6 without TSR vs. ~3 with TSR), mean pairwise cosine similarity, and the fraction of variance explained by top principal components — that the mechanism works precisely as motivated, by suppressing anisotropic dominant common components in the frozen LLM space. This diagnostic rigor makes the framework's behavior interpretable and gives the paper a solid engineering-science foundation.
Critical weaknesses
  1. Novelty relative to prior frozen-LLM-for-ECG work is not established, which is a core claim of the paper. The paper positions itself against "text-as-auxiliary-modality" methods (MERL and successors) but does not adequately differentiate from Li et al., "Frozen Language Model Helps ECG Zero-Shot Learning" (MIDL 2024), which already uses a frozen LLM's text representations to supervise ECG representation learning. Since the paper's principal contribution — using a frozen LLM as a fixed text encoder for ECG learning — overlaps substantially with this prior work, the paper must explicitly delineate the technical distinction (TSR, scale-aware encoding, negative expansion notwithstanding, what fundamental problem does it solve that the prior work does not?), provide head-to-head empirical comparisons, and benchmark the improvement. Without this clarification, the novelty claim is currently unsubstantiated, which is the main factor holding down my score.
  2. Ablations are reported under a non-optimal training configuration and at a single low-data setting, weakening the reliability of the component-level conclusions. The authors state that subsequent component ablations use the 1:1 loss-weight setting as baseline, even though their own hyperparameter analysis shows the model's optimum is at 20:1. This means the reported component effects (TSR: −6.30, FiLM: −0.47) are conditioned on a suboptimal configuration, and their rankings/magnitudes may not hold at the operating point the final model actually uses. Additionally, all ablations are run only at 1% labeled data. Combined with the statically fixed TSR hyperparameter k=4 and the absence of trainable-LLM comparisons, the experimental evidence does not fully support the generality of the design decisions. These gaps are consequential because the paper's value proposition rests heavily on its proposed mechanisms; if their benefits are configuration- or data-regime-specific, the engineering contribution is less transferable than claimed.
Questions for authors

How does the proposed framework fundamentally differ from "Frozen Language Model Helps ECG Zero-Shot Learning" (Li et al., MIDL 2024), which also uses a frozen LLM to provide text supervision for ECG learning? Could you provide a head-to-head comparison against this method under your evaluation protocol, and clearly articulate what specific, unresolved problems (beyond the TSR/FiLM/negative-expansion mechanisms) your work addresses that it does not?

Would the TSR and FiLM ablation conclusions remain consistent under the optimal 20:1 loss-weight setting and at 100% training data, rather than only at 1% data with the 1:1 setting? Given that TSR shows a very large effect, confirming it holds under the actual operating configuration would substantially strengthen the paper.

How sensitive is the method to the number of removed common components k in TSR? A sweep over k (e.g., k ∈ {1, 2, 4, 8, 16}) with performance reported would clarify whether the k=4 choice is dataset/LLM-dependent or robustly transferable.

Reviewer_d2XtOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates whether better text representations can improve ECG representation learning. Instead of focusing on cross-modal alignment methods, the authors focus on text representation and use a frozen LLM to directly encode clinical reports. The framework includes dual-branch encoders, i.e., LLM-based text encoder and scale-aware ECG encoder, and text-space rectification and memory-efficient contrastive learning are used for cross-modal alignment. Extensive experiments demonstrate strong performance across linear probing, cross-domain transfer, and zero-shot evaluation.

Critical strengths

Exceptional presentation and storytelling: I find the paper very well written and structured. Motivation, related work, methodology, and experiments follow a coherent narrative. Each module has a clear motivation, and the experimental results are followed by explanations and discussions rather than just reporting numbers. This makes the whole paper easy to follow and convincing.

Strong and comprehensive empirical evaluation: The method achieves consistently competitive results across multiple datasets, label-efficiency settings, cross-domain transfer, and zero-shot evaluation. The ablation studies and representation analyses, particularly the cosine similarity and PCA analyses of text-space rectification, provide useful evidence supporting the design choices.

Critical weaknesses

Limited direct validation of the central hypothesis: I think the most interesting part of this paper is the question it raises: whether text representation quality is a bottleneck for ECG representation learning. While the experiments clearly show that the proposed framework works well, I am not fully convinced that they demonstrate the role of text representation quality itself.

Questions for authors

Can the authors provide more direct evidence that text representation quality is a bottleneck for ECG representation learning? In particular, I would recommend to show controlled ablation in the main text: comparing different text encoders, such as conventional or domain-specific encoders, while keeping the rest of the framework unchanged. This would help clarify how much of the improvement actually comes from better text representations.

Reviewer_TJGkOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3(+1)
Read review
Paper summary

The paper argues that the text side is the main bottleneck in ECG-language pretraining. It replaces the jointly trained text encoder of prior methods with a frozen LLM (Llama-3.1-8B, mean-pooled penultimate-layer states) and adds three components: text-space rectification (TSR), which removes the mean and the top four principal directions of the highly anisotropic LLM embeddings; a FiLM branch conditioned on per-lead log standard deviations, which restores amplitude information lost to per-lead normalization; and out-of-batch negatives drawn from a deduplicated report bank and an ECG queue, made possible by caching text embeddings offline. Training uses only a global contrastive loss, with the ECG-to-text direction weighted 20:1, for one epoch on MIMIC-IV-ECG. The method reaches an average linear-probing AUROC of 88.49 over 18 settings, 2.71 points above MELP. It is best in 17 of these settings and in 5 of 6 cross-domain settings, and it also improves average zero-shot AUROC. In the supplement, replacing Llama with frozen biomedical BERT-style encoders while keeping everything else fixed lowers the average by 3.3 to 3.7 points.

Critical strengths
  1. The controlled text-encoder comparison in the supplement (Supp. Tables 7 and 8) gives direct support to the paper's thesis. Only the frozen text encoder is changed, and Llama outperforms frozen MedCPT, BioClinicalBERT and PubMedBERT by 3.3 to 3.7 points on average, with larger gaps at 1% labels. With frozen MedCPT the framework averages 85.21, close to MELP's 85.78. Most of the improvement over prior ECG-language methods therefore appears to come from the text representation rather than from the added modules or training details.

  2. The gains are consistent and clearly larger than run-to-run noise. The method is best in 17 of 18 linear-probing settings and in 5 of 6 cross-domain settings, with large margins in the low-label regime (4.5 points on PTBXL-Form 1% and 5.6 points on CSN 10% over the best baseline), whereas standard deviations across pretraining seeds are mostly between 0.1 and 0.6 (Supp. Table 9). All of this comes from a single global contrastive objective and one epoch of pretraining on one A40. A recipe this simple is likely to be picked up by others, which adds to its value.

Critical weaknesses
  1. It is not yet established that the gains come from the clinical knowledge of the LLM. This does not undermine the main empirical result, but it limits the stronger interpretation in the title and introduction. Fig. 1 frames the contribution as frozen-LLM supervision versus joint ECG-text learning, yet the paper never compares, within its own framework, against a trainable or fine-tuned text encoder (as in MERL) or against a domain-adapted one such as MELP's cardiology language model. Supp. Table 7 includes only frozen BERT-style encoders, whose mean-pooled sentence embeddings are known to be weak, and these runs reuse k = 4, which was chosen for Llama. What the table shows is that a frozen 8B LLM beats frozen BERT, not that LLM knowledge is the key factor. A simpler explanation should also be ruled out. As far as I know, MIMIC-IV-ECG reports are largely machine-generated statements drawn from a limited vocabulary. After deduplication, a 20:1-weighted ECG-to-text loss over report-bank negatives behaves much like classification over a fixed set of report prototypes, and the benefit may come from a better similarity structure among these prototypes rather than from clinical knowledge. The number of unique reports is not given, and Llama-8B and Qwen2-7B perform almost identically (88.49 vs. 88.45), so there is no sign yet that performance tracks the capacity or knowledge of the language model.

  2. Novelty and positioning are overstated, and contributions 2 and 3 are not well supported by the ablations. The introduction describes the question as fundamental yet largely overlooked, but METS (MIDL 2023) and ETP (ICASSP 2024) already align ECGs with reports through a frozen language model, and neither is cited or compared. The added components are also known: TSR is All-but-the-top (Mu and Viswanath, 2018), the ECG queue follows MoCo, and FiLM comes from Perez et al. (2018), which is not cited either. Combining known techniques is perfectly acceptable, but it means the paper stands or falls on its empirical insight, which makes the first weakness more important. The measured effects of the new modules are small. FiLM gives 81.63 ± 0.29 against 81.17 ± 0.35, about 1.5 standard deviations apart, and the single-seed sweep over out-of-batch negatives is non-monotone, with a total range of roughly 2 standard deviations. By contrast, the 20:1 loss weighting adds about one point (from 81.63 to 82.59), more than FiLM and the extra negatives combined, yet it is treated only as a hyperparameter. I suggest toning down contributions 2 and 3 and discussing the relation to METS and ETP.

Questions for authors
  1. Were the hyperparameters in Fig. 4 and Supp. Tables 11 and 12, as well as k = 4, selected on validation or test AUROC? Was k re-tuned for the BERT-style encoders in Supp. Table 7? If test AUROC was used, please report results with validation-based selection.

  2. Could you report, within the same framework, results with (a) a trainable or fine-tuned MedCPT text encoder, (b) MELP's frozen cardiology language model, and (c) a multi-hot statement-label objective? These comparisons test the paper's thesis most directly.

2 comments

Thank you for the rebuttal, The authors have addressed my major concerns. The response is sufficient to resolve my earlier doubts, and I will rase the soundness score to 3.

We appreciate your positive assessment of our rebuttal and your decision to raise the soundness score to 3. We are sincerely grateful for your rigorous and thoughtful review, as well as your valuable suggestions. With our best regards.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.