- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
LEAP-LLM is an ECG-LLM interface for question-conditioned interpretation. The authors argue that existing models encode the ECG independently of the clinical question, forcing representations to retain broadly irrelevant information, and that cross-lead aggregation obscures lead-specific detail a given question needs. Instead of encoding first and fusing later, the clinical question guides ECG feature extraction while explicit lead and temporal structure is preserved: task-trained instruction representations distill the question into queries that drive two complementary branches — a Lead branch capturing question-relevant temporal patterns within each lead, and a Temporal branch summarizing information across leads at each temporal position. The resulting compact, question-conditioned tokens are fed to the LLM. Validation covers four ECG benchmarks, token-count and prefill-FLOP reduction, and an analysis of lead-importance patterns.
Critical strengths
- The efficiency result is substantial and practically meaningful. The reported reduction in modality tokens (89.0%) and prefill FLOPs (7.2×) addresses a real deployment bottleneck for ECG-LLMs, where long signal token sequences dominate inference cost. This is the kind of gain that matters independently of accuracy, and it is the main reason I view the paper as above clear rejection.
- The two-branch decomposition maps cleanly onto clinical reading practice. Separating within-lead temporal evidence from cross-lead evidence at each time step is a natural factorization, and the reported anatomical selectivity of lead attention (inferior MI attending to II, III, aVF) is a reassuring sanity check that the interface learned something structurally meaningful rather than a shortcut.
Critical weaknesses
- The efficiency claim is established only in the single-question regime, which is not the realistic use case. Because feature extraction is conditioned on the question, every new question about the same recording requires re-extraction; the 89% token reduction and 7.2× prefill gain therefore describe per-question cost, not per-recording cost. In multi-turn consultation or multi-question review of one trace — the setting that actually motivates an ECG-LLM — the advantage shrinks and may reverse relative to a question-agnostic encoder that extracts once and reuses. The paper does not quantify this anywhere, so a headline contribution rests on an unstated assumption. This is the weakness I would most want addressed, as it determines whether the claimed gain survives in practice.
- The paper is not positioned against the prior work it most directly continues, making the novelty hard to judge. Its motivating claim — that cross-lead aggregation loses the localized, lead-specific detail a question needs — is precisely what fine-grained waveform-level ECG-text alignment was developed to preserve, and the ECG-language alignment stage sits on top of a line of text-supervised ECG pretraining using LLM-enhanced clinical descriptions (e.g., ESI, Yu et al.,; FG-CLEP, Li et al). Generalist ECG-MLLMs supporting flexible input and waveform localization are also closely related to this design. GEM and PULSE are discussed, but neither strand is, so it remains unclear whether question-conditioned selection is necessary on top of what these methods already retain.
Questions for authors
- What is the total cost when K questions are asked about the same recording? Could you report prefill FLOPs and latency as a function of K, and identify the K at which the advantage over a question-agnostic encoder disappears?
- How does LEAP-LLM compare to generalist ECG-MLLMs that already support flexible input and waveform localization with multi-task understanding?
- Is question-conditioned extraction necessary, or does fine-grained waveform-level alignment already retain the lead-local detail you argue is lost? Are there questions your method answers that alignment-based representations cannot?
- The lead-importance analysis is correlational — attention indicates where information is concentrated, not causal importance. Could you provide an intervention study (e.g., masking or replacing the identified leads) to establish a causal link to predictions?
6 comments
Question 2/3 (Weakness 2): Comparison with other ECG methods and benefit of question conditioning
Comparision with General ECG-MLLM
We have compared LEAP-LLM with GEM, PULSE and ECG-R1 on downstream performance and inference efficiency (Tables 1–2 Main Manuscript). We also identified anyECG-chat [1] as closely related work that extends ECG-MLLMs to flexible inputs and waveform localization. Compared with these broader capabilities, LEAP-LLM focuses on compact, question-dependent representations for 12-lead ECG interpretation. Both LEAP-LLM and anyECG-chat incorporate lead and temporal information, but anyECG-chat’s MLP connector produces question-independent ECG features, whereas LEAP-LLM conditions extraction on the question. Flexible input handling is also compatible with our approach, variable-duration and reduced-lead recordings could be accommodated through segmentation, padding and appropriate masking. Our controlled ablation supports the benefit of question conditioning under matched settings. Our extraction module could also be incorporated into such generalist models, provided their encoders expose lead- and time-resolved features, complementing their broader capabilities with adaptive evidence selection.
Comparison with ECG-Text Alignment methods and Importance of Question Conditioning We thank the reviewer for highlighting ESI [1] and FG-CLEP [2]. We have expanded Related Work to clarify their relationship to LEAP-LLM. Our experiments support the benefit of question-conditioned extraction under a fixed token budget.
ESI learns ECG representations through contrastive and captioning objectives using ECGs paired with LLM-generated descriptions [2]. FG-CLEP aligns ECG segments with individual report tags through tag-specific representations and uses LLMs to supplement waveform information missing from reports [3]. We agree that such representations can retain local waveform detail. However, LEAP-LLM addresses the subsequent step, selecting and aggregating that information before it reaches the LLM. In particular, FG-CLEP uses report tags to guide alignment, whereas LEAP-LLM uses the question at inference time to determine which ECG evidence to emphasize.
This distinction matters when constructing a compact ECG representation. Passing all (N x L) lead-time features directly to the LLM would increase its input length and computational cost, while including information irrelevant to the current question. Compressing these features reduces this cost but requires deciding which information to retain. LEAP-LLM uses the question to guide this compression, producing (N+L) tokens that emphasize relevant evidence while retaining separate lead-specific and time-indexed representations.
Our controlled ablation directly tests whether this conditioning helps. With the encoder, LLM backbone, lead-time structure and training setup held fixed, adding question conditioning improves performance by 3.1 percentage points on CSN and 3.8 points on G12EC (Table R3, rows b–c, in our response to Reviewer zX9Y). The question-swap analysis further shows that attention follows the queried anatomical territory, while lead-token interventions demonstrate selective dependence on anatomically relevant lead representations.
A sufficiently informative question-agnostic representation could support the same answers. Our contribution is to make evidence selection adaptive when the interface must compress the ECG into a limited number of tokens. This is particularly relevant to questions requiring localized evidence, as demonstrated by our territory-specific MI analysis. Encoders such as ESI or FG-CLEP could complement this approach: their aligned representations would still require an interface to a generative LLM, where our method could be applied.
(continued in the next comment)
Question 4: Intervention evidence beyond attention
We agree that attention alone does not establish causal importance. Our lead-token intervention study show that the model’s answers depend selectively on anatomically relevant lead representations by testing whether changing the selected lead evidence affects the model's answer. We show that replacing leads associated with the queried MI territory reduces gold-answer likelihood substantially more than replacing unrelated or randomly selected leads, providing evidence beyond attention visualization.
Evaluation Protocol: We replace selected patient-specific lead tokens with the mean representation of the same leads in a reference cohort, immediately before they enter the LLM, and measure the reduction in gold-answer likelihood.
Table R7. Reduction in gold-answer likelihood (%) after replacing lead tokens
| Replaced lead tokens | Inferior MI | Anterior MI |
|---|---|---|
| Inferior (II, III, aVF) | 50.5 | 5.5 |
| Anterior (V1–V4) | 6.8 | 55.0 |
| Three random leads | 17.2 | 17.6 |
Replacing the matching territory reduces likelihood 9.2× and 8.1× more than replacing the other territory, and about 3× more than replacing three random leads. These interventions show selective functional dependence on anatomically relevant lead representations at the LLM interface, which the attention analysis alone could not establish. This experiment is presented in our manuscript (Section 5.2, with details in Appendix H). We have revised the manuscript to clarify the distinction between attention visualization and intervention-based evidence.
References
[1] Li, Haitao, et al. "anyecg-chat: A generalist ecg-mllm for flexible ecg input and multi-task understanding, 2025a." URL https://arxiv. org/abs/2506.00942.
[2] Yu, Guo and Sano. ECG Semantic Integrator (ESI): A Foundation ECG Model Pretrained with LLM-Enhanced Cardiological Text. TMLR, 2024. https://arxiv.org/abs/2405.19366
[3] Li et al. FG-CLEP: Fine-grained Contrastive Learning for ECG-Report Alignment with Waveform Enhancement. arXiv:2505.11939, 2025. https://arxiv.org/abs/2505.11939
We thank the reviewer for the detailed feedback. We quantify the efficiency trade-off across multiple questions per recording and clarify how question-conditioned extraction complements fine-grained ECG–text alignment, supported by controlled ablation and question-swap analysis. We also clarify how lead-token interventions demonstrate selective dependence on anatomically relevant representations beyond attention visualization.
Question 1 (Weakness 1) : Cost of multiple questions about one recording
We report total prefill FLOPs and latency across multiple questions per recording. LEAP-LLM has the lowest prefill FLOPs through four questions and the lowest latency through eight questions among the evaluated settings, GEM becomes more efficient beyond these respective points as it reuses question-independent ECG tokens.
Evaluation Protocol: For each model, we cache every part of the prompt that is identical across questions about the same recording and recompute only the rest for each new question. For GEM, ECG-R1 and PULSE, the cached part includes the ECG and image tokens, which do not depend on the question. LEAP-LLM's ECG tokens are extracted with the question, so only its system prompt is cached. We vary K∈{1,2,4,8,16,32} and report the total prefill cost of one recording with all K questions. Latency is the median of 10 runs at batch size 1 on one NVIDIA L4. ECG-to-image rendering for GEM and PULSE is not counted, which favours those baselines.
Table R6. Prefill cost of one recording with K questions
Table R6 shows that LEAP-LLM has the lowest prefill FLOPs among the compared models at K=1,2,4 and the lowest latency at K=1,2,4,8. GEM becomes more efficient at larger question counts as it reuses its question-independent ECG representation. LEAP-LLM remains faster than ECG-R1 and PULSE at every evaluated question count, including K=32.
(continued in the next comment)