- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 3
Read review
Paper summary
The core idea builds on multi-query probing to better read out a frozen encoder and alleviate cross-site failures of pretrained ECG encoders. To better address cross-site transfer at the readout stage, the authors build on multi-query probing and propose an attention-alignment loss that forces each query to attend to a specific window: the P, QRS, ST, or T wave, respectively. The alignment also ensures that each query assigns non-zero attention to every beat in a record through the decomposed placement and allocation terms. The alignment loss is added to the cross-entropy loss while the encoder remains frozen and the classification head and queries are trained for the target task. This assignment is shown to be more effective than unassigned multi-query probing across five cohorts. R-peaks are manually annotated in CPSC2018, which will be released with the manuscript. The expert annotations improve upon the off-the-shelf detector by approximately 5.5 F1 points.
Critical strengths
The work is well grounded in the ECG field. The attention alignment is carefully designed based on a substantial understanding of ECG characteristics. The core idea is intuitive, and it introduces only an additional loss at the readout stage, requiring no R-peak detection at inference.
The experiments and visualizations are well designed. Table 3 shows strong improvements over average-pooling baselines and unassigned probing when training on PTB-XL. The authors conduct a solid ablation of the two components of the alignment loss, registration and allocation. The attention maps show better beat coverage and more concentrated attention from each query on its designated waveform.
Critical weaknesses
Although the core idea is intuitive, the paper is difficult to follow without a substantial ECG background. The contribution appears to be limited to local ECG diagnosis tasks using source-only cross-site transfer with a frozen encoder.
The gains over UP are small on two of the four encoders (only 0.1 mean AUROC) and modest on the other two (1.1 and 1.2), relative to the reported variation. The benefit over unassigned probing is therefore not consistently compelling, especially given that the proposed method fixes the number of queries at four.
BAP uses fixed, R-peak-based phase windows during training, which may impose a rigid prior: waveform timing varies across patients and conditions, while unassigned intervals and R-peak errors can introduce noisy supervision.
In Table 2, most baselines use average pooling, and no strong contextual attentive readout is included. ABMIL assigns each token an independent score and is therefore relatively weak. The authors should consider stronger baselines, such as TransMIL.
Questions for authors
- When referring to average pooling, do you mean averaging the tokens, or using the [CLS] token? [CLS] may provide a stronger summary than mean pooling. Please clarify this choice and, if applicable, compare against it.
- Could you provide ablations with (K<4) and (K>4)? Does the improvement persist as the number of queries varies? This would clarify the upper bound of UP.
- Table 2 did not consider a stronger attentive readout baseline, e.g., TransMIL? ABMIL assigns each token an independent score and does not explicitly model token–to–token interactions in the readout.
- What are the training costs and additional assumptions of BAP—R-peak detection, fixed phase-window construction, and the alignment loss—relative to standard domain-adaptation methods? Most domain-adaptation methods use target-site data to update the encoder. Under a frozen-encoder setting, data augmentation and a contextual attentive readout are simple alternatives. What are the trade-offs between these strategies and transfer performance?
6 comments
Thank you for the response
The authors have responded well to the questions I have raised. The comparison with TransMIL is useful evidence for the method. The ablations in general are rigorous. My view of the paper is more positive now after the rebuttal.
Thank you very much for your kind feedback, and we are glad that you found the comparison with TransMIL useful and the ablations rigorous. With the discussion closing very soon, we would be sincerely thankful if you could find a moment to look at our latest results, which extend this comparison to clinical outcomes.
Beyond local diagnoses. Regarding your comment that the contribution appeared limited to local diagnoses, we evaluated BAP on all-cause mortality, trained on CODE-15% and tested on its held-out patients and on the external SaMi-Trop and MIMIC-IV cohorts, and on the 12 structural heart disease labels of EchoNext, including new patients of 2021 and 2022. BAP achieves the highest mean AUROC and AUPRC of the five readouts, TransMIL included, on all five test sets. On MIMIC-IV, an external cohort from another country and health system, it exceeds TransMIL by 1.7 AUROC points [1.3, 2.1] and UP by 0.5 [0.1, 0.8]. On the new EchoNext patients, it exceeds UP by 0.8 macro-AUROC points [0.2, 1.5]. Reviewer f1sW, who suggested these tasks, commented that this "application to truly consequential outcomes significantly improves the paper".
Detector-based supervision. The alignment targets of BAP in these experiments use NeuroKit2 R-peaks alone, the detector-based supervision discussed in your third comment.
These results are now in Table 5 and App. F of the revised paper, and the introduction includes the ECG primer promised in our first response.
If our responses resolve your concerns, we would be deeply grateful if you could kindly consider raising your score. Your assessment means a great deal to us, and we are ready to answer any remaining question right away. Thank you again sincerely for your time and effort.
Best regards,
The Authors
Thank you for your constructive comments, and the revised version has been uploaded.
Thank you again for your constructive comments. We have uploaded the revised paper, with all changes marked in blue.
To make the paper easier to follow without an ECG background, the introduction now includes a short ECG primer, and Sec. 2 defines local diagnoses and the scope of BAP. TransMIL and a two-layer Transformer have been added to Tab. 14 (App. E.6) as contextual readouts, and BAP trained with noisy R-peaks is now reported in Tab. 9 (App. E.1).
As the discussion period is ending soon, we would greatly appreciate it if you could kindly take a look at our responses and let us know whether they address your concerns. We are happy to provide any further clarification.