- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper "Cohort-level control for conditional ECG generation" addresses the proper integration of patient metadata into generative models for ECG data. The authors observed that existing models only weakly followed requested age and sex attributes. In addition, the authors demonstrate that when attributes, such as height or weight, are frequently missing in the training dataset, generated samples with missing attributes out of this category are biased towards the training set mean with respect to other attributes such as age. The authors fix this problem by means of a hybrid sampling procedure within a conditional flow model and provide experimental evidence that this approach fixed the latter problem.
Critical strengths
- The authors provide clear experimental evidence that both problems (attribute-following and bias towards the mean for missing values) exist and describe the mechanism that leads to the latter.
Critical weaknesses
- Related work: A clear comparison of the proposed approach (apart from the hybrid sampling) in comparison to literature approaches such as DiffuSETS, SE-Diff or SSSD-ECG is missing. This makes it very difficult to see which components might be relevant for performance improvements. Also relevant prior work on conditional ECG generation is missing https://openreview.net/forum?id=iTPbysd9s1
- The authors put forward to problems with conditional ECG generation. They don't propose a specific solution for attribute-following of non-missing attributes such as age and sex, where they use standard guidance. Nevertheless, their experimental results seem to indicate that their approach shows an improved behavior in this respect.
- Architectural details: A big weakness is the almost total lack of architectural details of the proposed model, which remains limited to a few sentences in the appendix. This includes details on the actual diffusion model, the latent encoder, training objectives. No code is provided and the submission is from my perspective therefore not reproducible at all.
- Experimental details: The selected label sets are not documented, neither the subset of rare labels. No preprocessing code is provided, so the statements remain impossible to assess.
- Narrow scope: Conditional ECG generation is already a very narrow domain. The particular aspect of attribute-following in particular related to missing attributes is even narrower. The identified problems exist, see strength #1, but they are so specific that they are only relevant for a very small fraction of the ICLR community.
- Comparison to prior work: "We ran the official SSSD-ECG implementation without its optional CUDA extension, which substantially increases its measured latency." Comparing CUDA to non-CUDA implementation does not yield a like-by-like assessment of the proposed method compared to literature approaches. Similarly, the comparison to DiffuSETS involves a conditional encoder that was trained by the authors (and also remains undescribed). No comparison in strict alignment with literature results is performed which would allow me to assess the prospects of the proposed model.
- Ill-defined comparison: The submission suggest comparability to related work. However, when conditioning on more attributes (e.g. providing demographic attributes in addition to diagnostic labels or providing height and weight in addition to age, sex and diagnostic labels) constrains the samples to be generated more, which might quite naturally lead to better sample quality. From this perspective a comparison between generative models conditioned on different label sets are not reliable.
- Privacy: The authors do not mention nor investigate the privacy aspect of things. A model that copies training set samples would score great under their "train on synthetic, test on real" metric. Improved model performance could be due to similarity to training samples.
Summary
- The scope of the submission is very narrow. Architectural and experimental details are almost completely missing and therefore hardly reproducible. Comparison to prior work is partially ill-defined.
Questions for authors
- How does the overall setup compare to competing diffusion/flow matching approaches on a specific level?
- Why does their model show better attribute-following also for age and sex, where the proposed hybrid sampling is not used?
- What is the authors' mechanistic understanding of the improved attribute-following behavior of their model compared to literature approaches?
- Would the approach be applicable to other problems in order to broaden the impact of the submission?
We thank the reviewer for their time and for the detailed list of points.
Comparison with prior approaches (W1, Q1). The adapted DiffuSETS and SE-Diff share our latent space. Our generator differs from them in three components: a flow-matching objective with a transformer backbone instead of a diffusion UNet; diagnoses and attributes as structured inputs with presence indicators instead of a text embedding with raw age, sex and heart rate; and classifier-free guidance on age and sex, where the baselines sample without guidance. SSSD-ECG is a signal-space diffusion model with structured state-space layers and label conditioning. DCDM-ECG is relevant prior work on conditional ECG generation.
Age and sex (W2, Q2, Q3). Age and sex use standard guidance; contribution (i) is the generator. Its response does not come from the presence flags or factor dropout (age response 0.63 without both; Table 6), nor from the latent space, which the adapted baselines share. The baselines sample without guidance, as in their released code; we consider this the main reason, but have not isolated it.
Reproducibility (W3, W4). Objective and sampler: Sec. 4.1; transformer: Appendix B; parameter counts: Table 8. Settings not stated in the submission: an autoencoder latent of 4 × 128; 20k iterations at batch 128 on PTB-XL and 60k at batch 512 on MIMIC-IV-ECG; AdamW with learning rate 2e-4 and weight decay 1e-5; condition dropout 0.1 and factor dropout 0.15; 4 attention heads and patch size 4. Label sources and counts are in Appendix A and Sec. 5.1, and the rare subsets (training prevalence below 2%) in the captions of Tables 4 and 5; the individual statements are not listed, and we will add them. Code will be released on publication.
Comparison with the literature (W6). Cost is compared in FLOPs, which the CUDA extension does not change (16,710 GFLOPs per sample for SSSD-ECG, against 52.5 for ours). The text encoders are off-the-shelf and frozen, not trained by us. DiffuSETS embeds text through its authors' hosted service, and MIMIC-IV-ECG text may not be sent to an external service under its data use agreement, so the ports use text-embedding-3-small on PTB-XL and a local bge-large-en-v1.5 on MIMIC-IV-ECG. Neither DiffuSETS nor SE-Diff reports TSTR; on heart-rate error, which both report, the ports reach 6.8 and 8.2 bpm on PTB-XL against 6.73 and 8.24 in the original papers. SSSD-ECG enters with its published TSTR of 0.840, against 0.903 for ours.
Condition sets (W7). Without demographics our model reaches TSTR 0.904 (0.901 with them; Appendix E), so the additional conditions do not explain the gap.
Privacy (W8). General response, item 4.
Scope (W5, Q4). Eq. 2 holds for any classifier-free guidance over presence indicators, for example image or EHR-conditioned generation with partly recorded demographics.
Summary. The shift arises when height is supplied for patients whose height was not recorded, and it moves samples towards the recorded population (older patients), not towards the mean (Sec. 1, Sec. 5.4).