- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes an ECG world model for predicting drug-induced changes in ECG waveforms. It incorporates physiological ODE priors into a latent diffusion model through energy regularization. A frozen ECG foundation model scores the generated ECGs, and multiple stochastic samples are used to estimate the mean and variability of diagnostic risk scores for comparing candidate interventions. Evaluations on controlled drug-response datasets and real-world clinical records report improvements in waveform prediction, preservation of clinically relevant ECG intervals, and prediction of drug-induced changes in diagnostic scores.
Critical strengths
Broad experimental coverage. The authors evaluated the method using both controlled drug-response datasets and real-world clinical data, complemented by cross-dataset testing, ablation studies, and robustness analyses. The evaluation encompassed waveform errors, ECG intervals, and changes in diagnostic scores, providing empirical support for the method's effectiveness from multiple perspectives.
Critical weaknesses
(1). Insufficient validation of risk scores and uncertainty components. The authors construct a risk proxy by averaging probabilities across multiple heterogeneous diagnostic labels but fail to adequately justify the metric's applicability when comparing different medications. Furthermore, they do not directly demonstrate that incorporating standard deviation—as opposed to relying solely on the mean—leads to superior intervention selection. The experimental results regarding "diagnostic changes" primarily reflect the fidelity of the generation process rather than the effectiveness of the ranking module; this undermines the persuasiveness of the paper's core contribution: the comparative evaluation of interventions.
(2). The applicability of physiological priors to complex heart rhythms remains unclear. As the authors acknowledge in the discussion of limitations, the framework's reliance on classical ODE priors may struggle to accommodate severely irregular rhythms, such as ventricular fibrillation. The paper does not sufficiently explain how the generated ECG and subsequent diagnostic scores are affected when the prior is mismatched. Given that physiological constraints are central to the method's design, this uncertainty limits my ability to assess its reliability in complex pathological scenarios.
Questions for authors
-
Ventricular fibrillation is a clinically significant factor when evaluating pharmacological interventions. The paper acknowledges the limitations of its ODE prior in such scenarios. Could the authors explain how the model handles these situations and discuss the impact on the generated ECGs and subsequent risk scores?
-
The paper states that the framework supports the representation of continuous dosages, yet discrete intervention tokens were used in the experiments. What experimental evidence supports the model's ability to handle continuous dosages?
-
Could the authors clarify the specific meaning of the values in the predictive distribution and explain their relationship to clinical risk?
4 comments
- In a separate controlled drug-response evaluation, full EPK reduced MAE for all six ECG interval endpoints. The heart rate used as a conditioning input at inference is extracted directly from the original pre-dose ECG recorded at the corresponding visit. Measured post-dose heart rate is not used as a model input.
| Lead and interval | Full EPK MAE (ms) ↓ | No EPK MAE (ms) ↓ |
|---|---|---|
| Lead II QTc | 27.822 | 31.793 |
| Lead II PR | 16.707 | 17.945 |
| Lead II TpTe | 10.415 | 10.662 |
| Lead aVF QTc | 27.902 | 31.125 |
| Lead aVF PR | 14.705 | 16.541 |
| Lead aVF TpTe | 10.140 | 11.223 |
Q2. Continuous dosing and the evaluated action space
- We evaluated 8 fixed drug regimens comprising 4 monotherapies, 3 combination therapies, and placebo. Each regimen was encoded as a separate discrete action (Appendix E.7). We have not evaluated prediction at continuous dose levels, so our empirical conclusions are limited to these fixed regimens.
Thank you for the response and additional experiments, which partially address my concerns. As the authors acknowledge, dispersion across generated ECGs should not be interpreted as calibrated clinical uncertainty. The current results support the method as a drug-conditioned ECG generative model. However, I remain unconvinced that it can reliably compare the clinical consequences of alternative interventions, a capability central to the paper’s broader “world model” claim. I will therefore maintain my current score.
We thank the reviewer for the constructive follow-up and for acknowledging that our results support ECG-WM as a drug-conditioned ECG generative model.
We would like to clarify that the primary objective of ECG-WM is to learn action-conditioned cardiac dynamics and simulate ECG responses under different pharmacological interventions. In this context, the term "world model" refers to a learned action-conditioned transition model that predicts future physiological states given the current ECG and a specified intervention. This modelling objective is distinct from developing a clinically validated treatment-selection system.
Beyond ECG generation, our randomised crossover evaluation provides quantitative evidence for comparing drug-associated electrophysiological responses across interventions. Specifically, the model demonstrates agreement with observed within-subject QTc response rankings across treatment periods, and incorporating predictive uncertainty improves selected ranking metrics. We believe these results provide meaningful initial evidence for intervention-aware simulation, while recognising that QTc-based comparisons do not capture the full range of clinical consequences associated with treatment decisions.
To make this distinction explicit, we will clarify in the revised manuscript's Limitations section that the current evaluation focuses on electrophysiological responses, and that comprehensive clinical treatment-selection applications require broader outcome assessment and further clinical validation. This clarification defines the intended scope of ECG-WM without diminishing its demonstrated contributions to physiology-guided, drug-conditioned ECG simulation and intervention-response modelling.
We appreciate the reviewer's recognition of our generative modelling results and hope this clarification helps distinguish the methodological contribution of ECG-WM from its potential future clinical applications.
W1 and Q3. Interpretation and validation of the predictive score distribution
We compare generated and observed ECGs using the mean probability across the frozen classifier's 17 labels. These labels cover sinus rhythm, pathological findings, and signal quality, so a higher mean has no uniform interpretation as clinical risk. For a given input and regimen, the mean and standard deviation of this diagnostic summary across generated ECGs describe its predictive distribution. Across 200 held-out test samples, predicted and observed changes in the summary had a Pearson correlation of 0.620, a Spearman correlation of 0.598, and 75.5% directional agreement.
We agree that dispersion across generated ECGs should not be interpreted as calibrated clinical uncertainty. Our additional evaluation reveals substantial undercoverage of nominal 90% prediction intervals, with mixed CRPS results across endpoints. We therefore restrict our conclusions to empirical predictive-distribution assessment and exploratory uncertainty-aware QTc ranking, rather than calibrated clinical risk estimation.
Full EPK increased coverage for latent-variable prediction intervals and yielded a higher CRPS. At 90% nominal coverage, empirical coverage increased from 69.66% to 80.09%, and CRPS increased from 0.097785 to 0.103025.
We compared rankings based on the predictive mean alone and on the mean plus a weighted standard deviation. The cost measures positive QTc prolongation from baseline.
λ weights the standard deviation of the predicted cost. At λ=0, the ranking uses the predictive mean alone. ΔQTc is the change from the pre-dose baseline, and ΔΔQTc subtracts the corresponding change under matched placebo. Regret is the difference in observed cost between the model's top-ranked regimen and the best regimen according to the observations. Top-1 measures agreement on the best regimen, and Kendall τb measures agreement across the full ranking.
For lead II ΔQTc without placebo adjustment, λ=1 gave the best results across all three ranking metrics. Relative to the mean-only ranking, regret fell from 10.300 to 8.812 ms and Top-1 agreement increased from 55.62% to 58.81%.
W2 and Q1. Irregular rhythms and prior mismatch
The MIMIC test set includes ECGs with irregular rhythms. We report MSE over the entire test set. The synthetic jitter and latent-space Gaussian noise experiments in Appendix B.9 measure the dispersion of generated samples under perturbations. Waveform prediction accuracy and the reliability of diagnostic outputs in ventricular fibrillation remain to be validated. Accordingly, we explicitly scope our clinical claims to the evaluated sinus and non-arrhythmic/mildly irregular regimes, and treat extreme arrhythmia settings such as ventricular fibrillation as an out-of-distribution regime requiring dedicated validation before clinical deployment.
Full EPK reduced the dispersion of generated samples under synthetic perturbations. Figure 9 in Appendix B.9 compares full EPK, EPK at one-quarter strength, and EPK disabled. Under strong jitter, full EPK reduced the mean pairwise ℓ₂ distance from approximately 6.9 to 5.7 and the latent standard deviation from approximately 0.21 to 0.17 relative to disabling EPK. With Gaussian noise in latent space, the corresponding reductions were approximately 6.6 to 5.1 and 0.20 to 0.15. The weaker prior produced intermediate results.