- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper introduces MARC for predicting whether heart failure (HF) is recorded in a patient’s final hospital visit based on diagnosis codes from prior visits. Built on the TCEM graph-temporal backbone, MARC introduces three components: (H) CCS hierarchy-guided code embedding via residual attention, (E) patient-state-conditioned edge gating on a co-occurrence graph, and (R) a batch-level similarity regularizer (training-only). Evaluated on MIMIC-III (7,493 patients) and a MIMIC-IV subset (10,000 patients) with fixed 1,000-patient test sets across 5 random seeds, MARC achieves marginal gains in AUROC/AUPRC over seven baselines, supported by a full factorial ablation.
Critical strengths
- Systematic Factorial Ablation: The full module ablation (Tables 2–3) is exceptionally thorough, with transparent reporting that module contributions are non-additive and dataset-dependent.
Critical weaknesses
- Flawed Task Definition & Lack of Clinical Relevance: The endpoint predicts prevalence/recording rather than incident HF. Patients with prior HF diagnoses are highly likely to have HF re-recorded. Without reporting prior HF code prevalence or benchmarking against a trivial "copy-forward" rule (predict positive if prior HF exists), it is unclear if models are merely detecting historical codes. Incident HF prediction—the clinically meaningful setup—is not evaluated.
- Statistically Unsound Claims: Performance gains over the top baseline are marginal (0.64 AUROC on MIMIC-III, 0.73 on MIMIC-IV). Because evaluation relies on a single fixed split (seeds only vary training noise, not test sampling uncertainty), the estimated CI half-width ( AUROC points via Hanley–McNeil) vastly exceeds the reported improvements. No cross-validation, CIs, or hypothesis tests (e.g., DeLong, paired bootstrap) are provided. Additionally, on MIMIC-IV, matches the full model (), rendering redundant.
- Outdated & Incomplete Baselines:
- Omits the Chet benchmark protocol (Lu et al., AAAI 2022) despite identical cohort and task setups.
- Fails to compare against directly cited graph-temporal and knowledge-enhanced SOTA baselines, including GT-BEHRT (ICLR 2024) and KARE (ICLR 2025). Most compared baselines date back to 2016–2021.
Questions for authors
- Incident HF & Baseline Rule:
- What percentage of test positives/negatives have HF codes in prior visits?
- What are the AUROC/AUPRC scores of a simple "copy-forward" rule?
- Can you evaluate MARC and baselines strictly on the incident HF cohort (patients with no prior HF history)? (Sustained performance here would significantly improve my evaluation.)
- Statistical Significance: Can you provide -fold cross-validation (or repeated random splits) along with DeLong or paired bootstrap CIs against the strongest baseline?
- Missing SOTA Comparisons: Can you benchmark against GT-BEHRT, KARE, and Chet under your exact protocol?
- Data & Numerical Discrepancies:
- Why does TCEM in Table 1 ( AUROC on MIMIC-IV) differ from the "Base encoder" in Table 3 ()?
- In Appendix D, Table 11's "Full objective" () matches Table 8's "No hierarchy", contradicting the main full model (). Which numbers are correct?
- Implementation Clarification: Inter-visit time intervals are clipped to days. Since readmission gaps are typically much longer, does this clipping destroy the temporal dynamics meant for the state updater?