acceptodds
Withdrawn from ICLR 2027

MARC: Multi-Scale Adaptive Relation Calibration for Longitudinal EHR-based Heart Failure Prediction

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Predicting whether heart failure is recorded at a subsequent visit from longitudinal electronic health records (EHRs) requires modeling both cross-visit dependencies and the relations among diagnosis codes. Existing methods exploit temporal information, medical knowledge, and graph structure, yet provide limited control over how these relations are integrated into code representations, graph propagation, and training supervision in a specific prediction context. We propose MARC, an adaptive relation-calibration framework built on the principle of preserving support while calibrating strength. MARC comprises three calibration mechanisms: knowledge-aware hierarchical semantic calibration retains code-specific information; state-aware edge calibration controls the propagation strength of observed co-occurrence edges; and class-aware patient relation calibration regularizes within-batch patient relations during training. Evaluated on heart-failure prediction cohorts constructed from MIMIC-III and MIMIC-IV, MARC achieves the highest mean AUROC, F1, and AUPRC and the lowest mean Brier score on both cohorts. A complete factorial study further reveals that the contributions of the three calibration mechanisms depend on the dataset and module combination, rather than combining as fixed additive gains. The source code is available at https://anonymous.4open.science/r/marc-reproducibility-5109/.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Heart Failure Prediction, Electronic Health Records, Relation Calibration, Medical Ontology, Graph Representation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_pLZNOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces MARC for predicting whether heart failure (HF) is recorded in a patient’s final hospital visit based on diagnosis codes from prior visits. Built on the TCEM graph-temporal backbone, MARC introduces three components: (H) CCS hierarchy-guided code embedding via residual attention, (E) patient-state-conditioned edge gating on a co-occurrence graph, and (R) a batch-level similarity regularizer (training-only). Evaluated on MIMIC-III (7,493 patients) and a MIMIC-IV subset (10,000 patients) with fixed 1,000-patient test sets across 5 random seeds, MARC achieves marginal gains in AUROC/AUPRC over seven baselines, supported by a full 2×2×22 \times 2 \times 2 factorial ablation.

Critical strengths
  • Systematic Factorial Ablation: The full 2×2×22 \times 2 \times 2 module ablation (Tables 2–3) is exceptionally thorough, with transparent reporting that module contributions are non-additive and dataset-dependent.
Critical weaknesses
  1. Flawed Task Definition & Lack of Clinical Relevance: The endpoint predicts prevalence/recording rather than incident HF. Patients with prior HF diagnoses are highly likely to have HF re-recorded. Without reporting prior HF code prevalence or benchmarking against a trivial "copy-forward" rule (predict positive if prior HF exists), it is unclear if models are merely detecting historical codes. Incident HF prediction—the clinically meaningful setup—is not evaluated.
  2. Statistically Unsound Claims: Performance gains over the top baseline are marginal (0.64 AUROC on MIMIC-III, 0.73 on MIMIC-IV). Because evaluation relies on a single fixed split (seeds only vary training noise, not test sampling uncertainty), the estimated 95%95\% CI half-width (∼±2.5\sim\pm 2.5 AUROC points via Hanley–McNeil) vastly exceeds the reported improvements. No cross-validation, CIs, or hypothesis tests (e.g., DeLong, paired bootstrap) are provided. Additionally, on MIMIC-IV, H+RH+R matches the full model (93.5793.57), rendering EE redundant.
  3. Outdated & Incomplete Baselines:
    • Omits the Chet benchmark protocol (Lu et al., AAAI 2022) despite identical cohort and task setups.
    • Fails to compare against directly cited graph-temporal and knowledge-enhanced SOTA baselines, including GT-BEHRT (ICLR 2024) and KARE (ICLR 2025). Most compared baselines date back to 2016–2021.
Questions for authors
  1. Incident HF & Baseline Rule:
    • What percentage of test positives/negatives have HF codes in prior visits?
    • What are the AUROC/AUPRC scores of a simple "copy-forward" rule?
    • Can you evaluate MARC and baselines strictly on the incident HF cohort (patients with no prior HF history)? (Sustained performance here would significantly improve my evaluation.)
  2. Statistical Significance: Can you provide kk-fold cross-validation (or repeated random splits) along with DeLong or paired bootstrap 95%95\% CIs against the strongest baseline?
  3. Missing SOTA Comparisons: Can you benchmark against GT-BEHRT, KARE, and Chet under your exact protocol?
  4. Data & Numerical Discrepancies:
    • Why does TCEM in Table 1 (92.4392.43 AUROC on MIMIC-IV) differ from the "Base encoder" in Table 3 (92.6692.66)?
    • In Appendix D, Table 11's "Full objective" (85.77/73.8885.77/73.88) matches Table 8's "No hierarchy", contradicting the main full model (86.23/74.7586.23/74.75). Which numbers are correct?
  5. Implementation Clarification: Inter-visit time intervals are clipped to [0,15][0, 15] days. Since readmission gaps are typically much longer, does this clipping destroy the temporal dynamics meant for the state updater?
Reviewer_RsJgOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes MARC, an adaptive relation-calibration framework built on the principle of preserving support while calibrating strength. There are three important components: 1. knowledge-aware hierarchical semantic calibration retains code-specific information; 2. state-aware edge calibration controls the propagation strength of observed co-occurrence edges; and 3. class-aware patient relation calibration regularizes within-batch patient relations during training. The authors also conduct factorial ablation studies and additional analyses to examine the contributions of each component. There are increasing many papers tyring to construct and leverage graphs structure for knowledge borrowing from broader concepts, patient state, and within-batch relations. However, it is unclear what is foundamentally new, and the evaluation is insufficient.

Critical strengths
  1. The framework is clearly presented, and the three components are clearly explained.
  2. The complete factorial ablation study and additional analyses of the proposed components help explain the model’s behavior.
Critical weaknesses
  1. It is not clear what the novelty is, especially both knowledge-aware hierarchical semantic calibration and class-aware patient relation calibration come from existing literature.
  2. It would be better if authos can highlight key idea or differences given existing literature.
  3. All experiments focus on heart failure prediction, despite MARC being presented as a general framework for longitudinal EHR modeling. MIMIC-III and MIMIC-IV also originate from the same healthcare system. I suggest evaluating additional tasks, such as in-hospital mortality, 30-day readmission, prolonged length of stay, and phenotyping. Multi-label tasks, such as medication recommendation or diagnosis prediction, would further demonstrate the framework’s generalizability.
Questions for authors

Q1. Can the authors provide results on additional clinical prediction tasks?

Q2. How could MARC incorporate medications, laboratory measurements, and procedures?

Q3. What is the key novelty compared with GRAM and other hierarchical representation methods, or existing graph learning and representation regularization approaches?

Reviewer_oSqXOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper uses diagnosis codes and time gaps from a patient’s earlier visits to predict whether heart failure is recorded at their last visit. This may include patients who already had heart failure. MARC adds three parts to the TCEM model: H controls how much medical hierarchy information is added to each diagnosis code’s representation; E adjusts existing links between diagnosis codes based on the patient’s current state; and R helps keep patients with different labels from having overly similar representations during training.

MARC gets the best average results on all four metrics in Table 1 for both MIMIC-III and MIMIC-IV. The authors also test all eight combinations of the three parts. R helps the most on its own, H adds some benefit when used with R, and E gives smaller, less consistent improvements. The method is clear, but the results do not yet show why all three parts are needed instead of a simpler model. The paper also needs to explain more clearly how the patient groups were selected and why the backbone baseline results differ across tables.

Critical strengths
  1. The ablation study is useful. Tables 2 and 3 test all eight combinations of H, E, and R, with averages and standard deviations from five runs. The results show that R helps the most, H can add value when used with R, and E does not always help. The authors openly discuss these mixed results, which makes the analysis more convincing.

  2. The paper clearly explains how the model uses relations and data. H keeps the original diagnosis-code information while adding hierarchy information. E only changes existing graph edges, and R is only used during training. The graph is built without validation or test visits. The appendices also explain graph pruning, hierarchy coverage, model settings, and checks of each component. These details make the method easier to understand and help avoid using information that should not be available during training.

Critical weaknesses
  1. The results do not clearly show why the full model is needed, especially E. The three components build on existing ideas, so stronger comparisons are important. In Table 2, H+R has better AUPRC than the full model (78.82 vs. 78.64), and adding E only slightly improves AUROC. In Table 3, both models have the same AUROC (93.57), with almost no AUPRC difference. Appendix Table 9 also shows that removing E improves three metrics. The paper should show whether E reliably helps enough to justify the extra complexity. R also needs comparison with simpler supervised contrastive or label-based losses. So far, the evidence supports R or H+R more strongly than the full model.

  2. Some task and baseline details need clarification. TCEM and the base encoder are described as the same backbone, but their MIMIC-IV results differ across Tables 1 and 3. The paper also needs clearer patient selection rules, heart-failure codes, sampling details, and the proportion of patients with prior heart failure. Since existing heart failure is allowed, a simple “previously diagnosed with heart failure” baseline would help show the task’s difficulty. The authors should also explain the 15-day limit on visit gaps and report results by prediction horizon. Both datasets come from the same medical-center system, so the findings currently support diagnosis-record prediction in these cohorts rather than broader heart-failure risk prediction.

Questions for authors
  1. Could you specify the inclusion criteria, heart-failure code list, patient sampling, and prevalence of prior heart failure in both cohorts? Please report a simple prior-heart-failure baseline and performance for patients without prior heart failure. What is the distribution of time from the last input visit to the target visit?

  2. What are the paired differences and confidence intervals between the full model and H+R under the same splits and training budget? Does E reliably improve over fixed edge scaling or a simple gate without patient-state input? Please provide any existing results that address this.

  3. How does R compare on the test sets with a standard supervised contrastive loss or a simple label-based relation target? Appendix C.1 gives a physical batch size of 16 and an effective size of 32 for MIMIC-IV. Is the relation loss computed within each 16-patient microbatch or across all 32 patients? How does this affect the available class pairs and results?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.