- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes CERE (Complementary Evidence Reasoning for EHRs), a framework for integrating structured electronic health records (EHRs) and unstructured clinical notes by identifying complementary information that is not already captured by structured medical codes. The framework consists of three optimization stages: (1) structured context grounding, which converts diagnosis, procedure, and medication codes into continuous soft prompts using a discriminative reconstruction objective; (2) residual evidence learning, which employs an LLM to generate code-conditioned clinical reasoning and extracts task-specific representations through predictive LoRA adaptation; and (3) predictive-guided reasoning refinement, which uses group-relative reinforcement learning to optimize reasoning trajectories based on downstream prediction performance. The resulting representations are integrated with structured EHR features through a RETAIN-based predictor. Experiments on MIMIC-III and MIMIC-IV evaluate the framework on hospital readmission prediction and medication recommendation, demonstrating improvements over conventional EHR models, LLM-based approaches, and frozen clinical note encoders.
Critical strengths
S1. The paper addresses an important limitation of existing multimodal EHR models: the potential redundancy between structured medical codes and clinical notes. Learning complementary rather than merely combined representations is a meaningful research direction.
S2. The three-stage framework establishes a logical progression from structured context grounding to task-specific representation learning and reinforcement learning-based reasoning refinement. Each stage addresses a distinct aspect of the proposed problem.
S3. Using downstream prediction loss to guide reasoning refinement is an interesting approach. It provides a mechanism for connecting LLM-generated reasoning with predictive utility rather than relying solely on linguistic plausibility.
S4. The appendices provide architectural specifications, training procedures, prompts, hyperparameters, and reinforcement learning details, improving the reproducibility of the proposed method.
Critical weaknesses
W1. Although the central claim is that CERE identifies information absent from structured records, the experiments primarily measure downstream prediction performance. These improvements do not necessarily demonstrate that the learned representations specifically capture complementary rather than redundant information. A more direct evaluation of evidence complementarity would strengthen the paper.
W2. CERE requires teacher-generated descriptions, LLM reasoning generation, multiple optimization stages, patient-level RL rollouts, and a final predictor refit. The paper does not provide a sufficiently detailed comparison of computational costs, inference latency, or performance-efficiency trade-offs against simpler fusion methods.
W3. The experiments use two MIMIC datasets originating from the same medical center, with MIMIC-IV further restricted to 20% of eligible patients. Consequently, the results do not establish robustness across different healthcare institutions, documentation practices, or patient populations.
W4. The case studies illustrate changes in evidence prioritization after RL, but they do not independently establish that the selected evidence is clinically accurate, genuinely absent from structured records, or consistently useful across patients.
Questions for authors
Q1. The paper defines the residual representation as z^{res}=F(S\mid V). However, this formulation does not mathematically enforce the removal of redundant information or the isolation of complementary information. The representation is conditioned on structured codes and optimized for prediction, but it may still encode information already present in those codes. The authors should clarify whether complementarity is an intended inductive bias or a property explicitly enforced by the learning objective.
Q2. During Stage 3, reasoning trajectories are rewarded using a frozen predictor trained on Stage-2 representations. After RL, the predictor is reinitialized and retrained from scratch. Consequently, the reward optimizes performance under a predictor that differs from the final evaluation model. Although this procedure is explicitly described, the paper does not sufficiently investigate whether the learned evidence-selection policy remains optimal after the final predictor refit.
Q3. The authors conclude that RL provides additional benefits, which is supported by the higher mean scores. However, the reported confidence intervals overlap for several comparisons between CERE and its variant without RL. Overlapping confidence intervals do not establish a lack of significance, but paired statistical tests would be needed to support stronger claims about the reliability of the improvements.
Q4. The case study suggests that RL shifts attention toward clinically relevant risk factors. However, the selected factors are not systematically compared against the structured codes to verify that they represent genuinely complementary evidence. The appendix appropriately acknowledges this limitation for its additional case study. A quantitative assessment of evidence novelty and factual consistency would better support the paper’s central hypothesis.
Q5. The RL reward combines prediction loss, formatting quality, and representation drift. Since the same adapters influence both generated reasoning and extracted representations, the policy may improve the frozen predictor’s loss without necessarily improving the semantic quality of evidence selection. The authors acknowledge that the regularization terms do not guarantee the absence of reward exploitation. Nevertheless, the paper would benefit from an empirical analysis separating improvements in reasoning quality from changes in embedding geometry.
Q6. The frozen note-encoder baselines provide useful comparisons, but CERE additionally benefits from task-specific LLM adaptation and reinforcement learning. The existing ablations partially address these differences, yet a comparison with a similarly adapted note encoder without complementary reasoning would help isolate the contribution of the proposed reasoning strategy.
Q7. he paper defines the medication prediction target as prescriptions at the next visit and uses historical visits as inputs. However, the clinical interpretation of this setup would benefit from further clarification, particularly regarding the prediction time, availability of discharge notes, and whether all included information would be accessible when recommendations are made.
Q8. The baseline description refers to RePrompT as integrating structured longitudinal EHR encoders with “doctoral notes”. Based on the surrounding discussion, this appears to be a typographical error for “clinical notes” or a similar term.
5 comments
Response to Reviewer QE7v - Part 2
Q2: Transfer from the frozen reward predictor to the refitted predictor.
The frozen Stage-2 predictor provides a fixed evaluation function during policy refinement, and the final refit adapts the downstream predictor to the refined representations. Importantly, Table 3 already evaluates whether the refinement benefits persist after refitting: Appendix A.5 specifies that both CERE and the variant without RL regenerate their representations and retrain a newly initialized predictor using identical settings. The table below reports the differences calculated by subtracting the “w/o RL Refinement” row from the “CERE” row in Table 3. CERE achieves higher mean performance across all eight comparisons, supporting transfer of the refinement benefit to a separately fitted predictor of the same architecture.
| Dataset | Readmission AUROC | Readmission PRAUC | Medication PRAUC | Medication F1 |
|---|---|---|---|---|
| MIMIC-III | +0.51 | +0.34 | +0.49 | +0.51 |
| MIMIC-IV | +0.63 | +0.56 | +0.79 | +1.15 |
Q3: Reliability of RL improvements.
We propose a paired permutation test comparing CERE with its variant without RL for each dataset, task, and metric, using the mean difference across six matched runs. We randomly swap predictions between models at the patient level, keeping visits and labels together and swaps consistent across runs. We recompute this difference over 10,000 permutations and calculate two-sided p-values with a plus-one correction. The maximum and minimum of p-values are [0.0021, 0.0028] for AUROC, [0.0029, 0.0035] for PRUAC, [0.0036, 0.0045] for F1, shows statistically significant different between pairs across all seeds. We will update the paper to reflect this.
Q5: Reasoning quality versus representation changes.
Stage 3 jointly improves evidence generation and its representation for downstream prediction; changes in embedding geometry are therefore part of the intended optimization. To examine whether refinement also improves generated content, we conducted the additional held-out evidence evaluation reported in our response to W1/W4/Q1/Q4. Also, Table 3 shows higher mean predictive performance after retraining a newly initialized predictor, which supports benefits that persist beyond evaluation by the frozen reward predictor.
Q6: Task-adapted note-encoder baselines.
Thank you for this suggestion. We used frozen note encoders to measure the benefit of adding pretrained text representations without task-specific encoder training or explicit reasoning. Our evaluation also includes a task-adapted no-reasoning control: the “w/o Residual Reasoning” variant in Table 3, described in Section 3.5, retains CERE’s backbone, structured conditioning, predictive LoRA adaptation, and downstream predictor while extracting representations without generated reasoning.
Following your suggestion, we have additionally fine-tuned the baseline text encoders for each prediction task, updating the encoder and predictor using the downstream prediction loss while keeping the structured inputs, fusion architecture, and data splits unchanged. The results below show that CERE maintains higher performance across both datasets and all reported metrics, extending the comparison beyond frozen encoders.
MIMIC-III
| Model | Readmission AUROC | Readmission PRAUC | Medication PRAUC | Medication F1 |
|---|---|---|---|---|
| BERT - tuned | 73.39 | 36.47 | 35.09 | 20.66 |
| ModernBERT - tuned | 73.78 | 36.73 | 35.27 | 21.07 |
| BioClinical-MBERT - tuned | 74.03 | 37.12 | 35.46 | 21.43 |
| Qwen3-Embedding-4B - tuned | 74.28 | 37.49 | 35.73 | 21.89 |
| CERE | 75.13 | 38.43 | 36.33 | 22.48 |
MIMIC-IV
| Model | Readmission AUROC | Readmission PRAUC | Medication PRAUC | Medication F1 |
|---|---|---|---|---|
| BERT - tuned | 65.83 | 36.12 | 37.79 | 22.23 |
| ModernBERT - tuned | 66.26 | 36.74 | 38.13 | 22.61 |
| BioClinical-MBERT - tuned | 66.89 | 37.32 | 38.41 | 23.08 |
| Qwen3-Embedding-4B - tuned | 67.47 | 38.19 | 38.86 | 24.03 |
| CERE | 68.71 | 39.43 | 39.54 | 25.11 |
Q7: Prediction time and discharge-note availability.
As defined in Sections 2.1 and 3.2, we predict medications at admission (T+1) using completed admissions through (T), after their documentation is available. Historical medication records and discharge notes are therefore valid inputs, and medications and notes from the target admission are excluded.
Q8: Typographical correction.
Thank you for identifying this error; we will correct it in the final version.
Follow-up on our responses
Dear Reviewer QE7v, Before the discussion closes, we would appreciate hearing your thoughts on our responses and additional results. If there are concerns we have not fully addressed, please let us know so we can respond. Thank you for your time and feedback.
Response to Reviewer QE7v
Thank you reviewer QE7v, for the detailed assessment and for raising questions and concerns about our research. We address them below:
W1, W4, Q1, and Q4: Complementarity and the meaning of “residual.”
Complementarity is an intended inductive bias, rather than a mathematically guaranteed exclusion of redundant information. Section 2.1 explicitly uses “residual” to describe the conditional predictive role of the note representation. The objective is to extract useful evidence from notes given the structured record, rather than enforce independence between the two representations. Structured-code grounding, code-conditioned reasoning, and prediction jointly with structured inputs implement this objective by providing the coded context and helping the model to identify additional evidence relevant to the target.
The relevant comparisons from Table 3 are reproduced below. Removing structured conditioning or generated reasoning lowers mean performance across all eight dataset–metric combinations. The variant without residual reasoning retains structured conditioning, predictive LoRA adaptation, and the downstream predictor, providing a task-adapted direct-encoding comparison.
We further assessed the generated evidence directly on 3,000 randomly selected held-out cases. Clinical agent evaluators, blinded to model-variant identities, reviewed the original notes, supplied codes and their descriptions, and generated outputs. Each output received a score between 0 to 1 for factual accuracy, information beyond the supplied codes, and task relevance. CERE achieved the highest scores across all three criteria:
W2: Computational cost and inference latency.
Thank you for raising this point. We prioritize predictive accuracy over computational efficiency in medical applications because prediction errors can have serious clinical consequences. For transparency, we report both training and inference times. The tables shows MIMIC-III readmission task training/inference time.
W3: Dataset scope.
Although both datasets originate from the same institution, this does not imply a clinically homogeneous patient population. The official MIMIC-III documentation explicitly describes a diverse ICU population, and the original dataset paper reports patients across coronary care, cardiac surgery recovery, medical, surgical, and trauma/surgical intensive care units [1]. This breadth supports its value as a clinically heterogeneous benchmark for evaluating our method.
[1] Scientific Data (2016) MIMIC-III, a freely accessible critical care database.
Our choice of MIMIC-III and MIMIC-IV follows established evaluation practice in prior work, including GraphCare [2] and KARE [3], both of which evaluate their EHR prediction methods exclusively on these two datasets.
[2] ICLR (2024) GraphCare: Enhancing Healthcare Predictions with Personalized Knowledge Graphs.
[3] ICLR (2025) KARE: Reasoning-Enhanced Healthcare Predictions with Knowledge Graph Community Retrieval.