- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes a multi-granularity recurrent KV-memory compression framework for longitudinal EHR modeling that treats historical evidence in tiers (recent, intermediate, distant). On MIMIC-IV it approaches full-history accuracy on medication and diagnosis prediction while substantially reducing inference cost.
Critical strengths
The ablations are thorough: the contribution of each memory tier is systematically isolated, a level of completeness rarely seen in this area, and the efficiency claims are backed by a clear mechanism and direct measurements.
Critical weaknesses
Both tasks are evaluated only on a single institution and a test set of roughly 300 patients in MIMIC-IV, the main comparisons lack paired uncertainty estimates, and generalization therefore remains unknown.
The medication task also uses admission-level diagnoses and procedures to predict medications initiated within the first 24 hours, so target information may already be present in the input, and no ablation is provided that rules out this confound.
Questions for authors
Can you validate on a second dataset or institution? Can you run a dedicated ablation for the input-leakage concern in the medication task? What is the training-time compute comparison, given that the intermediate alignment requires full-history KV during training?
5 comments
Response to Reviewer 55GN (Part2)
Question 1: Can you validate on a second dataset or institution? Can you run a dedicated ablation for the input-leakage concern in the medication task? What is the training-time compute comparison, given that the intermediate alignment requires full-history KV during training?
Response: Thank you for these helpful suggestions. We have added the external evaluation and potential input-leakage ablation, as detailed in our responses to Weakness 1 and Weakness 2, respectively. Below, we report the additional training cost of intermediate attention alignment.
Training cost of intermediate alignment. We agree that the full-history reference introduces additional computation during the recurrent memory learning stage. We therefore conducted a detailed comparison of the computational cost of intermediate alignment, as shown below. Intermediate alignment adds only 8.48 seconds per step, as caching the Full History KV states reduces recomputation across successive memory updates while increasing peak GPU memory by 5.49 GiB.
| Configuration | Training time (s/step) | Peak GPU memory (GiB) |
|---|---|---|
| Without | 38.78 | 42.74 |
| With | 47.26 | 48.23 |
Revisions in the manuscript. We have added this cost comparison to to provide a more comprehensive discussion of intermediate alignment.
References
[1] Wornow, Michael, et al. "Ehrshot: An ehr benchmark for few-shot evaluation of foundation models." Advances in Neural Information Processing Systems 36 (2023): 67125-67137.
[2] Shang, Junyuan, et al. "Gamenet: Graph augmented memory networks for recommending medication combination." proceedings of the AAAI Conference on Artificial Intelligence. Vol. 33. No. 01. 2019.
[3] Wang, Yanda, et al. "Self-Supervised Adversarial Distribution Regularization for Medication Recommendation." IJCAI. 2021.
A Gentle Reminder of the Feedback
Dear Reviewer 55GN,
Thank you again for your constructive suggestions. Your comments have helped us substantially improve the quality of our manuscript. Following your comments, we have:
- Added external evaluation on EHRSHOT.
- Reported paired bootstrap confidence intervals for performance differences.
- Added an ablation removing current-visit procedures to examine their influence on medication prediction.
- Quantified the additional training time and GPU memory required by intermediate alignment.
As the discussion deadline approaches, we would appreciate your thoughts on whether our responses address your concerns, along with any further suggestions.
Best regards,
The authors
Response to Reviewer 55GN (Part1)
Thank you very much for recognizing the memory and inference efficiency analysis and for suggesting additional evaluations to strengthen the study. We appreciate your comments on external validation, paired uncertainty, potential leakage, and training cost. We have incorporated the revisions prompted by your suggestions into the manuscript and marked them in red. These insightful suggestions have substantially improved the quality of our manuscript, and we sincerely thank you again. We address each point below.
Response: Thank you very much for raising these concerns. We agree that evaluation on an additional data source and paired uncertainty estimates are important for assessing the robustness of our findings.
External evaluation. We evaluated Qwen3-4B on 30-day readmission prediction using EHRSHOT [1] (NeurIPS 2023), which contains longitudinal EHRs from the Stanford Medicine Research Data Repository (STARR), a data source independent of MIMIC-IV. The test set contains 2,189 samples from 1,190 patients. As shown in the following table, ReLMem achieves comparable performance to Full History with only a difference of 0.14 percentage points, suggesting that ReLMem retains effectiveness on another data source and clinical task.
Paired uncertainty estimates. We further performed 10,000 paired bootstrap resamples over the same 300 test patients, as shown below. Each resample uses the same patient for a baseline and Full History, allowing us to estimate the confidence interval of their performance difference directly. ReLMem has the smallest estimated performance loss among the evaluated methods, and both confidence intervals include zero.
Revisions in the manuscript. We have added the external evaluation to Section 5.5.6 (Lines 510-514), and reported the paired uncertainty estimates in Appendix G.5 (Lines 1674-1693).
Response: Thank you for highlighting this potential confound. We agree that current-visit information, particularly procedures, may include information unavailable at the time of medication prescription. We would like to clarify that our task formulation follows established prior work. Nevertheless, we sincerely recognize this concern and have conducted an additional ablation study following your suggestion.
Task formulation. We followed the medication prediction formulation adopted in GAMENet [2] (AAAI 2019) and SARMR [3] (IJCAI 2021). Both studies use current-visit diagnoses and procedures, together with prior patient history, to predict medications prescribed during the first 24 hours of hospitalization.
Procedure ablation. We also removed procedure information from the current visit and repeated the evaluation, as shown below.
Removing procedures reduces average F1 by only 0.51 percentage points for Full History and 0.30 points for ReLMem, suggesting that current-visit procedure information has a limited effect on predictive performance in this evaluation.
ReLMem still remains close to Full History after removing procedures, indicating that the compression comparison is also robust to this input change.
Revisions in the manuscript. We have added this ablation to Section 5.1 (Lines 330-332) and Appendix G.4 (Lines 1659-1673) to clarify the impact of current-visit procedure information on medication prediction.