acceptodds
Under review as a conference paper at ICLR 2027

Coverage-Aware Reasoning with Medical Tokens for Diagnosis Prediction

Abstract

Large language models (LLMs) offer promising potential for next-visit diagnosis prediction, owing to their ability to integrate longitudinal clinical evidence and reason over it in natural language. However, reinforcement learning for LLM reasoning commonly rewards each trajectory according to the correctness of its final answer. In next-visit diagnosis prediction, multiple diagnoses can be simultaneously valid, but independently rewarding one diagnosis per trajectory does not distinguish repeated hits from coverage of different diagnoses. The policy can therefore concentrate on a few correct diagnoses, leaving others uncovered. Meanwhile, LLM tokenizers can split ICD codes into several generic tokens with limited clinical meaning, requiring multiple decoding steps to predict each diagnosis and hindering reasoning over a large disease vocabulary. To address both challenges, we propose CARing, a framework that represents diagnoses with compositional Semantic IDs (SIDs) and optimizes reasoning trajectories for multi-label coverage. Concretely, we first encode ontology-enriched disease semantics into compact SIDs through residual quantization, and ground the resulting SID tokens in natural language and longitudinal EHR contexts through multi-task alignment and reasoning-enriched training to unlock transferable LLM reasoning. CARing further improves unordered multi-label prediction through a coverage reward for reinforcement learning and multi-positive supervision. At inference time, the model supports both efficient direct constrained decoding and multi-chain reasoning with rank fusion. On MIMIC-III and MIMIC-IV, CARing exceeds all EHR-trained baselines in weighted and attains the highest top- recall at every reported cutoff, including R@30 of 46.04% and 46.52% in reasoning mode.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Electronic health records, next-visit diagnosis prediction, large reasoning models, semantic IDs, reinforcement learning, multi-label learning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_Y2X1OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposed CARing, a LLM and GRPO-based approach for EHR Diagnosis Prediction. In particular, the method first learns a medical concept tokenizer with RQ-VAE and is injected with ICD ontology for enhanced medical knowledge, which is dubbed Semantic IDs (SIDs). In order for the LLM to be able to comprehend and reason on these SIDs, the paper introduced six multi-task alignment tasks between SIDs and clinical text, and to enhance CARing’s reasoning and world knowledge the models utilized knowledge distillation with a strong LLM's reasoning paths. Afterwards, the paper pointed out the mismatch where in standard GRPO rewards each trajectory independently and cannot distinguish repeated correct predictions from coverage of distinct diagnoses. The authors addressed this mismatch with rewards at the visit level, with a coverage reward to avoid collapsing the predictions into a few popular predictions only, and a Multi-positive Supervision SFT signal. The method achieves competitive Diagnosis results on two popular EHR datasets MIMIC-III and MIMIC-IV.

Critical strengths
  1. Technically sound framework with clear motivations: The motivations behind each component are clean and logical, which are 1. LLMs can't reason directly on Semantic IDs 2. RL rewards are not fit for unordered multilabel diagnosis, which can cause the model to collapse on a few popular predictions only. The proposed solutions to these problems are technically sound, and proven to work empirically.

  2. Competitive experimental results: The method is tested on two popular EHR datasets MIMIC-III and MIMIC-IV, which achieves superior results compared with previous methods.

Critical weaknesses
  1. Limited methodological novelty and great similarities to previous works: While the paper clearly positioned their motivations, most of the solutions proposed by the authors are adapted from previous established works. In particular, the idea of injecting ICD codes' representation with its ontology is well-established [1], [2], [3] as well as using codebooks for medical tokenizers [4],[5]. The Multi-task Alignment training tasks between SIDs and text are quite similar to those proposed by existing recommendation papers [6], [7], [8] (using semantic IDs sequence to generate future SIDs, using semantic IDs to generate their textual descriptions). Frequency-weighted GRPO for more diverse predictions are also established by [9], which was cited by the paper. Most notably, the paper's main contributions are quite similar to an uncited recommendation paper [6], from RQ-VAE quantization of items, alignment pretraining with past SID sequence to future SIDs and SIDs to language, followed by Enriched Reasoning with a large teacher model, followed by GRPO. These similarities make the paper too incremental to previous works to be accepted for ICLR's novelty bar.

  2. Lack of comparison with more state-of-the-art LLM methods for EHR Diagnosis: While the paper did include methods using Pretrained Language Models' embeddings like BoxLM and HieraMed, the paper excluded significant baselines that can autoregressively model EHR data with LLMs like MERA [10], EHR-R1 [11]. If time and resources are not feasible for exactly reproducing these models, then maybe a light ablation of components similar to the paper's proposal will suffice .

[1] Lu, Chang, et al. "Collaborative graph learning with auxiliary text for temporal event prediction in healthcare." arXiv preprint arXiv:2105.07542 (2021). [2] Yang, Kai, et al. "Kerprint: local-global knowledge graph enhanced diagnosis prediction for retrospective and prospective interpretations." Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 37. No. 4. 2023. [3] Choi, Edward, et al. "GRAM: graph-based attention model for healthcare representation learning." Proceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining. 2017. [4] Xiaorui Su, Shvat Messica, Yepeng Huang, Ruth Johnson, Lukas Fesser, Shanghua Gao, Faryad Sahneh, and Marinka Zitnik. 2025. Multimodal medical code tokenizer. In Proceedings of the 42nd International Conference on Machine Learning (ICML'25), Vol. 267. JMLR.org, Article 2262, 57030–57045. [5] Hang Lv, Kaisong Zhang, Yanchao Tan, and Xing Chen. 2026. Towards Efficient and Interpretable Medical Concept Representation via Ontology-driven Residual Vector Quantization. In Proceedings of the ACM Web Conference 2026 (WWW '26). Association for Computing Machinery, New York, NY, USA, 8645–8648. https://doi.org/10.1145/3774904.3792930 [6] He, Yingzhi, et al. "Reasoning over semantic ids enhances generative recommendation." Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 2. 2026. [7] Zheng, Bowen, et al. "Adapting large language models by integrating collaborative semantics for recommendation." 2024 IEEE 40th International Conference on Data Engineering (ICDE). IEEE, 2024. [8] Liu, Zhanyu, et al. "Onerec-think: In-text reasoning for generative recommendation." arXiv preprint arXiv:2510.11639 (2025). [9] Anschel, Oron, et al. "Group-aware reinforcement learning for output diversity in large language models." Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 2025. [10] Ma, Mingyu Derek, et al. "Memorize and rank: Elevating large language models for clinical diagnosis prediction." Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 39. No. 23. 2025. [11] Liao, Yusheng, et al. "Ehr-r1: A reasoning-enhanced foundational language model for electronic health record analysis." arXiv preprint arXiv:2510.25628 (2025).

Questions for authors

Q1 More comparative results with relevant LLM baselines, as stated in Weaknesses, if time and resource allows of course. Q2 Comparison with the original Qwen 3-1.7B models: How much of the gains in performance are thanks to the original checkpoints' capabilities? Do Qwen3-1.7B fine-tuned on raw ICD codes through the same pipeline without SIDs achieve comparable results? Q3 Main improvements come from non-headlines contributions: while the paper’s' stated major motivations are the coverage rewards, from the ablation section 4.3 the component that drove the most improvements was Reasoning Activation by the large teacher (from 20.64 to 25.09 in w-F1, Figure 3), while those achieved by coverage rewards are quite incremental (from 23.57 in S4 to 25.41 in S6, Figure 4). I would like the authors to clarify on the relative importance of this component, since as of now most of the gains seem to come from distillation from an already capable teacher with better downstream performance, and not from the paper's main contributions.
Q4 Computational Efficiency of the method: While CARing achieved significant improvements over deep learning and PLM-based baseline, the training and inference cost are substantially larger due to LLM's long reasoning chains, along with heavy multi-step SFT and GRPO. Does the performance gap (around 5-7% from Table 1) justify a significant uptick in computing cost and speed in real clinical scenarios? Q5 Effectiveness of the reasoning trace: While the paper achieved impressive downstream classification results, the actual quality of the reasoning generated by the models are not evaluated. The interpretability and accuracy of reasoning traces is a critical issue in healthcare, which can greatly affect medical professionals' decision making. Also during multi-positive loss, the method replaces an output diagnosis while retaining the reasoning trajectory that preceded it. Can this technique result in the model’s final predictions being disconnected from its generated reasoning traces? Q6 Generalization in low-resource datasets: The paper claimed CAring having better generalization capabilities because of MIMIC-IV's 7,000 training sets with many missing codes from test split, however 7,000 are still a relatively large number of samples. What is the performance of the model in even lower resource settings (5, 10, 20, 50…% of the chosen samples)? Will the improvements by training processes such as Reasoning Activation, Coverage Reward… hold under fewer samples? Does performance on rare/new codes suffer substantially?

Reviewer_8aPjOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes CARing for next-visit diagnosis prediction, addressing the tendency of reinforcement learning to repeatedly predict a few correct diagnoses while leaving others uncovered. CARing represents diagnoses as compositional semantic IDs derived from disease descriptions and ICD ontology paths, and grounds them through multi-task alignment and reasoning-enriched training. It combines a coverage reward with multi-positive supervision and supports direct decoding and multi-chain reasoning with rank fusion. On MIMIC-III and MIMIC-IV, its reasoning mode achieves R@30 of 46.04% and 46.52%, respectively, while outperforming all evaluated EHR-trained baselines in weighted F1.

Critical strengths
  1. The paper identifies a meaningful distinction between individual trajectory correctness and collective diagnosis coverage. The coverage reward has an intuitive interpretation: its raw group total equals the number of distinct correct diagnoses, connecting the optimization objective directly to diagnosis coverage.

  2. The framework integrates medical semantic representations, clinical language alignment, and multi-label optimization. The training-stage ablations and curves tracking repetition and correct diagnosis coverage provide useful evidence for understanding the differences between ordinary and coverage-based rewards.

Critical weaknesses
  1. My main concern is that the methodological novelty is limited and key baselines are missing. Coverage Reward allocates credit inversely to diagnosis frequency, making it conceptually close to GAPO, yet it is compared directly only with ordinary EM Reward, without comparisons against GAPO or SetPO. The lack of controlled comparisons between diagnosis representations under the same backbone and training settings also leaves the independent contribution of compositional medical SIDs unclear.

  2. Another concern is that the experiments use only Qwen3-1.7B and two related EHR datasets, with MIMIC-IV further restricted to 10,000 patients. This makes it unclear how well the findings generalize to other models and clinical settings. In addition, the paper does not adequately report variation across random seeds or statistical significance, and the additional inference cost of multi-chain reasoning remains unclear, making the reliability and practical benefits of the improvements difficult to assess.

Questions for authors
  1. Could the authors compare against GAPO and SetPO under matched training and computational budgets, and include raw ICD codes or atomic disease tokens as representation baselines to establish the contribution of each core component?

  2. Could the authors validate the findings on a larger MIMIC-IV cohort, independent EHR data, or additional backbones, and report means, variances, and confidence intervals across multiple random seeds?

  3. Could the authors evaluate multi-positive supervision on its own and with the standard EM reward, and compare inference with and without explicit reasoning using the same checkpoint? It would also be helpful to report inference settings and costs to clarify how much of the improvement comes from training versus additional computation at inference time.

Reviewer_QuXwOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

background: Electronic Health Records (EHRs) are among critical real-world evidence in medical reasoning. A key ML task around EHRs is next visit diagnosis/treatment prediction: can one predict what happens in the next visit of a patient based on its history of past visits? LLMs have opened a new realm of possibility to analyze EHRs. A key challenge, however, is that EHRs use multiple code standards that LLM tokenizers (particularly smaller ones) struggle to properly preserve meaning.

paper summary: This paper introduces CARing model that uses Semantic IDs for disease names, where a three-level token is build based on Residual Quantization VAE of a text embedding of each disease's name and ICD hierarchy path. Then it leverages multiple stages like language alignment to “teach” the llm what these SIDs mean, and SFT from teacher reasoning traces and followed by GRPO. One of the novelties of their training is rewarding coverage of diagnosis predictions, rather than the typical RL setup that may encourage a mode collapse in predictions. The e2e setup is then tested on MIMIC-III,&IV inpatient admissions datasets. Reports the best R@10-40 on both datasets, and the best w-F1 on EHR-trained models. Results also demonstrate that without coverage reward, GRPO collapse onto a few correct diagnoses.

Critical strengths

The solution proposed in the paper for next visit diagnosis prediction task is real and proposed solution is quite sound: Rewarding each GRPO rollout for its own correctness is a poor fit in EHR context because target is a set of diagnoses not a single one. The experiments reported in Fig 4-5 goes into length to show the collapse mechanistically. And their proposed solution is practically cheap, adding a properly designed coverage rewards mitigates this. I find this the most transferable contribution of the paper, and the most rigorously supported in their experiments.

Critical weaknesses
  • I have a hard time to understand where SID tokenization helps at all. The ablation study does not isolate this. No version of the trained pipeline using text only or raw ICD codes only are shown. The model only surpasses simple EHR baselines (e.g., BoxLM) only after S4 where traces of the teacher model come into training.
  • The evaluation protocol might be biased. CARing gets validity enforced, so decoding always return a valid full list. However, DeepSeek’s free-text are filtered post generation and kept up to at most 40. What if we allow CARing to generate tokens freely and treat it the same as DeepSeep protocol?
  • Many baseline studies are missing. For instance, what would happen if we just finetuned Qwen3-1.7B on the next visit diagnosis prediction task? This would be a much simpler compute compared to the full training proposed in the paper.
  • MIMIC datasets are ICU visits. I expect a simple frequency-based model (or even a dumber model of use last visit as the prediction) do relatively good here. I liked to see those baselines also.
  • Timing between visits is completely ignored in their modeling. A 5 day vs 1 year wait between visits can be very different.
  • Some key parameters are not reported (R, B, \kappa).
Questions for authors
  • For DeepSeek, what fraction of generated codes were invalid? How about CARing without decoding enforcement?
  • What are the compute GPU-hours for training?
  • The ablation setup should determine if SID is useful in the first place or not: How does Qwn3-1.7B finetuned on T4 (text history->text disease name) would perform here? How about using (a) raw IDC code strings as outputs and (b) random 3-token IDs for the 4491 codes instead of the learned SIDs?
  • Is cross-data set transfer possible? Like train on MIMIC-III and test on MIMIC-IV?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.