acceptodds
Under review as a conference paper at ICLR 2027

LongRx: Closed-Loop Medication Recommendation with Action-Coverage Reinforcement Learning

Abstract

Medication recommendation is commonly formulated as one-shot set prediction, although prescription construction naturally involves iterative information acquisition, medication adjustment, and drug–drug interaction verification. We propose LongRx, a closed-loop medication decision agent that sequentially constructs prescriptions through five parameterized actions: RETRIEVE, ADD, DELETE, VERIFY, and EXIT. Starting from an empty candidate set and prescription, LongRx learns state-dependent decision sequences rather than following a predefined workflow. To improve sequential learning, we combine terminal prescription supervision with process-level feedback on local prescription changes. Our Coverage-Aware Tree Optimization (CATO) combines opportunity-aware action coverage with tree-relative credit, broadening exploration of under-explored valid decisions and comparing their downstream outcomes from the same prescription state, while concrete medication edits and retrieval queries remain policy-generated. In an idealized fixed-state, single-draw model, the coverage correction contracts pairwise action log-odds. Experiments on MIMIC-III, MIMIC-IV, and eICU show improved overlap with recorded prescriptions and lower rates of DrugBank-listed interacting medication pairs relative to RES-MR. These metrics characterize agreement with historical prescribing and a database-based interaction proxy; they do not establish clinical appropriateness or improved patient outcomes.Our code is available at https://anonymous.4open.science/r/LongRx.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Medication recommendation, Electronic Health Records, LLM agents

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_tVLFOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes LongRx, which formulates medication recommendation as a sequential decision process with RETRIEVE, ADD, DELETE, VERIFY, and EXIT actions. The main technical contribution is CATO, which combines tree-based credit assignment with coverage-aware sampling of under-explored valid actions. Experiments on MIMIC-III, MIMIC-IV, and eICU show modest but consistent improvements in prescription overlap and DDI rate, with extensive ablations and matched-budget comparisons.

Critical strengths
  • The experimental evaluation is thorough. The paper includes matched rollout/resource comparisons, component ablations, exploration baselines, statistical testing, cross-dataset evaluation, and controls for access to the DDI verifier. These make the empirical improvements fairly convincing.

  • The closed-loop formulation is useful and well motivated. Learning when to retrieve, edit, verify, and stop is more flexible than using a fixed medication-generation workflow, and the experiments support the value of this design.

Critical weaknesses
  • Limited novelty: Most ingredients—multi-turn GRPO, tree/shared-state credit assignment, and exploration-aware branching—have close precedents. The main new piece appears to be the opportunity-normalized action coverage rule, which feels incremental.

  • Baseline coverage: The paper does not directly compare against some closely related recent methods such as GiGPO/ATPO, making it harder to judge the added value of CATO over stronger agentic RL baselines.

  • Application novelty: In medication recommendation, related agentic systems such as SafeRx-Agent already combine sequential decision making with safety verification, so the distinction of LongRx could be better established empirically.

Questions for authors
  • Can you clarify the novelty of CATO relative to recent multi-turn/tree-based GRPO methods such as GiGPO and ATPO? In particular, what capability does opportunity-normalized coverage provide that these methods cannot achieve with their existing exploration mechanisms?

  • Why are closely related agentic medication systems such as SafeRx-Agent not included as empirical baselines?

  • How were the 250 extra medications selected for the blinded clinical evaluation? Was the sampling procedure fixed and uniform across methods/patients?

Reviewer_b55AOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes a closed-loop medication recommendation agent, LongRx, which models prescription construction as a sequential decision problem, distinguishing it from traditional one-off prediction methods. Five parameterized actions are designed, and a Coverage-Aware Tree Optimization (CATO) algorithm is proposed. Experimental results on three public datasets show that, compared to mainstream baselines, this method improves both the F1 score and drug-drug interaction (DDI) metrics.

Critical strengths

Closed-loop modeling has practical value for clinical decision support and its expressive power is superior to fixed procedures. CATO's chance normalization exploration mechanism provides a new approach for research on the diversity of reinforcement learning actions; the experiments are relatively complete, covering ablation, cross-dataset transfer, and safety analysis.

Critical weaknesses

There are flaws in mathematical consistency: the reported drug dosage exceeds the theoretical upper limit of the RETRIEVE mechanism, the action cost contradicts the definition, and the credibility of the experiment is compromised; large model placeholder examples raise doubts about the authenticity of the data.

Questions for authors

Q1: Explain the inconsistency in search capacity: The single RETRIEVE limit is 16, and reports of newly added drugs exceeding the theoretical limit are being submitted; explain the R-A overhead in Table 37, providing implementation details and original patient logs.

Q2: Verify the source of experiments: Confirm that all quantitative results are from real experiments and not placeholder samples; submit the reproducibility code, evaluation logs, and outputs of each random seed.

Reviewer_dXikOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes LongRx, a medication recommendation agent that constructs prescriptions through retrieval, medication addition and deletion, interaction verification, and termination. The agent is trained with terminal prescription rewards and process-level feedback. The proposed Coverage-Aware Tree Optimization (CATO) encourages exploration of underused valid actions and evaluates alternative decisions through shared-state branch comparisons. Experiments report consistent but modest improvements in prescription agreement and drug–drug interaction (DDI) rates.

Critical strengths
  1. The framework integrates candidate retrieval, prescription editing, and DDI verification into a unified policy.
  2. Opportunity-aware coverage addresses a concrete exploration issue by accounting for how often each action is available.
  3. Component ablations and cross-dataset evaluations support the consistency of the reported improvements.
Critical weaknesses
  1. Relative to RES-MR, F1 improves by only 1.06 percentage points on both MIMIC-III and MIMIC-IV, while DDI decreases by 0.44 and 0.75 percentage points, respectively. The practical value of these improvements relative to the additional system complexity remains unclear.
  2. Insufficient motivation for multi-step modeling. The paper does not clearly establish why this task requires sequential interaction. If avoiding drug interactions is a central motivation, a one-shot system could retrieve candidates and jointly select a prescription using the same DDI information or constraints. The paper needs stronger evidence that adaptive retrieval and editing provide benefits beyond such a pipeline.
Questions for authors
  1. What specific dependencies make adaptive multi-step interaction necessary, rather than simply one possible implementation of prescription construction?
  2. How does LongRx compare with one-shot retrieval and joint medication selection using an equivalent candidate budget and the same DDI checker or constraints?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.