acceptodds
Under review as a conference paper at ICLR 2027

PRUDENT: Representing Evidence-Based Clinical Decision Making

Abstract

Offline reinforcement learning for clinical treatment recommendation typically combines observed practice, clinical knowledge, and outcome-based objectives into a single optimisation problem, with the resulting policy determining the recommended action. While this formulation provides a natural way to optimise treatment, it obscures distinctions that evidence-based medicine requires at the point of care, including where evidence comes from, how strong it is, where it applies, and when it is silent. We instead take the clinical decision, rather than the optimised policy, as the object of study. We introduce PRUDENT, an evidence-based decision framework in which routine clinical practice anchors the decision while guidelines and learned policies remain explicit evidence sources, each retaining its strength, coverage, and ability to remain silent. PRUDENT characterises the decision context along two continuous dimensions, measuring how atypical the current state is relative to routine precedent and how strongly the evidence sources that speak disagree. Together, these dimensions form a decision plane that distinguishes routine decisions, known controversy, consensus extrapolation, and the true edge. PRUDENT represents each evidence source as a transport route from routine practice toward the action it supports, using conditional flow matching to learn these routes. The decision context governs how far departure from routine is warranted, while evidence strength limits how far each source may be followed. The resulting candidates are arbitrated separately for each action dimension using only the sources that speak, and the final action is projected onto the clinically feasible set. On MIMIC-IV, the decision plane identifies a consensus extrapolation region comprising 22% of decisions in which patients have lower SOFA and lactate and higher mean arterial pressure, receive substantially less fluid and vasopressor treatment, yet experience 1.76× the mortality of patients in the routine region. These findings are observational and require no counterfactual estimate. In controlled simulation with known optimal control, PRUDENT adapts its departure from routine across five regimes, moving away when routine practice becomes unreliable while remaining close when it remains appropriate. Together, these results show how evidence-based clinical decision making can be represented without collapsing heterogeneous evidence into a single optimised policy.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
clinical decision support, evidence-based medicine, offline reinforcement learning, flow matching, interpretability, MIMIC-IV

Reviews

Among:
Shape:
1234this paper, Bottom 25%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.75 over 4 reviews, above 25% of the others.
Reviewer_i8UXOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper argues that offline-RL treatment recommendation folds heterogeneous evidence (routine practice, guidelines, learned policies) into one scalar objective and loses each source's provenance, strength, coverage, and ability to stay silent. PRUDENT anchors each decision at routine practice (a behavior-cloned policy) and represents other sources (rules compiled from SSC 2021 with their GRADE strength, and an IPW-IQL policy) as distributions over the same action space with per-dimension silence. It diagnoses the decision context by atypicality a(s) (a rank-calibrated Mahalanobis distance) and disagreement b(s) among the sources that speak, moves from the anchor toward each source along a conditional-flow-matching route by a distance a(s)κ_j (or, when a value signal is available, by the value-maximizing distance capped at a(s)κ_j), combines the endpoints with grade-dependent softmax weights, and projects onto hard constraints. On MIMIC-IV, the a-b plane isolates a "consensus extrapolation" region (22% of windows) whose 90-day mortality is 1.76 times that of the routine region. In a PK/PD simulator with five regimes, the value-guided gate has lower regret than the best fixed transport distance in every regime.

Critical strengths

S1. The position is thoughtful and relevant. Keeping evidence separate by source, strength, coverage, and silence matches how evidence-based medicine and GRADE are meant to work, and the requirement that a silent source must change neither the decision nor the diagnosed context is a clean formal statement.

S2. The real-data methodology is careful: the paper declines to report off-policy value on MIMIC-IV and explains why (effective sample size 0.27), uses stay-clustered and dyadic bootstraps, freezes reference samples, and documents the guideline compilation, including a human gold standard frozen before LLM compilation, inter-annotator agreement, and the two recorded departures from a mechanical reading of the guideline.

S3. The simulator includes regimes where the premise fails (F and G), and the paper reports oracle comparators that beat the method. This is honest.

S4. The consensus-extrapolation observation is an interesting hypothesis-generating finding, and it is checked across partition thresholds and reference seeds.

Critical weaknesses

W1. The headline simulation result depends on an oracle value signal. The gate beats the best fixed transport distance (chosen with hindsight per regime) only in its value-guided form, which uses the true three-step return of the known dynamics; Appendix E.5 says the claim is therefore conditional on a usable value signal, and that this branch is not used on MIMIC-IV.

W2. Several validations hold by construction. Silence invariance and exact reconstruction (Section 5.6, residual 2e-16) follow from masking and the deterministic arbitration formula, so they verify the implementation rather than a scientific claim. The sign of the transport-versus-mixture result (Section 5.5) follows from Jensen's inequality for a concave reward with two sources placed on opposite sides of the optimum, as Appendix E.1 acknowledges, so only its size is informative. The paper also concedes that the learned routes are nearly straight (Section 4.3, Appendix A.2) and keeps conditional flow matching for multimodal or constrained sources, but none of the sources tested is multimodal or constrained, so in the reported experiments it adds little over linear interpolation between paired draws. Please either show a case that needs a learned velocity field or simplify this component.

W3. The routine-practice anchor is mis-specified for the decision that matters most. 73% of windows have no vasopressor and 19% have no fluid (Appendix B.5), yet the anchor is a homoscedastic Gaussian with a softplus mean (Appendix D.1). Appendix D.2 concedes that a zero-inflated anchor would represent the give-or-withhold decision explicitly, and defends the choice because the anchor is never inverted. But every route starts from a draw of this anchor, so the give-versus-withhold decision is poorly represented in the actions PRUDENT outputs. Please show the effect of this approximation, or use the zero-inflated model already built for the importance weights as the anchor.

Questions for authors

Q1. In the simulator, what is PRUDENT's regret when the value-guided branch uses an estimated value function? Q2. Does the consensus-extrapolation region persist after excluding windows that follow a DNR or comfort-measures order? Q3. How sensitive are the decisions to the pre-specified κ_j values? Q4. How would PRUDENT behave with a second learned policy (for example one trained with a different reward), so that two sources speak on vasopressors?

Reviewer_DXkeOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The authors propose using 3 policies (guidelines, behavioral cloning of observed care, and RL-optimized) to determine the treatment decision, along with a method for arbitrating between the policies when there is disagreement. They run some experiments to show what happens when the evidence sources disagree.

Critical strengths
  1. The framing around studying treatment decisions that depart from routine is interesting.
Critical weaknesses
  1. The evidence sources are not different guidelines or different practices that difference clinicians follow, as would be logical for studying disagreement between evidence sources. Instead, in Section 4.1, the authors give only 3 evidence sources: 1 guideline, 1 practice that is an aggregate over all observed data, and an RL-optimized policy. Can the experiments be designed with different sources of evidence?
  2. The RL method is underspecified in Appendix D.2. For instance, what reward function is used? What does it mean that a LayerNorm followed by GRU encodes a window of four states?
  3. The method for arbitrating the policies is unclear. Where did the value signal in equation 5 come from? What does the GRADE-weighted violation score of route kk referred to under equation 6 mean? What is the xj(tj∗(s)∣s)d⋅x_j(t_j^* (s) | s)_{d \cdot} term in equation 7? What does "how far a route may travel" in the first paragraph of section 4.4 mean? In the step that enforces clinical admissibility in equation 9, what does the operator do if the action is inadmissible?
  4. The experiments and discussion sections are not clear. The structure as a series of 6 questions is unconventional for a research paper. Several of the questions and their answers are unclear. For example, in the start of the section for question 3, what does "a gate that always departs and one that never departs" refer to? What are the A, A', B, F, and G referred to there? What is the paired difference in regret plotted in Figure 3? How does the simulated environment in part E relate to the outcomes, i.e. SOFA, lactate, MAP? If the system is known, why use value iteration rather than solving or simulating the outcome? Why use simulated data at all rather than real data like MIMIC-IV? What does "only two of 93 guideline recommendation blocks compile into state-conditional action rules" in the discussion section mean? As in the guidelines weren't referenced or the guidelines weren't converted into treatment rules? Why is the effective sample 0.27 in the discussion paragraph?
  5. No baselines are provided for comparison.
Questions for authors

Questions are written in the weaknesses section.

Reviewer_UK5NOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes the PRUDENT framework, which integrates routine clinical practice, guidelines, and an offline reinforcement learning policy for fluid and vasopressor dosing, while retaining each evidence source's strength, coverage, and ability to remain silent.

Critical strengths

The framework retains the strength and coverage of each evidence source, and distinguishes silence from agreement, allowing the resulting decision to remain attributable to the evidence that produced it.

Critical weaknesses
  1. The paper is difficult to follow despite careful reading. It repeatedly emphasizes concepts and design principles, while important details are scattered across later sections and appendices. This makes it hard to understand how the method works and assess what the experiments actually demonstrate.
  2. The technical novelty is limited. The method mainly combines existing components, and the paper fails to show why these design choices are necessary.
Questions for authors
  1. Why does higher atypicality allow greater departure from routine practice? What evidence suggests that the other sources are more reliable than routine practice in these states?
  2. Simulations use an oracle value signal, but MIMIC-IV uses the value-free branch. Removing the value signal worsens performance in all five regimes. How do these simulations validate the method used on MIMIC-IV?
  3. Why set the transport ceiling to 0.8 for data-derived evidence and 0.5 for weak recommendations?
  4. Why use conditional flow matching instead of other methods such as linear interpolation?
  5. How are the metrics in Figure 2(c) scaled? They have different units, so can the line lengths and endpoint positions be compared across rows?
Reviewer_DHkcOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a different way to use ML for treatment recommendations in intensive care. Most work in this area, including reinforcement learning approaches, folds everything that is known into a single optimisation target. The paper instead treats routine clinical practice, learned from what clinicians actually did, as the starting point of every decision. Other sources of evidence are kept separate and can justify moving away from that starting point. The method is applied to fluid and vasopressor dosing in MIMIC-IV - a standard reference problem in the field.

Critical strengths

The PRUDENT paper focuses on the clinical decision, rather than the optimised policy. Routine practice (estimated by behavioural cloning) anchors every decision, while guidelines and the RL policy act as separate evidence sources with evidence strength limiting how far the decision may depart from routine care. This mirrors how a clinician weighs heterogeneous evidence at the point of care, and demotes the learned policy from target to one source among several.

Critical weaknesses

The paper describes a system that weighs several kinds of evidence: guidelines, a learned policy, and routine practice. It is meant to adjust how much it trusts each one depending on the patient. Sadly, on real ICU data, very little of that happens: the guideline almost never speaks. When the guideline does speak, it always gets about the same weight, roughly 40% against 60% for the learned policy. The authors’ own analysis shows the weighting cannot vary much, so “context-dependent arbitration” barely exists in practice. What remains is a simple rule: start from what clinicians usually do, then move toward the learned policy’s recommendation, moving further the more unusual the patient is. Note that unusual patients are exactly where a learned policy has the least data and should be considered least trustworthy. The simulation results therefore belong to a version of the method that cannot be used on real data. The version used on real data does not beat the trivial alternative. The version that performs well needs to know the true short-term outcome of each dose - a true oracle- which is impossible to achieve with real patients.

Questions for authors

Why does the RL source receive a higher ceiling and prior than a GRADE-weak guideline, given its effective sample size?

In the simulation, can a value-free gate match or beat the best fixed transport distance and plain source selection? On MIMIC-IV, do its decisions differ meaningfully from atypicality-scaled linear interpolation toward the RL policy?

Does the consensus-extrapolation finding survive once its obvious explanations are removed? Concretely: recompute the region and its mortality gap after three changes: excluding stays with DNR/DNI or comfort-care code status, which MIMIC-IV records; computing disagreement without the RL source, or with one trained without the mortality term in its reward; restricting the 30 mL/kg rule to initial resuscitation in patients with suspected infection (if you do not have an ICU clinician in the author team, check with them please - 30 mL/kg exceeds the action ceiling for anyone over 68 kg, so for the many US patients in MIMIC the “guideline dose” is simply the maximum fluid.).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.