acceptodds
Under review as a conference paper at ICLR 2027

Repairing Clinical-Agent Plans with Data-Induced Process Constraints

Abstract

Tool-using clinical agents can place laboratory, imaging, and medication orders, yet their free-text plans make the care process difficult to verify. Hand-crafted checklists can support auditing, but they transfer poorly across settings and risk circular evaluation. Here, we induce machine-checkable process constraints from a held-out split of MIMIC-IV admissions. The method applies Wilson-bounded prevalence mining to identify required actions, clinical concepts, and temporal precedence, incorporating modality-ordered payloads. A deterministic minimal-edit operator then repairs structured plans using four atomic edits and disease-agnostic logic. Across four cohorts, repair raised hard-constraint satisfaction from 0.55–0.82 to 0.91–1.00 (appendicitis, pulmonary embolism, urinary tract infection, and cholecystitis). By contrast, Reflexion, self-consistency, and an LLM critic recovered little of these gains. Replacing the induced specifications with size-matched donor-cohort specifications reduced macro satisfaction by 0.115 and inserted appendectomies into 38 of 40 pulmonary embolism trajectories. LLM-generated specifications covered less than half of the mined requirements and reduced satisfaction by up to 0.307. Induction requires only tens of admissions and is robust to most threshold settings. These results indicate that verifiable clinical agent behavior can be derived from observed care processes rather than from another language model.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Clinical Agents, Process Constraint Induction, Machine-Checkable Specifications, Clinical Process Mining, Agent Behavior Verification

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_otCYOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The authors introduce a framework for correcting the mistakes produced by agents when issuing clinical care plans. Instead of relying on a fixed checklist or written summary they use Wilson’s bounded prevalence mining to learn the required actions from other patients in the cohort and then issue atomic edits to the agents plans. The authors evaluate their approach on a single dataset with DeepSeek V4 Flash and show that their methods provides gains in the satisfaction of hard constraints over popular other LLM based methods such as REACT or Self Reflexion.

Critical strengths

Agents operating within electronic health records is a timely topic and it is important to move beyond the evaluation of endpoint to analyze the trajectories for clinical safety and adequate resource usage.

Critical weaknesses

Experimental evaluation is very weak, only a single model on a small subset of MIMIC patients. Please add evaluation on more datasets such as VivaBench (Chiu et al.), MIMIC Gastro (Hager et al.) and AgentClinic (Schmidtgall et al.) as well results using a variety of SOTA open and closed source models.

Technical novelty is very incremental as it is just a simple verification process for LLM plans using established technologies and there are also no theoretical results.

Questions for authors

I do not agree with the authors statement that free-text plans make process difficult to verify as this is already standard in clinical practice when a pharmacist checks a prescription from a physician and this is unlikely to be replaced by autonomous AI agents in the future.

Furthermore the authors state that hand authored checklists require disease specific engineering but most hospitals already operate under national or local guideline that cover precisely these cases. As such it appears to me that the current systems just learns the correct policy (potentially with noise) from a set of previous patients instead of applying it directly.

Can the authors please add a baseline where the agent is given access to the guidelines (checklist or free text) and asked to ensure all items on there are followed.

Furthermore, could the others please elaborate on the significance of the corrections issues by their system as the LLM outputs of the dataset used were analyzed extensively by Ferber et al. who found it to be safe and guideline concordant.

Why can the reflection and LLM critic baselines not get access to the full history?

Figure does not add much information beyond Table 1 I fele. Space should be used for additional datasets.

Can the authors please run the agent as implemented by Ferber et al. in the original paper since they are using their dataset.

The claim that appendectomies where introduce in PE trajectories donor cohort specifications needs more explanations. Currently it reads like the implemented baseline is degenerate.

There is no clear discussion of the limitation of the approach including the limited evaluation, reliance on prior samples being available (will not be the case for rare diseases) and the problem of adapting to shifts in the guidelines as historical patients will have been treated under the old guidelines.

Minor comments: Missing space before Ferber et al on page 1 Figure 1 is very cluttered Table 1 caption please state what is plotted to make it easier to parse

Reviewer_hfTROpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

Problem

What is the problem/question?

AI agents attempting to take action in the context of clinical care need to be auditable and alignable to constrained processes that meet hospital or human-expert policies or guidelines, but this can be hard to maintain from LLM-induced text outputs alone. However, deriving machine-usable forms of policies/guidelines can be very difficult given the challenges inherent in health data.

Why is the problem/question important/impactful?

Without guidance and constraints, AI agents pose a much greater risk surface in healthcare and thus cannot be as widely used, or induce risks to patients and organizations upon use. Greater control over the outputs of these agents could significantly improve the risk-benefit tradeoffs of these technologies.

Why is the problem/question hard?

It is hard because GenAI based tools output free-text and are, by design, incredibly flexible, and may not be easy to anchor to select, structured decision guidance. Simultaneously, we don't have concrete guidelines that cover all aspects of clinical care, so we don't even know what constraints to apply in general. The constraints we do have cover only a small fraction of medical situations, and are often not presented in a machine readable form.

What naive solutions have or should have been tried to solve this problem/question?

Manually deriving checklists, rule-sets, or concrete guidelines can be used to solve this problem, and has been used effectively historically.

Why are these naive solutions insufficient?

While effective, they are very effort intensive to derive, and may not transport well across hospitals.

Solution

How do the authors propose to solve it (in brief)

The authors propose to automatically identify implicit policies or guidelines from real records of care, and express them in a machine readable/checkable manner, such that future tool-use can be anchored to these implicit policies.

Unfortunately, their methods and beyond section (and really much of earlier sections) have clarity problems that are sufficiently significant that I cannot understand how they solve the problem, or if it is valid.

Critical strengths

Key Strengths

These are reasons I would advocate the paper be accepted

  1. I think that it is important to align agent use in health to specified guidelines, to ensure AI augmented care is safe and complies with policies, so this is a nice direction.
  2. I think identifying clinical practice guidelines in a consistent form from care is an interesting direction.

Minor Strengths

These are strengths, but not sufficient alone to warrant advocating acceptance

Critical weaknesses

Key Weaknesses

These are reasons I would advocate the paper be rejected

Your paper has very significant presentation / clarity problems relating to density of information presented and use of notation/terminology that will be unfamiliar to a large part of your audience or is not properly defined. These issues are present throughout, but concentrated in the abstract and starting in the methods section. You need to have a "plain language" / general ML audience way of introducing the work that clarifies what you do and find, and I'd strongly recommend having an overview section prior to the methods where you describe what the parts of your solution are and give their overview prior to diving into details in the Methods section. In the Methods section, you also need to introduce notation precisely before you first use it, and you should generally give an example or clear free-text illustration of what the notation means. E.g., much of section 3.1 has notation introduced without context, formal definitions, or examples. You also have sentences that have way too little context or build up, like "Induction excludes every frozen evaluation identifier and records split hashes", when you haven't stated what induction, frozen evaluation identifer, or split hashes mean. Ultimately, these problems are sufficient enough that I cannot understand what you are doing in this work, which makes me unable to evaluate this for inclusion in the conference, and indicates it would not be impactful to the audience because they may suffer from similar challenges understanding your work.

Minor Weaknesses

These are weaknesses, but not sufficient alone to warrant advocating rejection

Questions for authors

Action Items for Rebuttal

Things that, if addressed, may change my score

Unfortunately, changes to the text that would increase clarity sufficiently would warrant a re-review, so my position won't change during rebuttal.

Questions

Things that I want to know, but are unlikely to result in me changing my score

Reviewer_EwxYOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes to make clinical-agent plans verifiable by inducing a declarative process specification from a held-out split of MIMIC-IV admissions and repairing structured plans against it deterministically. Induction applies Wilson-bounded prevalence mining to identify required tool actions and clinical concepts, diagnostic-to-therapeutic temporal precedences, and modal order payloads; the resulting specification couples machine-checkable constraints with executable order exemplars. A deterministic operator then repairs each structured plan with four atomic edits (add item, insert action, move action, defer finish) under a budget of K = 24 iterations, leaving the executed prefix immutable. Across eight cohorts (1,103 induction and 527 held-out admissions; 170 live evaluation admissions in appendicitis, pulmonary embolism, urinary tract infection, and cholecystitis), repair raises issued-call hard-constraint satisfaction from 0.550–0.824 to 0.917–1.000, while reflexion, self-consistency, and an LLM critic recover little of the gain. Donor-cohort and LLM-authored specifications are used as fixed-target controls: a size-matched donor specification reduces macro replay satisfaction by 0.115, and an appendicitis donor inserts appendectomies into 38/40 pulmonary-embolism trajectories; LLM-authored specifications recover 0.36–0.47 of the mined hard concepts.

Critical strengths

1. The repair operator is model-independent, deterministic, and bounded.

The repair algorithm is identical across all cohorts — the only hand-authored clinical component is a disease-agnostic normalization map (p.1) — and it applies four fixed-cost atomic edits in a fixed order, stopping after a no-edit scan or K = 24 iterations (§3.4; Table 6). It adds no model calls (one plan sample, no reflection; Table 3), and the paper proves termination and preservation of satisfied presence requirements (Propositions 1–2). Because each specification carries a content fingerprint, scores and violation traces are reproducible. This cleanly separates generation from verification, and is the component most likely to outlive the specific results.

2. The specification couples checkable constraints with executable order payloads, and the fixed-target controls are well designed.

The induced specification pairs requirements, avoidances, and precedences with modal order payloads, making constraints directly actionable rather than merely descriptive (p.2). Holding repair fixed and changing only the specification content, a size-matched donor specification reduces macro replay satisfaction by 0.115 (an appendicitis donor inserts appendectomies into 38/40 pulmonary-embolism trajectories, 0.996 → 0.719), and LLM-authored specifications recover only 0.36–0.47 of the mined hard concepts (§5.3; Table 11). These controls credibly show that the data-induced content matters.

Critical weaknesses

1. The same specification drives repair and the primary outcome, so the headline result measures conformance to the paper's own standard, not clinical correctness.

The primary outcome is HSAT against the same specification the repairer optimizes; the paper states this plainly (§3.5, p.4) and acknowledges it is the circularity it criticizes in hand-authored checklists (p.2). The "independent" recalls are string-matched against orders from the same MIMIC-IV records used to induce the specification, and they are only partly favorable: urine recall shows no consistent improvement, and the deterministic medication-bundle score declines by 0.075 against plain and 0.079 against the critic (both Holm p = 0.0088, p.8). Donor/LLM controls are always scored against the own specification (Fig. 3a), so they show that content matters, not that repaired plans are more often correct. No clinician adjudication or expert validation is reported, and repair can change the HSAT denominator by activating precedence constraints (p.16). A separate scoring standard or a clinician-adjudicated outcome would be needed to establish clinical validity.

2. The repair mechanism is not isolated from the alternative of giving the specification to the planner or critic.

The planner is never shown the specification (Table 3: "Authored rules" off for every arm), and the LLM critic is deliberately a "no-specification critic" (§4, p.5). The paper gives no explicit reason beyond positioning the method against generation-time enforcement (p.3), and it does not run the obvious alternatives: planner prompted with the specification (with and without repair), or critic given the specification. Consequently the comparison with reflexion/critic conflates "has the specification" with "rule-based vs LLM", and the paper itself concedes these arms measure the "combined value" of both ingredients (p.5). Adding these conditions would show whether deterministic repair is necessary or whether prompting with the specification alone suffices.

3. All experiments use a single backbone, so the comparative conclusions are model-specific.

"Every arm uses deepseek-v4-flash; the same backbone supplies the host's medication judge" (§4, p.4). The claim that reflexion/self-consistency/critic "recovered little" is explicitly limited to this backbone (p.5), and the LLM-authored-specification result is tied to one authoring prompt (p.7). The deterministic core is model-independent, but the empirical comparison is not. The judge also shares the backbone with the agent under test and is unstable on repeated evaluation (mean spread 0.116/0.149; Appendix O), so correlated bias cannot be excluded. Results on two to three backbones from different families, checking both that gains persist and that the baselines still fail, are needed.

Questions for authors
  1. Why is the induced specification withheld from the planner, and what would the alternative show? The paper does not state a reason beyond positioning the method against generation-time enforcement. Could the authors report (a) the planner prompted with the same specification but no repair, (b) the planner prompted with the specification plus repair, and (c) the LLM critic given the same specification? These three conditions would separate the value of the specification content from the value of deterministic repair, and would make the comparison against reflexion and the critic informationally fair.

  2. Will the authors add experiments with additional backbones? Which models, and will they check both that the repair gains persist and that reflexion/self-consistency/critic still fail? Since the medication judge shares the backbone with the agent under test, will the authors also report an independent judge or a human spot-check for the medication outcomes?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.