acceptodds
Under review as a conference paper at ICLR 2027

EvoCare: Towards Efficient and Robust Clinical Diagnosis with Proactive and Interactive Self-Evolving AI Agents

Abstract

Clinical diagnosis starts with incomplete evidence, so the quality of each question shapes the final decision. Yet successful consultations alone reveal little about which questioning strategies will help future patients. We introduce EvoCare, a self-evolving agent that turns consultation histories into reusable inquiry policies. It contrasts related successes and failures to propose targeted rules, then tests each update on a separate cohort before adoption. Across three clinical diagnosis benchmarks and two language-model backbones, EvoCare outperforms interactive and experience-reuse baselines; on StreamBench-DDxPlus, it improves top-rank pathology accuracy by 2.9-6.3 percentage points over the strongest baseline. Further analyses confirm the contribution of each component and show efficient questioning, low repetition, and consistent performance across models. EvoCare points toward a new generation of clinical AI that learns from experience to navigate diagnostic uncertainty more effectively.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
AI agents, self-evolving agent, proactive clinical diagnostic

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_tawzOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

EvoCare studies how frozen language models can improve clinical diagnosis under limited inquiry budgets by updating external skills between cases. The method pairs successful and failed trajectories with similar behaviors and chief complaints and proposes local rules with applicability conditions.The rule must apply to at least one selection case. The candidate policy must also strictly improve the mean diagnostic score on an independent selection cohort to be accepted.The policy is then frozen for testing. The paper compares agent and policy optimization methods on three benchmarks, iMedQA, StreamBench-DDxPlus, and AgentClinic-MedQA, using DeepSeek-V4.1-Flash and GLM-5.3-flash as the doctor backbones. On DDxPlus, the GTPA@1 point estimates improve over the strongest baseline by 6.3 and 2.9 percentage points, respectively. The paper also includes ablations that remove alignment or bypass the gate.

Critical strengths

1.Learning and testing are clearly separated, and the results cover multiple tasks and models. Section 3.3 separates acquisition, selection, and test data, with final testing conducted after the policy is frozen. Table 1 reports better diagnostic point estimates than the listed baselines across three benchmarks and two doctor backbones. The GTPA@1 improvements on DDxPlus are 6.3 and 2.9 percentage points, respectively, indicating practical potential in the evaluated settings. 2.The ablations provide substantive evidence for combined alignment and controlled acceptance. In Table 3 on page 7, removing alignment reduces GTPA@1 from 60.5 to 55.6, while bypassing the gate reduces it to 37.9. Relative to the initial skill, the full system corrects 72 previously incorrect diagnoses and changes 28 previously correct diagnoses to incorrect ones; the corresponding counts for direct activation are 42 and 98. This analysis of case transitions shows the value of screening updates more clearly than a single aggregate score.

Critical weaknesses

1.The comparison settings and total costs need clarification. The paper specifies a common initial skill and frozen data splits, but the adaptations and evaluation budgets for SkillOpt and GEPA remain unclear.Table 3 counts only optimizer tokens, while Table 4 reports inquiry behavior. Neither establishes lower costs for the full learning process.Please provide baseline configurations and total costs measured on the same basis, or narrow the efficiency claims. 2.Evidence for safety and robustness remains limited. The authors report 28 correct-to-incorrect diagnostic transitions in Table 3, but do not assess error severity. Reusing a small selection cohort poses an overfitting risk. Sample bootstrapping and comparisons across two backbones do not capture variation in the complete learning process.Please provide existing repeated experiments and error analyses, or limit the claims to the measured diagnostic performance.

Questions for authors
  1. Which update and selection mechanisms were retained when adapting SkillOpt and GEPA to the clinical tasks? What optimizer models, accessible data, and evaluation budgets were used for each method? Apart from removing behavioral clustering and chief-complaint matching, does No aligned selection retain the same gate, candidate-rule constraints, and other procedures?
  2. After accounting for acquisition, rule proposal, all selection-cohort consultations, patient simulation, and scoring, what are the call counts, input/output token usage, and elapsed time for each method? Can these be provided from existing logs, together with clarification of whether evaluation results for the incumbent policy are cached? If not, please limit the efficiency conclusions to optimizer cost and early-stopping rates.
  3. Was the complete learning process repeated independently? How many proposals and acceptances actually occurred, and can existing results be used to provide confidence intervals for the main performance differences using pairing at the case level? If this evidence is unavailable, please distinguish uncertainty from case sampling, stability across runs, and applicability to the two backbones.
  4. Please identify where the supplementary materials provide the patient simulator, scorer, action-sequence extraction, rule triggering, initial/final skills, and prediction files. Please also specify the DDxPlus ranking rule when the gold-standard diagnosis is absent from the top-5. Have the Broken cases in Table 3 been analyzed by error type or severity? If not, please limit the safety terminology and conclusions to performance gating on the selection cohort.
Reviewer_7oLXOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3(+1)
Read review
Paper summary

This paper proposes EvoCare, a self-evolving clinical diagnosis agent that converts consultation histories into reusable inquiry policies. It contrasts aligned successful and unsuccessful trajectories to propose conditional rules and evaluates each update on a separate selection cohort before activation. Across three benchmarks and two language-model backbones, EvoCare reports the strongest point estimates among the evaluated methods, including 2.9-6.3 percentage-point gains in top-rank pathology accuracy on StreamBench-DDxPlus.

Critical strengths
  • The paper addresses the important and underexplored problem of learning transferable inquiry strategies across clinical consultations rather than treating every encounter independently.
  • EvoCare combines aligned success-failure comparisons, modular policy updates, and validation-based admission in a coherent framework that limits ungrounded strategy changes.
  • The evaluation covers three benchmarks, two language-model backbones, multiple competitive baselines, component ablations, question-budget settings, and detailed inquiry-behavior analyses.
Critical weaknesses
  • The paper reports bootstrap means and standard deviations but no paired confidence intervals or significance tests for method differences, so claims of consistent superiority are insufficiently supported, particularly when several gains are small relative to the reported uncertainty.
  • The "clinical safety gate" repeatedly evaluates up to 28 policy updates on the same small selection cohort and accepts any strictly positive mean improvement, while permitting individual cases to regress and measuring neither clinical harm nor high-risk diagnostic failures.
  • All patient interactions are simulated, so the reported gains may not transfer to real clinical consultations; moreover, the evaluation does not test robustness to alternative patient simulators or semantic-equivalence judges, making it unclear how much performance depends on the chosen simulation and evaluation pipeline.
Questions for authors

See Weaknesses.

Reviewer_cxgJOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes EvoCare, which improves a doctor agent’s inquiry strategy across consultations in interactive diagnosis without updating model weights. The strategy is a document of conditional rules. After each batch of cases, EvoCare clusters traces by behavioral similarity and reviews success–failure pairs that share chief-complaint terms. It then proposes one rule edit, which is adopted only if it achieves a strictly higher mean score than the incumbent on a disjoint selection cohort. On iMedQA, AgentClinic-MedQA and StreamBench-DDxPlus with two backbones, EvoCare has the best point estimate on all metrics against MediQ, MedAgentSim, GEPA and SkillOpt. Ablations show the gate is essential: removing it drops DDxPlus GTPA@1 from 60.5 to 37.9.

Critical strengths
  1. Propose-then-verify design with a convincing ablation. Separating rule proposal from acceptance on a disjoint cohort addresses a real failure mode of experience reuse: unvalidated lessons that cause regressions. Relative to the seed skill, the ungated variant repairs 42 test cases and breaks 98, while EvoCare repairs 72 and breaks 28. This is a useful, transferable finding, and the rule-based skill is in principle auditable.
  2. Clear gains on DDxPlus. On the largest test set, the GTPA@1 margin over the strongest baseline exceeds the reported SDs. The evaluation also goes beyond accuracy, covering question budgets, early stopping and repetition.
Critical weaknesses
  1. The evidence does not support “consistently better on every metric.” Results appear to come from a single evolution run, and the bootstrap SDs cover test-set sampling only. In 6 of the 12 cells of Table 1, including all four AgentClinic cells, EvoCare’s margin over the strongest baseline is smaller than its own SD, and no paired tests are reported. Table 5 fluctuates on the same scale; for example, EvoCare scores 72.6 on iMedQA at 25% exposure but 71.7 at 100%. Some textual claims also exceed the data. The “strongest elasticity” on DDxPlus is tied by MedAgentSim, and “lower learning overhead than competing methods” has no supporting cost comparison.
  2. The closest baseline is missing and the source of the gains is unclear. EvoClinician, the closest prior work, is run in the motivating pilot (Fig. 1f–g) but excluded from the main results. EvoCare also uses an extra selection split with repeated evaluations on it, and it is unclear whether the baselines had the same data and budget. The gate is the dominant component: removing it costs 22.6 pp, versus 4.9 pp for removing cluster alignment. Success–failure contrasting also already appears in ExpeL. The gains may therefore come largely from validation-based selection rather than from the proposed attribution.
Questions for authors
  1. Can you report results over ≥3 independent evolution runs, with paired significance tests?
  2. Did the baselines have access to the selection split? Please report the total learning cost for all methods, including evaluations on the selection cohort.
  3. Can you add EvoClinician as a baseline, and a control that applies the same gate to SkillOpt or GEPA?
  4. How many candidates were proposed and accepted, and can you show examples of learned rules? With selection cohorts of only 42–69 cases, how do you rule out acceptances driven by noise?
  5. Which models serve as the patient simulator, the AgentClinic judge, and the Review/Strategy agents?
Reviewer_QKCUOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

EvoCare updates a diagnostic agent's textual skills between cases while keeping the underlying language model fixed. It selects aligned successful and unsuccessful interactions, proposes bounded rule edits, and accepts edits through a separate selection cohort. Three diagnostic benchmarks and two model backbones are used to evaluate accuracy, questioning efficiency, and the effect of these components.

Critical strengths
  • Acquisition, selection, and test cohorts are separated.
  • Table 3 provides useful component evidence. Removing aligned selection reduces accuracy, while bypassing the acceptance gate substantially increases the number of previously correct cases that become incorrect.
Critical weaknesses
  • The experiments do not adequately separate the proposed evolution mechanism from the benefit of validation-based edit selection. Bypassing the gate lowers DDxPlus accuracy from 60.5 to 37.9, making this distinction consequential. A comparison with GEPA or SkillOpt using the same gate and total optimization budget is missing. Optimizer-token counts also omit gate evaluation, leaving the efficiency comparison incomplete.
  • The robustness and safety interpretation exceeds the evidence. Case-bootstrap error bars do not measure variation across independently repeated evolution trajectories. The gate checks mean selection-cohort accuracy, and Table 3 still reports 28 broken test cases relative to the initial skill. Figure 3 compares two question budgets; it cannot establish that each additional round resolves ambiguity or that stopping decisions reflect clinical necessity.
Questions for authors
  • How much improvement remains if GEPA or SkillOpt uses the same acceptance gate, selection cohort, and total optimization budget?
  • How sensitive are the final results to acquisition order and independently repeated evolution runs?
  • What are the complete learning costs, including candidate generation, all gate evaluations, rejected edits, and baseline optimization?
  • Which cases regress after accepted updates, and do these regressions concentrate in particular diagnoses? What evidence connects the stopping rule to diagnostic sufficiency?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.