- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
EvoCare studies how frozen language models can improve clinical diagnosis under limited inquiry budgets by updating external skills between cases. The method pairs successful and failed trajectories with similar behaviors and chief complaints and proposes local rules with applicability conditions.The rule must apply to at least one selection case. The candidate policy must also strictly improve the mean diagnostic score on an independent selection cohort to be accepted.The policy is then frozen for testing. The paper compares agent and policy optimization methods on three benchmarks, iMedQA, StreamBench-DDxPlus, and AgentClinic-MedQA, using DeepSeek-V4.1-Flash and GLM-5.3-flash as the doctor backbones. On DDxPlus, the GTPA@1 point estimates improve over the strongest baseline by 6.3 and 2.9 percentage points, respectively. The paper also includes ablations that remove alignment or bypass the gate.
Critical strengths
1.Learning and testing are clearly separated, and the results cover multiple tasks and models. Section 3.3 separates acquisition, selection, and test data, with final testing conducted after the policy is frozen. Table 1 reports better diagnostic point estimates than the listed baselines across three benchmarks and two doctor backbones. The GTPA@1 improvements on DDxPlus are 6.3 and 2.9 percentage points, respectively, indicating practical potential in the evaluated settings. 2.The ablations provide substantive evidence for combined alignment and controlled acceptance. In Table 3 on page 7, removing alignment reduces GTPA@1 from 60.5 to 55.6, while bypassing the gate reduces it to 37.9. Relative to the initial skill, the full system corrects 72 previously incorrect diagnoses and changes 28 previously correct diagnoses to incorrect ones; the corresponding counts for direct activation are 42 and 98. This analysis of case transitions shows the value of screening updates more clearly than a single aggregate score.
Critical weaknesses
1.The comparison settings and total costs need clarification. The paper specifies a common initial skill and frozen data splits, but the adaptations and evaluation budgets for SkillOpt and GEPA remain unclear.Table 3 counts only optimizer tokens, while Table 4 reports inquiry behavior. Neither establishes lower costs for the full learning process.Please provide baseline configurations and total costs measured on the same basis, or narrow the efficiency claims. 2.Evidence for safety and robustness remains limited. The authors report 28 correct-to-incorrect diagnostic transitions in Table 3, but do not assess error severity. Reusing a small selection cohort poses an overfitting risk. Sample bootstrapping and comparisons across two backbones do not capture variation in the complete learning process.Please provide existing repeated experiments and error analyses, or limit the claims to the measured diagnostic performance.
Questions for authors
- Which update and selection mechanisms were retained when adapting SkillOpt and GEPA to the clinical tasks? What optimizer models, accessible data, and evaluation budgets were used for each method? Apart from removing behavioral clustering and chief-complaint matching, does No aligned selection retain the same gate, candidate-rule constraints, and other procedures?
- After accounting for acquisition, rule proposal, all selection-cohort consultations, patient simulation, and scoring, what are the call counts, input/output token usage, and elapsed time for each method? Can these be provided from existing logs, together with clarification of whether evaluation results for the incumbent policy are cached? If not, please limit the efficiency conclusions to optimizer cost and early-stopping rates.
- Was the complete learning process repeated independently? How many proposals and acceptances actually occurred, and can existing results be used to provide confidence intervals for the main performance differences using pairing at the case level? If this evidence is unavailable, please distinguish uncertainty from case sampling, stability across runs, and applicability to the two backbones.
- Please identify where the supplementary materials provide the patient simulator, scorer, action-sequence extraction, rule triggering, initial/final skills, and prediction files. Please also specify the DDxPlus ranking rule when the gold-standard diagnosis is absent from the top-5. Have the Broken cases in Table 3 been analyzed by error type or severity? If not, please limit the safety terminology and conclusions to performance gating on the selection cohort.