- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes MedGuideEvolve, a framework for learning transferable clinical experience from historical patient cases under established diagnostic guidelines. The method contrasts failed cases, correctly diagnosed cases, and hard controls to construct evidence-linked experience cards with explicit applicability conditions. These cards are admitted into an experience library only after guideline-consistency checks and patient-disjoint validation, and are subsequently retrieved to support diagnosis of unseen patients. Experiments on CardioRareBench show consistent improvements over several experience-based baselines across three LLM backbones.
Critical strengths
-
The paper tackles an interesting problem in clinical experience reuse. Simply retrieving similar patient cases or abstracting successful trajectories can introduce misleading patterns. Learning conditional experience with explicit applicability boundaries is a sensible direction, particularly for rare-disease diagnosis.
-
The experimental results are encouraging. MedGuideEvolve achieves substantial improvements over the base models across three backbones. The patient-disjoint temporal evaluation and comparisons with several recent memory-based methods also provide useful evidence for the effectiveness of the overall framework.
Critical weaknesses
-
The contributions of several key components are not sufficiently isolated. Table 2 compares guideline-only, experience-only, and full settings, but does not separately evaluate contrastive induction, hard controls, validation-gated admission, or hierarchical routing. Since these are central to the proposed method, it is difficult to tell whether the gains come from the experience-evolution mechanism itself or simply from providing additional structured clinical knowledge.
-
I am not fully convinced that the admission criteria are strong enough to prevent harmful experience transfer. The method requires only three injected probes per active card, positive Top-3 net repair, and non-negative Top-1 net repair. These seem like rather weak criteria for establishing reliable transfer, especially given the limited development data. Non-negative net repair also allows individual patient regressions to be offset by improvements elsewhere. I would be interested in seeing how often candidate cards fail these checks and how sensitive the results are to the admission thresholds.
Questions for authors
-
Could the authors provide ablations removing hard controls, the admission gate, and hierarchical routing separately? This would help clarify which components contribute most to the reported gains.
-
How many experience cards were proposed, admitted, and rejected during evolution? How sensitive is the performance to the minimum probe count and paired-repair thresholds?
-
Are the improvements over the strongest baselines, particularly ExpeL and ExpRAG, statistically significant? It would also be useful to see whether the gains remain substantial when Marfan syndrome is excluded.
-
Could the authors explain why the full method achieves lower Macro-F1 than experience-only on Qwen3.6-27B? Are there particular disease categories where adding guideline constraints hurts performance?