acceptodds
Under review as a conference paper at ICLR 2027

MedGuideEvolve: Evolving Transferable Clinical Experience Cards from Cases under Established Guidelines

Abstract

Diagnosing rare cardiovascular diseases is a complex clinical reasoning task that requires integrating clinical evidence across time and specialties and differentiating among potential etiologies underlying overlapping phenotypes. How to distill guideline-application experience that transfers across patients from multi-case clinical trajectories while remaining faithful to an established clinical guideline remains a key unresolved problem in the field. Direct case reuse relies on inter-patient phenotypic similarity and therefore struggles to determine whether phenotypic similarity implies a transferable rationale for guideline application, while experience abstraction can overfit limited cases by encoding incidental patterns and losing the original evidentiary context and applicability conditions during generalization. We propose MedGuideEvolve, which compares erroneous cases, similar correctly diagnosed cases, and hard controls under an established clinical guideline to distill conditional guideline-application experience with explicit evidence links and applicability conditions. Candidate experience bundles are retained only when they satisfy schema, provenance, guideline, applicability, audit, support, and paired-repair gates, including non-negative paired Top-1 net repair on independent patients held out from experience formation. This admission boundary limits overgeneralization from incidental patterns while leaving final diagnostic adjudication to the established clinical guideline. On 217 label-blinded cases from a temporally held-out test set, MedGuideEvolve improves Qwen3.8-27B's Top-1 accuracy over Base by 23.96 percentage points to 82.95%, demonstrating that conditional guideline-application experience distilled within established guideline constraints can transfer to unseen patients while retaining provenance.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Rare Cardiovascular Diseases, LLM Agents, Clinical Reasoning, Large language models

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_p8sAOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes MedGuideEvolve, a framework for learning transferable clinical experience from historical patient cases under established diagnostic guidelines. The method contrasts failed cases, correctly diagnosed cases, and hard controls to construct evidence-linked experience cards with explicit applicability conditions. These cards are admitted into an experience library only after guideline-consistency checks and patient-disjoint validation, and are subsequently retrieved to support diagnosis of unseen patients. Experiments on CardioRareBench show consistent improvements over several experience-based baselines across three LLM backbones.

Critical strengths
  • The paper tackles an interesting problem in clinical experience reuse. Simply retrieving similar patient cases or abstracting successful trajectories can introduce misleading patterns. Learning conditional experience with explicit applicability boundaries is a sensible direction, particularly for rare-disease diagnosis.

  • The experimental results are encouraging. MedGuideEvolve achieves substantial improvements over the base models across three backbones. The patient-disjoint temporal evaluation and comparisons with several recent memory-based methods also provide useful evidence for the effectiveness of the overall framework.

Critical weaknesses
  • The contributions of several key components are not sufficiently isolated. Table 2 compares guideline-only, experience-only, and full settings, but does not separately evaluate contrastive induction, hard controls, validation-gated admission, or hierarchical routing. Since these are central to the proposed method, it is difficult to tell whether the gains come from the experience-evolution mechanism itself or simply from providing additional structured clinical knowledge.

  • I am not fully convinced that the admission criteria are strong enough to prevent harmful experience transfer. The method requires only three injected probes per active card, positive Top-3 net repair, and non-negative Top-1 net repair. These seem like rather weak criteria for establishing reliable transfer, especially given the limited development data. Non-negative net repair also allows individual patient regressions to be offset by improvements elsewhere. I would be interested in seeing how often candidate cards fail these checks and how sensitive the results are to the admission thresholds.

Questions for authors
  • Could the authors provide ablations removing hard controls, the admission gate, and hierarchical routing separately? This would help clarify which components contribute most to the reported gains.

  • How many experience cards were proposed, admitted, and rejected during evolution? How sensitive is the performance to the minimum probe count and paired-repair thresholds?

  • Are the improvements over the strongest baselines, particularly ExpeL and ExpRAG, statistically significant? It would also be useful to see whether the gains remain substantial when Marfan syndrome is excluded.

  • Could the authors explain why the full method achieves lower Macro-F1 than experience-only on Qwen3.6-27B? Are there particular disease categories where adding guideline constraints hurts performance?

Reviewer_neRvOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper looks at diagnosing rare cardiovascular diseases with LLMs, where the model reads a patient record and ranks the most likely diseases, and asks how lessons from past cases can be reused without the model copying surface details or drifting from clinical guidelines. The proposed method, MedGuideEvolve, writes short "experience cards" by comparing cases the model got wrong, similar cases it got right, and look-alike cases with a different diagnosis. A card is kept only if it doesn't reduce accuracy on held-out practice patients. At test time, cards are inserted into the steps of a clinical guideline, which makes the final decision. Tested on a new private dataset from one hospital with LLMs, the method beats the baselines on all models, with the largest gains on Marfan syndrome.

Critical strengths

Tested on a new private dataset from one hospital with three LLMs, the method beats the baselines on all models, with the largest gains on Marfan syndrome.

Critical weaknesses

The comparison may be unfair: the method gets clinical guidelines, and the baselines don't. MedGuideEvolve uses disease-specific guidelines that make the final call. The baselines are described as receiving only "the same de-identified, label-blinded patient case" (lines 904–905). Guidelines alone add a lot. On Qwen3.8, guideline-only raises Top-1 from 58.99 to 69.59 (Table 2). without running the baselines with the same guidelines, it's unclear how much of the gain comes from the experience cards, which are the paper's main contribution.

Questions for authors

Did any baseline in Table 1 receive the same guideline text as MedGuideEvolve? If not, could you report the baselines run with the same guidelines in context? This would show how much of the gain comes from the experience cards rather than the guidelines.

Reviewer_JK2POpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This papers proposes a method in learning reusable “experience cards” from clinical cases to extract features within the case that are transferable across future clinical scenarios. This is done by contrasting multiple cases (correct vs error vs control) in order to extract their differences, and these experiences are then added against established guidelines, thus generating a full pipeline for clinical diagnosis support. The method is trained and evaluated on a cardiovascular rare-disease dataset across three LLM backbones.

Critical strengths
  1. Problem formulation - Highlighting the distinction between guideline-defined diagnostic criteria and experience-guided evidence review is interesting, because empirically there are cases where a diagnosis is made that are not entirely according to guidelines, so the problme of how to learn from clinical experience is an adequate one.

  2. Numerically, the headline results appear strong, with significant improvements over base model across three model backbones. The paper also included numerous baselines for comparison, and a guideline-versus-experience ablation. However, the validity of these improvements depends on resolving several methodological concerns (see weakness / questions section below)

Critical weaknesses
  1. The evaluation is limited to a single dataset of cardiovascular rare diseases, with six disease categories in test set, and models were asked to output exactly five diagnostic candidates. From reading the paper, I am unclear whether the evaluated task is classification over those diseases, or open-ended differential diagnosis. Irrespectively, the results seem to suggest transfer to unseen patients within a narrow disease distribution, but at this current stage does not establish broader clinical generalization. Most signficantly, the experience library could be improving recognition of the benchmark diseases while biasing the model toward those diagnoses in more common cardiac presentations

  2. The methodology and experimental setup are unclear, e.g. paper did not cite where they got their guidelines sources were, or any actual examples of experience cards. In addition, the authors did not really explain how the guideline graphs, projection, evidence evaluation, or adjudication are implemented in depth (no code provided either). Moreover, the baseline descriptions do not establish whether other methods received equivalent information to guidelines or workflow resources etc. Consequently, the gains may partly reflect additional knowledge injection or a different reasoning pipeline, rather than the proposed mechanism of experience evolution

Questions for authors
  1. Where do the guidelines for each disease come from? Please give the sources and versions, e.g. the source of the Marfan disease workflow. How were the guideline graphs constructed, and were they clinically validated?

  2. Do the baselines receive the same guidelines and comparable inference budgets? The guideline-only ablation is helpful, but it does not show whether the advantage over other memory methods comes from experience evolution or additional guideline access. Would be great if the authors can detail the prompt / information given to other baselines

  3. The prompt asks for five diagnoses without listing answer options, but scoring uses a fixed ontology. What is that ontology, and are predictions restricted to it? How are diagnoses outside the benchmark categories and disease subtypes handled? This is most relevant as one of the key metrics is recall@5, and there are only six evaluation categories.

  4. The temporal split shows transfer to new patients, but does the library generalize beyond these disease groups? Evaluation on another dataset, e.g. one containing common cardiac presentations and alternative diagnoses, would help test whether the cards improve reasoning or simply increase selection of the benchmarked rare diseases.

  5. Could you show a few actual admitted cards, including their predicates, review actions, hard controls, and validation results?

  6. How many candidate bundles and admission rounds were evaluated, and how were the 128 Dev cases divided into fresh probe batches? Please also report uncertainty for the improvements over the strongest baselines and ablate the contrastive induction, admission gates, and routing.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.