Beyond Presentation: Toward Format-Robust Medical Evidence Reasoning in LLMs
Abstract
Medical evidence-source preference, where LLMs systematically favor certain evidence sources when reasoning about competing diagnoses, is an underexplored factor in reliable medical reasoning. However, whether such preferences reflect evidential content or are confounded by evidence presentation remains unclear. To investigate this question, we build PMC-ConForm, a benchmark of medical evidence conflicts from PMC-Patients case reports where evidence source and presentation format vary independently. Through detailed empirical analyses, we find that LLMs exhibit systematic medical source preferences, yet these preferences can shift substantially and even reverse under content-preserving format changes, indicating that evidence prioritization partly reflects presentation rather than evidence itself. To address this issue, we propose FORMEDER, a cross-format consensus self-distillation framework for format-robust medical evidence reasoning. We re-render the same evidence into multiple content-preserving formats and aggregate the frozen model's predictions into a soft consensus target shared across presentations. We then distill the model toward this target under each format while introducing a contraction objective to suppress residual format-dependent variation. The framework requires no external annotation and uses a single presentation at inference. Extensive experiments on eight LLMs demonstrate that FORMEDER substantially mitigates format-induced preference shifts and enhances cross-format choice consistency while faithfully preserving presentation-invariant judgments, and generalizes to unseen presentation formats.
Then back it, or bet against it.
Related papers
Open the market on this paper to see 7 more related papers.