PathWeaver: Enhancing Diagnostic Reasoning in Pathology Vision-Language Models with Explicit Reasoning Graphs

Pathology vision-language models (VLMs) increasingly generate diagnostic rationales, yet these rationales are typically expressed as free-form text, making it difficult to verify or supervise individual claims and to trace the chain of evidence that leads to a conclusion. We introduce PathWeaver, a framework that makes the diagnostic reasoning behind these rationales structured, explicit, and learnable. It represents each case as a directed acyclic graph (DAG) that links morphological observations, intermediate inferences, and diagnostic conclusions through typed relations. A DAG generator is trained on graphs curated from heterogeneous pathology QA sources and refined by node-level preference optimization, in which counterfactual pairs differ in a single node so that the preference signal localizes to one claim. A graph-conditioned response model then encodes the DAG with relation-aware graph attention to guide answer generation. PathWeaver outperforms general-purpose, medical, and pathology-specialized VLMs across pathology QA formats (66.6% multiple-choice, 55.5% spatial-locating accuracy) and on PathMMU (71.0%). Their explicit structure also enables composition: training-free graph weaving integrates DAGs across patches, improving PanCancer histologic diagnosis benchmark by 10.8% over the best single-view baseline. These results show that an explicit reasoning graph can serve as a single unit of supervision, conditioning, and composition, supporting transparent and inspectable AI-assisted diagnostic reasoning and laying a foundation for more trustworthy collaboration between pathology VLMs and pathologists.