Retrieval-Augmented Anatomy-Aware Reasoning for Interpretable Alzheimer's Diagnosis

Accurate Alzheimer's disease (AD) diagnosis from structural MRI (sMRI) increasingly relies on deep learning, yet existing models often provide limited insight into the anatomical evidence underlying their predictions. Recent vision–language approaches improve interpretability by generating diagnostic rationales, but global image–text alignment does not guarantee that anatomy-related statements are grounded in the corresponding brain regions, leading to statement–region attention drift. We propose RAR, a retrieval-augmented anatomy-aware reasoning framework that explicitly connects anatomical evidence, diagnostic reasoning, and prediction. RAR comprises three components: an Anatomy-Aware Visual Encoder that guides visual attention toward AD-critical regions using subject-specific anatomical priors; a Retrieval-Augmented Reasoning Module that enriches the current sMRI representation with compact rationale context from relevant training cases; and an Anatomy-Grounded Rationale Generator that aligns anatomy-related rationale tokens with their supporting brain regions through token-specific anatomical priors. Experiments on ADNI, NACC, OASIS-3, and the external AIBL cohort demonstrate competitive diagnostic performance, improved rationale quality, and a 20.6-point gain in token–region grounding Dice. These results support explicitly linking regional imaging evidence to diagnostic reasoning for more interpretable AD diagnosis. Code will be made publicly available.