A Semantically Aligned and Enriched Vision-Language Model for Breast Cancer Diagnosis on Routine Chest CT

Routine chest CT contains the breast tissue, however, physicians often omit breast findings due to their low contrast in CT scans. In this work, we explore whether the weakly-supervised vision-language contrastive learning provides scalable opportunistic breast cancer diagnosis using routine chest CT and reports. First, a benchmark dataset is constructed (BreastCT-12K), which comprises 12,696 patients with routine chest CT scans, CT reports, and temporally matched breast MRI reports. Since CT reports often lack breast description, we curate MRI reports to retain CT-visible findings to enrich the paired CT reports. We then develop a Report-Enriched and Aligned CHest CT vision-language model for Breast cancer diagnosis (**REACH-breast**). Since whole-volume contrastive pretraining may overlook small and side-specific abnormalities of breast, REACH-breast introduces two modules to enhance fine-grained feature alignment and representation: 1) Breast-Centric Hierarchical Alignment (BCHA) matches the left, right, and bilateral breast representations with corresponding text descriptions and bridges the bilateral and global context, preserving laterality while making whole-volume features breast-aware. 2) Side-aware Evidence–Patch Association (SEPA) links report evidence to specific breast-region within patches, strengthening fine-grained visual–semantic correspondence without lesion-level annotations. Using chest CT and fixed class prompts at inference, REACH-breast achieves 89.19% AUC for zero-shot malignancy classification on the test split of BreastCT-12K, significantly exceeding the state-of-the-art breast-adapted zero-shot foundation models by at least 7.08%. It also yields the best performance in bidirectional CT-report retrieval and report generation tasks. External evaluation confirms superior performance of REACH-breast. These results demonstrate the effectiveness of vision-language modeling for opportunistic breast assessment on routine chest CT.