Prototype-Driven Fusion of Pathology and Spatial Transcriptomics for Interpretable Multiple Instance Learning

Spatial transcriptomics (ST) measures gene expression in situ, complementing tissue morphology with spatially resolved molecular information. Together, these modalities provide rich morphological and molecular information, but identifying and integrating signals relevant to diagnosis and prognosis remains challenging under limited sample-level supervision. We introduce PathoSpatial, a multimodal multiple instance learning (MIL) framework integrating co-registered H&E images and ST through task-guided prototype learning. By combining supervised adaptation and cohort-level updates, modality-specific prototype banks capture recurring morphological and molecular patterns, which are then fused through gated attention for prediction. The prototypes make each prediction interpretable: we characterize them biologically and use Owen values to attribute each prediction to prototypes grouped by modality, linking it to the histological patches and transcriptomic spots behind it. Across three cohort-level ST datasets spanning survival prediction, treatment-response classification, and disease-status classification, PathoSpatial provides this interpretability while performing comparably to the strongest multimodal baselines, supporting the investigation of patient-specific patterns for precision medicine.