M2U: MULTIMODAL-TO-UNIMODAL CROSS-MODAL GEOMETRY TRANSFER FOR CANCER PROGNOSIS

Multimodal models typically assume that the same modalities are available during training and deployment. In practice, however, some modalities may be available in training cohorts but missing or prohibitively costly for new patients. We study this multimodal-to-unimodal setting in cancer prognosis, where paired histopathology and molecular omics are available during training, but predictions at deployment must be made from histopathology alone. Existing approaches typically distill predictions or features, reconstruct the missing modality, or align paired samples. Here, we introduce M2U, a framework that transfers patient-to-patient similarity structure from the privileged molecular space into the histopathology space and makes this structure queryable at deployment. Its Mapping Block introduces X-GeoSCL, a cross-modal objective that uses molecular representations to define omics neighborhoods and trains histopathology representations to reproduce these soft relational targets rather than merely matching individual pairs. Its Reasoning Block uses a query histopathology representation to retrieve patients from a train-only paired gallery, reads their stored molecular embeddings through cross-attention, and combines the recovered molecular evidence with direct histopathology predictions through a learned gate. We evaluate M2U on four TCGA cancer cohorts: kidney renal clear cell carcinoma, lung adenocarcinoma, brain lower-grade glioma, and pancreatic adenocarcinoma. The complete M2U framework achieves the highest mean AUC among all deployment-compatible methods on three of the four cohorts and outperforms both evaluated multimodal-to-unimodal baselines on all four. Within the Mapping Block, X-GeoSCL achieves the highest mean downstream AUC among the evaluated mapping objectives across all four cohorts. Ablations further show that the Mapping and Reasoning Blocks play complementary roles: transferring molecular neighborhood geometry enables cross-modal retrieval, but effectively exploiting this structure requires reading the retrieved molecular representations and integrating them with direct histopathology predictions. M2U can be applied to other settings in which a privileged modality is available during training but unavailable at deployment.