RPMF: Reliability-Aware Progressive Multimodal Fusion Framework for Multimodal Brain Disease Diagnosis

Multimodal learning has emerged as a promising paradigm for brain disease diagnosis by integrating complementary information from heterogeneous clinical modalities. However, the diagnostic reliability of different modalities varies substantially across individual subjects, suggesting that multimodal data should be integrated adaptively according to their reliability. Nevertheless, many existing multimodal methods estimate modality importance only implicitly and directly fuse all modalities in a single step, limiting their ability to prioritize reliable modalities while suppressing noisy or conflicting information. To address these limitations, we propose a Reliability-aware Progressive Multimodal Fusion (RPMF) framework for brain disease diagnosis. Specifically, RPMF first learns modality-specific representations and explicitly estimates subject-specific modality reliability using an entropy-based uncertainty formulation, allowing the model to quantify the diagnostic confidence of each modality. To characterize heterogeneous disease patterns, each modality is further represented by a set of latent prototypes, from which cross-modal compatibility is evaluated through prototype-level relation modeling. Guided by both reliability and compatibility, multimodal information is progressively integrated in a reliability-aware manner, where the most reliable modality serves as the initial anchor and subsequent modalities are sequentially incorporated according to the reliability and compatibility with the current fused representation. This progressive strategy enables the model to adaptively exploit useful modalities that are compatible with the current multimodal representation while suppressing noisy or conflicting information. Extensive experiments on multiple brain disease datasets demonstrate that RPMF consistently outperforms state-of-the-art methods.