Uncertainty-Aware Evidential Prototype Reconstruction for Robust Multimodal Federated Learning

Multimodal Federated Learning (MMFL) facilitates privacy-preserving collaboration across distributed multimodal data, yet existing methods universally assume clean labels. Real-world federated ecosystems, however, exhibit label noise compounded by modality heterogeneity, with structural patterns unique to multimodal settings: semantic mismatch across modalities, asymmetric corruption affecting individual modalities, and consistent label errors where coherent modalities collectively indicate incorrect labels. We propose FedEPR, a phased reconstruction framework grounded in evidential uncertainty quantification. By reformulating classification within Dempster-Shafer theory, our approach models epistemic uncertainty via Dirichlet distributions, enabling principled detection of inter-modal disagreement through a modal conflict factor. A globally-guided prototype calibration mechanism further anchors local representations to server-aggregated semantics, counteracting prototype drift under Non-IID distributions, while modality-decoupled evidential aggregation adaptively handles missing modalities and suppresses high-variance gradient updates from noisy clients. Extensive experiments demonstrate that FedEPR outperforms state-of-the-art methods by over 12% under severe noise conditions.