LFUF: Context-Guided Lévy Flow Recovery and Uncertainty-Aware Evidence Fusion for Incomplete Multimodal Emotion Recognition
Abstract
Multimodal Emotion Recognition in Conversations (MERC) aims to infer utterance-level emotions from complementary linguistic, acoustic, and visual cues. In real-world scenarios, however, one or more modalities may be unavailable, leading to substantial information loss. Moreover, multimodal observations are inherently heterogeneous in informativeness and reliability, and different modalities associated with the same utterance may provide inconsistent or even conflicting emotional evidence. Consequently, directly recovering and fusing multimodal representations can propagate unreliable information and degrade prediction robustness. To address these challenges, we propose Lévy Flow and Uncertainty-Aware Fusion (LFUF), a unified framework for context-guided missing-modality recovery and reliability-aware evidence fusion. LFUF first exploits speaker identities and conversational dependencies to construct contextual priors, which, together with the available modalities, condition a Lévy-stable flow-matching process for recovering missing representations. The observed and recovered representations are then converted into complementary evidence sources rather than being fused indiscriminately. To account for source reliability, LFUF quantifies inter-source consistency using the Jousselme distance and adaptively discounts unreliable evidence by reallocating part of its belief mass to the uncertainty set. Finally, local conflicts are explicitly resolved through PCR6-based evidence redistribution, yielding a conflict-aware fused belief for emotion prediction. Extensive experiments under diverse missing-modality settings demonstrate that LFUF consistently improves emotion recognition performance and exhibits strong robustness to incomplete and conflicting multimodal observations.
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.