acceptodds
Withdrawn from ICLR 2027

LFUF: Context-Guided Lévy Flow Recovery and Uncertainty-Aware Evidence Fusion for Incomplete Multimodal Emotion Recognition

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Multimodal Emotion Recognition in Conversations (MERC) aims to infer utterance-level emotions from complementary linguistic, acoustic, and visual cues. In real-world scenarios, however, one or more modalities may be unavailable, leading to substantial information loss. Moreover, multimodal observations are inherently heterogeneous in informativeness and reliability, and different modalities associated with the same utterance may provide inconsistent or even conflicting emotional evidence. Consequently, directly recovering and fusing multimodal representations can propagate unreliable information and degrade prediction robustness. To address these challenges, we propose Lévy Flow and Uncertainty-Aware Fusion (LFUF), a unified framework for context-guided missing-modality recovery and reliability-aware evidence fusion. LFUF first exploits speaker identities and conversational dependencies to construct contextual priors, which, together with the available modalities, condition a Lévy-stable flow-matching process for recovering missing representations. The observed and recovered representations are then converted into complementary evidence sources rather than being fused indiscriminately. To account for source reliability, LFUF quantifies inter-source consistency using the Jousselme distance and adaptively discounts unreliable evidence by reallocating part of its belief mass to the uncertainty set. Finally, local conflicts are explicitly resolved through PCR6-based evidence redistribution, yielding a conflict-aware fused belief for emotion prediction. Extensive experiments under diverse missing-modality settings demonstrate that LFUF consistently improves emotion recognition performance and exhibits strong robustness to incomplete and conflicting multimodal observations.

Primary area:
generative models
Keywords:
Incomplete Multimodal Learning, Flow Matching, Lévy Processes, Missing Modality Recovery, Multimodal Emotion Recognition, Uncertainty Quantification, Evidential Deep Learning, Evidence Fusion

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_xgmVOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes LFUF for incomplete multimodal emotion recognition in conversations. It combines context-conditioned recovery from a heavy-tailed source with Shortcut sampling, then applies consistency-based evidence discounting and PCR6 conflict redistribution. Experiments on IEMOCAP and MELD report improved recognition performance, with additional analyses of source distributions, fusion rules, and recovery stability.

Critical strengths
  1. Jointly addressing missing-modality recovery and the reliability of recovered evidence is well motivated. LFUF explicitly separates representation completion from evidence discounting and conflict redistribution, providing a useful framework for examining how recovery errors affect downstream emotion prediction.

  2. The appendices provide meaningful component-level analysis. The Gaussian-source comparison, controlled combinations of discounting and fusion rules, and Shortcut analysis help assess individual contributions beyond the final task scores. Appendix A also explicitly acknowledges the limits of the theoretical guarantees, including the potential bias introduced by robust regression.

Critical weaknesses
  1. HARDY-MER [1], ComP [2], and BALM [3] address emotion recognition under missing modalities but are absent from the comparison. Including these methods under matched evaluation settings, or explaining the relevant protocol differences, would strengthen the empirical assessment. Moreover, the motivation for heavy-tailed recovery is not sufficiently validated. The paper assumes that missing modalities induce heavy-tailed latent deviations, but provides no direct analysis supporting this assumption. Although Appendix B compares different values of α\alpha, it mainly reports downstream classification performance. Figure 3 provides qualitative visualization rather than quantitative evidence of improved reconstruction fidelity. Reporting per-modality reconstruction errors and distributional similarity would help establish whether the proposed recovery mechanism delivers the claimed benefits.

  2. Eq. (12) score γs\gamma_s measures agreement among seven evidence sources derived from the same three modalities, so correlated errors can receive strong consensus support while a correct minority source is discounted. Theorem 2 establishes contraction and conflict conservation, but does not establish that the score identifies unreliable sources. Appendix C demonstrates classification gains from the fusion choices, rather than validating source reliability or uncertainty quality. This leaves an important part of the claimed robustness mechanism insufficiently supported.

Refs :

[1] Hardness-Aware Dynamic Curriculum Learning for Robust Multimodal Emotion Recognition with Missing Modalities, Rui Liu, et al.

[2]Cross-modal Prompting for Balanced Incomplete Multi-modal Emotion Recognition,Wen-Jue He, et al.

[3] BALM: A Model-Agnostic Framework for Balanced Multimodal Learning under Imbalanced Missing Rates, Phuong-Anh Nguyen, et al.

Questions for authors

Clarification requests:

  1. What empirical evidence supports the assumption that missing modalities induce heavy-tailed latent deviations? Please clarify which deviations this assumption refers to and whether their distribution was examined.

  2. Are there existing per-modality reconstruction metrics comparing the L'evy and Gaussian sources, beyond downstream classification scores and the t-SNE visualization? If available, do these results support the claimed improvement in reconstruction fidelity?

  3. Do the existing analyses show whether γs\gamma_s tracks source correctness or reconstruction error? In particular, is there evidence that discounting preserves a correct source when other sources share an incorrect prediction?

Suggestion for revision:

  1. Please broaden the related-work discussion and baseline coverage, particularly for recent AAAI and ACM MM methods such as HARDY-MER [1] and ComP [2]. Additional comparisons under matched evaluation settings would help establish the contribution relative to existing approaches and strengthen the empirical conclusions. Comparisons with very recent CVPR 2026 methods can be considered optional.
Reviewer_8tuiOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

LFUF addresses incomplete multimodal emotion recognition in conversations by conditioning a deterministic flow—initialized from a clipped symmetric alpha-stable source—on observed modalities and bidirectional context, recovering missing representations via a robust regression objective and EMA-based shortcut self-consistency in few steps. The completed representations yield seven evidence sources discounted by Jousselme agreement and fused through a fixed binary PCR6 tree. On IEMOCAP (four-/six-class) and MELD (missing rates 0.0–0.7), LFUF outperforms six baselines, with ablations, sensitivity, and stability analyses.

Critical strengths
  1. Table 1 shows LFUF beats all baselines in every setting, with IEMOCAP4 gains of 2.6–6.1 points over HyperEF; Table 2 confirms the value of contextual conditioning and modality recovery.

  2. Appendix C varies discounting and the combination rule separately, showing discounted PCR6 gains ~1.3/1.7 accuracy/F1 points over undiscounted DST on MELD.

Critical weaknesses
  1. The stated missing-rate definition (κ ≤ 2/3) is incompatible with the reported κ = 0.7, and mask generation, correction, and effective rates are unspecified.

  2. Repeated-run variability, baseline rerun parity (features, masks, context), and architecture/model-selection/fusion-tree details are unreported, limiting reproducibility.

  3. The MELD gains over HyperEF at nominal rates 0.6/0.7 are only 0.2/0.1 points, and IEMOCAP's single held-out session with non-independent four-/six-class versions further weakens the robustness claim.

  4. Eq. (3)'s conditional-mean velocity differs from what the robust objective (Eqs. 4–5) actually estimates, and the unmeasured, unbounded transport-anchoring error η_T leaves Theorem 1 an error-propagation result, not evidence of learned conditional transport.

  5. The Gaussian source comparison uses p = 1.05 instead of the default 1.5, so heavy-tail benefit is not established under default settings.

  6. Eq. (12) measures agreement, not correctness—overlapping sources can share errors while a correct minority source is discounted—and no calibration, error-detection, or controlled-conflict validation is provided.

  7. Table 5 reports cross-step loss variation at eight evaluations rather than end-to-end efficiency or recovery fidelity at the default four steps.

Questions for authors

N/A

Reviewer_NbxCOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies incomplete multimodal emotion recognition and proposes LFUF, a framework combining context-guided Lévy flow recovery with uncertainty-aware evidence fusion. The method uses conversational context to recover missing modality representations and performs multimodal fusion through source reliability estimation and conflict handling. The authors conduct experiments on IEMOCAP4, IEMOCAP6, and MELD, reporting better recognition performance than the compared methods.

Critical strengths

S1. The research problem has practical relevance. Missing modalities are an important challenge in multimodal emotion recognition. Since recovered modality representations may contain errors, jointly studying missing-modality recovery and evidence reliability is well motivated. S2. The method forms a coherent framework. LFUF combines context-guided missing-modality recovery with uncertainty-aware evidence fusion. It considers both inter-source consistency and conflict, resulting in a coherent processing framework. S3. The experimental results are competitive. Table 1 shows that LFUF outperforms the compared methods at every reported missing-rate level on IEMOCAP4, IEMOCAP6, and MELD, with relatively substantial gains in some IEMOCAP settings. The authors also provide a Gaussian-source comparison, sensitivity analysis of the robust penalty exponent, and ablations of reliability discounting and evidence combination rules. Table 2 and Appendix D further examine the main components and the Shortcut recovery mechanism, respectively.

Critical weaknesses

W1. The performance benefit of the heavy-tailed source under the main experimental configuration still requires direct evaluation. Appendix B examines the effect of the stability parameter α, including the Gaussian-source case at α=2.0. However, this experiment fixes p=1.05, whereas the main experiments use p=1.5. Using p=1.05 throughout the sweep has a valid design rationale because it satisfies 1<p<α across the evaluated parameter range, but it does not directly establish the performance difference between source distributions under the main configuration. Table 3 additionally studies sensitivity to p with α=1.8 on MELD at a missing rate of 0.5. The performance point estimates vary relatively little in this fixed setting, but this does not directly replace a Lévy-versus-Gaussian source comparison under the main experimental configuration. The existing experiments therefore provide some support for the heavy-tailed source, but a direct comparison under the main settings would clarify its performance benefit. W2. The consistency-based reliability measure lacks direct validation against source-level prediction correctness. Equation (12) computes source reliability coefficients from the Jousselme distances between evidence sources. However, agreement between sources does not necessarily imply prediction correctness. When multiple sources make correlated errors, they may support one another, while a correct source that disagrees with the others may receive stronger discounting. Table 4 in Appendix C already compares reliability discounting and different evidence combination rules, showing that the proposed mechanism improves overall classification performance. The concern is therefore not a lack of classification evidence, but the absence of a direct analysis relating the reliability coefficients to the actual prediction correctness of individual evidence sources. Such an analysis would strengthen the interpretation that the mechanism identifies unreliable evidence.

Questions for authors

Q1. Could the authors confirm whether the baseline results in Table 1 use the same data splits as LFUF and whether these results were obtained by rerunning the methods or taken from the original papers? Please also explain how the 0.7 missing-rate level is generated, its actual missing proportion, and whether all methods are evaluated under comparable effective modality-missing conditions. Q2. Appendix B compares different α values using p=1.05. Could the authors additionally compare α=1.8 and α=2.0 under the main setting of p=1.5? Q3. Could the authors analyze the relationship between the reliability coefficients in Eq. (12) and the actual prediction correctness of individual evidence sources? If feasible, please also examine the mechanism’s behavior when multiple evidence sources make correlated errors.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.