- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes Calibrated Dynamic Fusion (CDF) for robust multimodal sentiment analysis. It combines Degradation Source Rectification (DSR), which uses unimodal supervision and gradient decoupling to reduce optimization-induced representation bias, with Multi-Peak Uncertainty Modeling (MPUM), which employs sample-conditional Gaussian mixtures to estimate within-component and between-component predictive uncertainty. The resulting uncertainty estimates guide sample-adaptive modality fusion. Experiments on CMU-MOSI, CMU-MOSEI, and MVSA-Single report competitive performance, supported by component ablations, calibration analyses, and controlled degradation experiments.
Critical strengths
-
The paper addresses a meaningful issue in uncertainty-aware multimodal learning by distinguishing representation degradation caused by insufficient optimization from that caused by corrupted inputs.
-
The experimental analysis provides useful support for the proposed framework.
Critical weaknesses
-
In Eqs. (11)–(13), the proposed aleatoric and predictive uncertainty terms correspond to the within-component and between-component variance of a Gaussian mixture. While this decomposition is mathematically valid, between-component disagreement does not necessarily represent uncertainty about modality quality, and a larger predictive variance does not always imply less task-relevant information. Although Table 3 reports improved expected calibration error and Figure 4 shows sensible uncertainty trends under image corruption, these results do not directly establish whether the estimated uncertainty can reliably distinguish informative from degraded modalities at the sample level. In particular, the ECE evaluation concerns predictive calibration rather than the correctness of the modality reliability rankings used in fusion. I would like to see more targeted analyses, such as evaluating the relationship between estimated uncertainty and unimodal prediction errors under matched corruption conditions, or comparing uncertainty-based modality rankings against independently assessed modality quality.
-
The main performance gains are established on the tri-modal CMU-MOSI and CMU-MOSEI benchmarks, whereas the corruption experiments are conducted primarily on the image-text MVSA-Single dataset. This leaves an important gap in evaluating whether DSR and MPUM improve reliability-aware fusion when acoustic and visual quality vary in the intended tri-modal setting. Furthermore, the comparisons omit closely related recent approaches such as EBMC [1], which addresses modality competition and instance-level reliability estimation, and QA-MoE [2], which uses uncertainty-aware expert routing for robust sentiment analysis.
Refs:
[1] Kang He et al. Enhance-then-Balance Modality Collaboration for Robust Multimodal Sentiment Analysis. CVPR 2026.
[2] Yitong Zhu et al. QA-MoE: Towards a Continuous Reliability Spectrum with Quality-Aware Mixture of Experts for Robust Multimodal Sentiment Analysis. ACL 2026.
Questions for authors
-
How do the authors establish that the within-component and between-component variances of MPUM accurately reflect modality reliability rather than general predictive ambiguity? Is there evidence that lower estimated uncertainty consistently corresponds to more informative or more accurate unimodal predictions under comparable corruption conditions?
-
Table 3 reports improvements in ECE, while the fusion weights are computed from the mixture uncertainty in Eq. (13). How closely do these uncertainty estimates align with independently measured modality quality or prediction errors, beyond conventional probability calibration?
-
Appendix A.3.3 demonstrates that under-optimization and input corruption can produce similar uncertainty responses. After applying DSR, does the model become better at distinguishing these two sources, or does it primarily reduce uncertainty through improved unimodal optimization? Can the authors clarify the evidence supporting the claimed rectification mechanism?
-
I would like to see robustness experiments on CMU-MOSI or CMU-MOSEI under controlled modality-specific corruption, ideally including uncertainty estimates, fusion-weight changes, and downstream performance across different noise severities. This would more directly test the proposed mechanisms in the main tri-modal setting.
-
Please consider expanding the comparisons to closely related recent methods, particularly EBMC [1] and QA-MoE [2], under matched protocols where feasible. Although these methods are relatively recent, their close relevance to modality competition, uncertainty estimation, and reliability-aware fusion makes a comparison or more detailed discussion valuable.