- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes MCUR, a teacher–student framework for multimodal emotion recognition with missing modalities. It introduces modality-combination- and category-based contrastive learning (MCB-CL) to regulate representations, and sample-wise uncertainty-guided regularization (SUGR) to weight task and distillation losses using teacher–student uncertainty differences. Experiments on MOSI, MOSEI, and IEMOCAP report improvements across fixed and random missing-modality settings.
Critical strengths
The paper tackles an important practical problem: emotion recognition when different samples have different available modalities. It considers both how representations change with missing inputs and how a full-modality teacher should guide a student with incomplete inputs. The experiments cover three common benchmarks and a range of missing-modality settings.
Critical weaknesses
-
The main contributions are not clearly distinguished from existing work. The framework builds on contrastive learning, representation alignment, and loss weighting, all of which have been studied before. The paper needs to explain more clearly what is new in the proposed objectives and why these particular components are needed.
-
The baseline comparisons are limited. Most methods in the main comparison are from 2023–2024, although the related work discusses newer approaches. Without comparisons to relevant recent methods under the same evaluation settings, it is difficult to judge how competitive MCUR is. The MOSI results are also placed in the appendix rather than alongside MOSEI and IEMOCAP in the main results table.
-
It is unclear whether MCB-CL actually uses modality-combination information in its default setting. In Eq. (5), the default values are μ1=μ2=1. Substituting Eqs. (2)–(4) appears to cancel the modality-combination terms, leaving a contrastive objective based only on emotion labels, provided the sample sets are defined consistently. If the implementation follows these equations, what part of the loss still enforces modality-combination-specific alignment? The authors should clarify this point and explain the difference from ordinary label-based contrastive learning.
-
The entropy difference used in SUGR does not necessarily indicate reliable teacher supervision. A teacher and student can predict different classes with equally high confidence and still have the same entropy. In that case, Eq. (9) gives a zero uncertainty weight despite the disagreement. The absolute difference also treats cases where the teacher is more uncertain and less uncertain than the student in the same way. It would help to explain why this is a suitable weighting signal and to compare it with a measure that directly captures prediction disagreement.
-
The manuscript appears not to strictly follow the ICLR formatting requirements. The formatting should be checked against the official template.
Questions for authors
Please refer to the concerns in the Critical Weaknesses.