- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes EmoR1, a reinforcement learning framework for multimodal emotion recognition that addresses neutral-boundary errors and modality credit assignment. The authors introduce NH-GRPO, combining neutral-aware rewards, hierarchical emotion rewards, and a counterfactual modality reward based on full-modal versus text-only predictions. They also construct MARI, a 103K-sample structured reasoning dataset, and NH-MERBench, a neutral-aware evaluation protocol. Experiments across seven benchmarks demonstrate competitive performance and improvements in neutral emotion recognition.
Critical strengths
The paper presents a relatively comprehensive framework encompassing dataset construction, structured supervision, reward design, and evaluation. Experiments cover seven benchmarks and multiple modality settings, with ablations on reward components and training configurations. The reported improvements in Neutral-F1 also suggest that explicitly addressing neutral-class errors is a useful direction.
Critical weaknesses
1. Limited methodological novelty.
The connection between the identified challenges and the proposed solutions remains weak. Despite framing neutral recognition as a calibration problem, NH-GRPO primarily relies on manually designed rewards over discrete labels. In the Neutral-Recovery bucket, the neutral reward is equivalent to , providing no fundamentally new supervision beyond exact-match classification. The paper does not explicitly model confidence or decision margins, nor does it evaluate probability calibration or compare against simpler cost-sensitive learning alternatives.
Similarly, MARI mainly introduces structured, model-generated reasoning annotations derived from existing datasets, without supervision specifically targeting neutral ambiguity or modality attribution. Its evidence-grounding reliability is also uncertain: the automatic auditor described in Appendix B receives textual records rather than the original audio-visual inputs, limiting its ability to verify modality-specific claims.
NH-MERBench also substantially overlaps with existing MER-UniBench datasets and uses the same hierarchical compatibility function as the training reward. Overall, the paper does not sufficiently establish the distinct value of these contributions beyond existing structured supervision and task-specific reward engineering.
2. Unreliable Modality Credit.
The proposed modality reward does not reliably reflect genuine audio-visual contributions. A full-modal rollout may outperform a text-only rollout simply because the latter produces a poor prediction due to sampling variation, even when the former relies primarily on textual cues. Such cases can receive positive modality rewards without actual multimodal reasoning, creating a potential reward-hacking shortcut.
More importantly, Appendix C indicates that the text-only score is cached per sample. Since this fixed reference term cancels under GRPO's within-group reward normalization when clipping is inactive, the modality reward effectively reduces to reweighting existing classification rewards rather than providing an independent modality-attribution signal.
These issues undermine the claimed effectiveness of counterfactual modality credit.
Questions for authors
Q1. Counterfactual Modality Reward. How can the authors demonstrate that the proposed reward genuinely encourages audio-visual evidence utilization, rather than benefiting from weak text-only predictions or merely reweighting existing classification rewards? More convincing empirical evidence is needed to establish the independent contribution of this reward to multimodal reasoning.
Q2. Neutral Calibration. How can the authors demonstrate that NH-GRPO genuinely improves neutral-boundary calibration rather than simply enhancing neutral-class prediction through reward engineering? The current evidence primarily relies on Neutral-F1, while Figure 1 compares metrics with different definitions. Stronger empirical evidence is needed to substantiate the calibration claim and establish the advantages over simpler classification-based approaches.
Q3. MARI annotation reliability. How are generated acoustic and visual descriptions verified when the automatic auditor does not directly access the original modalities? Can the authors report quantitative manual verification results and isolate the benefits of structured reasoning and auditing relative to MER-Caption+?
Q4. Missing comparisons with recent omni-modal emotion models. The paper omits relevant recent methods, particularly Omni-Perception Policy Optimization for Multimodal Emotion Reasoning (ICML 2026), which also uses Qwen2.5-Omni and explicitly optimizes modality utilization and evidence faithfulness. Can the authors include comparisons with OPPO and other competitive omni-modal emotion models, as well as a matched vanilla GRPO baseline? The full-modal improvement over Emotion-LLaMA v2 is only 0.62 points on average, making stronger comparisons and standard accuracy/Macro-F1 results important.