acceptodds
Under review as a conference paper at ICLR 2027

MER-R1: Multimodal Emotion Reasoning via Slow-Fast Thinking Synergy

Abstract

We find that explicit reasoning does not necessarily translate into better multimodal emotion recognition (MER) accuracy, even though it makes predictions more interpretable. Specifically, for reasoning-based MLLMs, *fast thinking* by triggering direct answers often outperforms *slow thinking* after deliberative reasoning. Our empirical analyses show that fast thinking improves recall with broader and more confident predictions, whereas slow thinking favors precision through conservative filtering of incorrect categories. Building on these insights, we propose **MER-R1**, a reinforcement learning framework that turns slow-fast complementarity into explicit optimization. Dual-objective disentanglement separates recall and precision into two optimization signals, allowing them to be jointly optimized rather than traded off against each other. Slow-fast confidence calibration further aligns the final slow-thinking answer with fast-thinking intuition, strengthening correct emotions while suppressing incorrect ones. In this way, MER-R1 unifies the recall-oriented intuition of fast thinking with the precision-oriented selectivity of slow thinking. We further provide theoretical justification for this synergy, showing that it mitigates variance-induced interference during optimization. Extensive experiments on MER-UniBench and MME-Emotion show that MER-R1 achieves state-of-the-art performance and makes reasoning genuinely benefit emotion recognition. Code is provided in the Supplementary Materials.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
RL, MLLM, Emotion

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_QmiZOpenReview ↗
Rating
3
Confidence
3
Informativeness
3(+1)
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates a "thinking paradox" in multimodal emotion recognition, where fast thinking can outperform slow thinking despite the latter's explicit reasoning process. The authors find that fast thinking tends to achieve higher recall, while slow thinking favors precision. To combine their strengths, the paper proposes MER-R1, a GRPO-based framework with two components: dual-objective disentanglement, which separately normalizes recall and precision advantages, and slow-fast confidence calibration (SFCC), which uses confidence differences between the two thinking modes to guide the final predictions. Experiments on MER-UniBench and MME-Emotion demonstrate improved recognition performance.

Critical strengths
  1. Interesting motivation and insightful analysis. The paper identifies a meaningful limitation of explicit reasoning in multimodal emotion recognition. The analysis of precision, recall, prediction diversity, and category-level confidence provides a clear motivation for the proposed method. The additional controls for prediction cardinality and reasoning length further strengthen the findings.
  2. Strong empirical results with useful ablations. MER-R1 improves the MER-UniBench mean from 77.87 to 83.50 on Qwen2.5-Omni and from 78.28 to 82.75 on MiniCPM-o-4.5. It also improves the MME-Emotion CoT score from 45.3 to 51.5. The component ablations and additional F1-based evaluations provide useful evidence for the effectiveness of the overall approach.
Critical weaknesses
  1. The theoretical claims are somewhat overstated. Appendix A shows that the dual-objective advantage has equal correlations with recall and precision rewards under non-degenerate conditions. However, this does not guarantee that optimizing the combined advantage will improve both objectives or prevent conflicts between them, as suggested by the discussion. The theoretical conclusions should therefore be stated more carefully. Additional analysis of the actual reward correlations during training would also be helpful.
  2. The specific contribution of SFCC to reversing the slow-fast gap remains unclear. Table 4 shows that adding SFCC improves the F1-based mean from 71.60 to 73.15, supporting its contribution to overall performance. However, the paper does not report separate fast- and slow-thinking results for Ex2 (without SFCC). It is therefore unclear whether SFCC itself reverses the slow-fast gap. Moreover, the gains of slow over fast thinking are relatively small (0.94 and 1.14 points in R-Mean across the two backbones). Reporting the fast-slow comparison for Ex2 and providing uncertainty estimates would make this claim more convincing.
  3. The confidence calibration mechanism needs further clarification. SFCC computes fast-conditioned confidence using the slow answer's prefixes rather than independently generated fast predictions. Since SFCC operates on emotion words in the slow answer, correct categories predicted only by fast thinking are not directly included in its calibration reward for that rollout. The authors should clarify how SFCC helps recover these missing categories and consider a comparison with a confidence objective that does not use fast-conditioned information. Additionally, the use of first-token probabilities in Eq. (1) and the handling of empty prediction sets in Eq. (11) deserve clarification.
Questions for authors
  1. Does the equal-correlation property of the dual-objective advantage imply any guarantee about optimization outcomes? Could the authors clarify the scope of the theoretical conclusions?
  2. Could the authors report fast- and slow-thinking performance for Ex2 and Ex3 separately, ideally with confidence intervals, to establish whether SFCC is responsible for reversing the slow-fast gap?
  3. How does SFCC encourage the recovery of correct categories omitted from the slow answer? Could the authors compare it with a confidence objective without fast-conditioned information and clarify the implementation details of the confidence calculation?
4 comments

We thank the reviewer for highlighting the insightful analysis and strong empirical results. We appreciate your careful feedback and provide our responses below.

W1&Q1: Scope of Theoretical Analysis

Our theoretical analysis establishes equal correlations between the dual-objective advantage and the recall/precision rewards, supporting more balanced optimization signals. We have revised the relevant statements in the manuscript to clarify the scope of our theoretical conclusions.

To further validate this property empirically, we computed the reward–advantage correlations during training:

Corr⁡(RR,A)=0.8178,Corr⁡(RP,A)=0.8167.\operatorname{Corr}(R_R,A)=0.8178,\qquad \operatorname{Corr}(R_P,A)=0.8167.

The two correlations are nearly identical, which is consistent with our theoretical analysis.

Revisions: The theoretical statements are refined in the Abstract (L18–20), Introduction (L80–85), and Appendix A.2 (L736–744), with empirical reward–advantage correlations added in Appendix A.2.

W2&Q2: SFCC Contribution to the Slow–Fast Gap

We reran inference three times for Ex2 (RD+AD, without SFCC) and Ex3 (MER-R1) from Table 4 and report the averaged results below.

ModelModeOV-MERD+MER23MER24MELDIEMOCAPMOSIMOSEISIMSSIMSv2R-Mean
Ex2 (RD+AD)Fast69.5065.0667.1446.1854.5883.3385.2587.2787.6160.49
Slow69.7664.6067.4447.3954.3484.2385.4186.8087.6160.71 (↑0.21)
MER-R1 (Ex3)Fast69.6063.5067.2446.7255.5685.8386.9087.8288.1060.52
Slow70.4565.5368.5548.3755.8985.9186.7786.9487.0061.76 (↑1.23)

We additionally computed paired bootstrap 95% confidence intervals for the R-Mean gaps.

Statistical comparisonGap(pp)95% CI(pp)
Ex2: Slow − Fast+0.21[−0.31, +0.72]
MER-R1: Slow − Fast+1.23[+0.85, +1.62]

Without SFCC, Ex2 shows only a small Slow–Fast gap (+0.21), with a confidence interval that includes zero. Adding SFCC increases the gap to +1.23, with a confidence interval entirely above zero.

These results provide further evidence that SFCC is a key contributor to establishing a consistent Slow Thinking advantage and reversing the thinking paradox.

Revisions: Additional Fast/Slow comparisons and bootstrap confidence intervals are provided in Appendix D.8 (L1096–1128, Tables 12–13).

W3&Q3: SFCC Mechanism and Confidence Calculation

1. Recovery of omitted emotions. GRPO samples multiple rollouts per input, allowing responses that recover omitted correct emotions to receive higher recall advantages. Within each rollout, SFCC further calibrates Slow Thinking confidence relative to Fast Thinking, strengthening correct predictions while suppressing incorrect ones. Thus, GRPO can reinforce rollouts that recover omitted emotions, while SFCC further strengthens the confidence of correctly predicted emotions.

This is supported by the retained-hitrate analysis in Appendix D.6 (Fig. 8), where MER-R1 consistently preserves more correct Fast Thinking predictions across all five datasets.

2. Ablation without fast-conditioned confidence. To assess the contribution of Fast Thinking information, we introduce an absolute-confidence calibration variant that replaces the Slow−Fast confidence difference Δi(e)\Delta_i(e) in Eq. (11) with the Slow confidence conf⁡islow(e)\operatorname{conf}_i^{slow}(e), keeping the remaining training configuration unchanged. This variant still encourages higher confidence for correct predictions and lower confidence for incorrect predictions, but removes the Fast-conditioned reference.

ModelModeOV-MERD+MER23MER24MELDIEMOCAPMOSIMOSEISIMSSIMSv2R-MeanMean-F1
Absolute ConfidenceFast70.0364.5167.6947.9658.1281.3184.3486.8486.3961.6671.91
Slow69.7465.6166.4547.5156.7182.4285.2885.3087.1661.20 (↓0.46)71.80
MER-R1 (SFCC)Fast69.6063.5067.2446.7255.5685.8386.9087.8288.1060.5272.36
Slow70.4565.5368.5548.3755.8985.9186.7786.9487.0061.76 (↑1.23)72.82

R-Mean averages F1 across five emotion-recognition datasets; Mean-F1 averages F1/WAF across all nine datasets.

Absolute-confidence calibration improves Slow Thinking R-Mean over the Baseline (58.11 → 61.20), but Slow Thinking still underperforms Fast Thinking by 0.46 points. In contrast, SFCC achieves a higher Slow Thinking R-Mean (61.76) and reverses the gap to +1.23. These results demonstrate the importance of fast-conditioned confidence for achieving slow–fast synergy, beyond absolute-confidence optimization.

3. Confidence computation details. For each generated emotion word, we compute confidence at its own first-token prediction position, rather than only at the beginning of the answer. Slow and Fast use the same policy, multimodal input, and preceding answer tokens; Fast omits the reasoning trace without independently generating a Fast answer. To compute category-level confidence, we collect the first tokens of all emotion words belonging to that category in the emotion wheel, removing duplicates and tokens shared across different categories. We then sum their predicted probabilities, take the logarithm, and average across applicable emotion wheels. Words without valid mappings are excluded, and empty correct/incorrect sets contribute zero raw calibration reward in Eq. (11).

Revisions: The absolute-confidence ablation is provided in Appendix D.10 (L1150–1170, Table 15). Detailed SFCC implementation is provided in Appendix C.2 (L905–956).

Appreciate the authors detailed responses. I am satisfied with their clarifications.

Thank you for your follow-up and for taking the time to consider our responses. We are glad that our clarifications addressed your concerns. We sincerely appreciate your thoughtful feedback and consideration!

Reviewer_mgeLOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper identifies a “thinking paradox” in multimodal emotion recognition (MER): fast thinking can outperform slow thinking. It diagnoses this via recall/precision trade-offs and category-level confidence, then proposes MER-R1, an RL framework with dual-objective disentanglement (separate recall/precision rewards and advantages) and slow-fast confidence calibration. Experiments on MER-UniBench and MME-Emotion, together with ablations and a MiniCPM generalization check, support the claimed SOTA and show that slow thinking can be made stronger than fast thinking.

Critical strengths
  1. The diagnosis of the thinking paradox is clear and useful: the recall/precision and confidence analyses motivate why fast and slow thinking are complementary, rather than simply preferring one mode. This is consequential because it turns an empirical curiosity into a concrete method design.

  2. The dual-objective disentanglement plus confidence calibration yields consistent gains and ablations, and the method reverses the slow-vs-fast trend under unified F1 evaluation. This is consequential because it supports the central claim that reasoning can genuinely improve recognition, not only interpretability.

Critical weaknesses
  1. The slow-fast confidence calibration is the key component for reversing the paradox, but the main text sketches it mainly through Fig. 4; the exact calibration objective, how fast/slow confidences are obtained during RL, and how correct/incorrect category sets are determined should be specified more clearly. This matters because the claimed mechanism and ablation gains hinge on this component.

  2. MME-Emotion is evaluated with a different judge model than the original benchmark. Although all methods are re-evaluated consistently, judge sensitivity and comparability to prior reported numbers are not fully established. This matters for the strength of the SoTA claim.

Questions for authors
  1. Please provide the exact formulation of the slow-fast confidence calibration loss and clarify how fast/slow category confidences are computed and which categories are treated as correct/incorrect during training.

  2. For MME-Emotion, can you report any judge-agreement or sensitivity analysis to justify the use of gemini-3.1-flash-lite-preview as the evaluator?

  3. Can you clarify how much of the gain comes from the proposed components versus the stronger RL data allocation, and whether AffectGPT-RL can be compared under matched RL sample counts?

2 comments

We appreciate the reviewer’s positive assessment of our analysis and how it motivates the proposed framework. We address your concerns below.

W1&Q1: SFCC Formulation and Training Objective

SFCC is implemented through confidence-based rewards within GRPO. Its implementation follows three steps:

1. Fast/Slow confidence computation. For each sampled response yi=(yicot,yians)y_i=(y_i^{\mathsf{cot}},y_i^{\mathsf{ans}}), we compute two distributions using the same policy:

p_i^{\mathsf{fast}}(e)=\pi_\theta(\cdot\mid x,y_{i,\lt e}^{\mathsf{ans}}).$$ Both use the same multimodal input and preceding answer tokens; Fast omits the reasoning trace without generating a separate answer. For each generated emotion word, we compute confidence at its own first-token prediction position. Following Eq. (1), category-level log-confidence is defined within each emotion wheel as: $$\mathsf{conf}^m(c)=\log\sum_{e\in\mathcal V(c)}p^m(e).$$ where $p^m(e)$ denotes the probability assigned to the first token of emotion word $e$. In our multi-wheel implementation, we compute this confidence independently for each valid wheel-category mapping and average the resulting log-confidence values. We remove duplicate first-token IDs and exclude tokens shared across categories within the same wheel. **2. Correct/incorrect categories and calibration rewards.** A generated emotion word is considered correct if its mapped category matches a ground-truth category in at least one corresponding emotion wheel; otherwise, it is incorrect. Words without valid wheel mappings are excluded. Let $\mathcal E_i^+$ and $\mathcal E_i^-$ denote these two sets. We compute: $$\Delta_i(e)= \mathsf{conf}_i^{\mathsf{slow}}(c(e))- \mathsf{conf}_i^{\mathsf{fast}}(c(e)).$$ $$R_i^+=\frac{\sum_{e\in\mathcal E_i^+}\Delta_i(e)} {\max(1,|\mathcal E_i^+|)},\quad R_i^-=-\frac{\sum_{e\in\mathcal E_i^-}\Delta_i(e)} {\max(1,|\mathcal E_i^-|)}.$$ Thus, SFCC uses Fast Thinking as a reference, encouraging Slow Thinking to assign **higher confidence to correct emotions and lower confidence to incorrect ones**. The rewards average confidence gaps across generated emotion words within each set, while empty sets contribute zero raw calibration reward. **Revisions:** Detailed SFCC implementation, including confidence computation and calibration rewards, is provided in Appendix C.2 (L905–956). ### W2&Q2: Judge Agreement on MME-Emotion We conducted additional evaluations using **DeepSeek-V4.1-Flash** and **GPT-6-Luna**, alongside our original **Gemini-3.1-Flash-Lite-Preview** evaluator. We kept the model responses and evaluation protocol fixed, changing only the judge. The table reports overall CoT scores. | Method | Gemini (Original Submission) | DeepSeek-V4.1-Flash | GPT-6-Luna | |---|---:|---:|---:| | PandaGPT | 30.00 | 34.80 | 36.19 | | Emotion-LLaMA | 31.80 | 38.56 | 43.57 | | AffectGPT | 38.60 | 44.58 | 46.18 | | Baseline | 45.30 | 53.48 | 55.85 | | **MER-R1** | **51.50** | **58.49** | **61.45** | | **MER-R1 − Baseline** | **+6.20** | **+5.01** | **+5.60** | **Cross-judge agreement.** All three judges produce exactly the same ranking: **MER-R1 > Baseline > AffectGPT > Emotion-LLaMA > PandaGPT**. At the method-ranking level, both Spearman's $\rho$ and Kendall's $\tau$ are **1.0** across all pairwise judge comparisons. **This consistent ranking across different LLM judges supports the robustness of our comparative conclusions to judge selection.** **Revisions:** The cross-judge evaluation and ranking consistency analysis are provided in Appendix D.9 (L1130–1148, Table 14).

Q3: RL Data Allocation

AffectGPT-RL†^{\dagger} is our reproduction of AffectGPT-RL using Qwen2.5-Omni-7B, following the original paper's SFT/RL data allocation, with 1K samples used for GRPO.

We construct a stronger Baseline by changing only the SFT/RL data allocation: 5K randomly sampled instances for SFT and the remaining 26,327 instances for RL, with all other settings unchanged. As shown in Table 1, this adjustment improves the MER-UniBench mean score from 75.60 to 77.87 (+2.27).

Importantly, MER-R1 uses exactly the same SFT/RL data allocation and training settings as our Baseline, differing only in the proposed components (RD, AD, and SFCC). Under this data-matched setting, MER-R1 further improves the mean score from 77.87 to 83.50 (+5.63). Thus, the additional gains over Baseline come from our proposed method rather than increased RL training data.

Revisions: The SFT/RL data allocation and training settings are clarified in Appendix C.1 (L885–903), with the corresponding performance comparisons in Table 1 (L339–341).

Reviewer_GTVMOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper investigates a thinking paradox in MER where the fast-thinking often performs on par with or better than slowing thinking for reasoning models. The paper analyses this from two perspectives. At prediction level, fast thinking shows higher recall. At confidence level, fast thinking is more confident on correct categories. Building on these, the paper introduces MER-R1, an RL framework designed for leveraging the complementary strengths of fast and slow thinking. Specifically, MER-R1 employs dual-objective disentanglement to separately optimize recall and precision and slow-fast confidence calibration to preserve correct predictions while suppressing incorrect ones. Experiments on MER-UniBench and MME-Emotion demonstrate improvements over existing baselines.

Critical strengths
  • This paper highlights an interesting thinking paradox in MER where fast thinking can outperform slow thinking. The proposed method is well-motivated based on the empirical findings.
  • The proposed method has been evaluated on MER-UniBench and MME-Emotion and demonstrated consistent gains over baselines.
Critical weaknesses
  • The paper does not specify the model size for Qwen2.5-Omni. Moreover, both backbones seem to be at or below 9B scale. It remains unclear whether the observed thinking paradox and MER-R1’s ability to reverse it generalize to larger models.
  • The ablations may not fully clarify the contribution of each component. Table 4 lacks an SFCC-only ablation and Table 3 shows slow/fast comparison only for the full model.
Questions for authors

Please refer to weaknesses.

3 comments

We thank the reviewer for recognizing the interesting thinking paradox and the motivation behind MER-R1. We address your questions and provide additional results below.

RQ1: Model Sizes and Generalization to Larger Models

Our backbones are Qwen2.5-Omni-7B and MiniCPM-o-4.5-9B. To further confirm that the thinking paradox persists at a larger scale, we additionally evaluated Qwen3-Omni-Thinking-30B on five emotion-recognition datasets.

MetricModeOV-MERD+MER23MER24MELDIEMOCAPR-Mean
RecallFast73.6086.7091.6568.0369.5977.91
Slow67.85 (↓5.75)78.91 (↓7.79)82.04 (↓9.61)59.77 (↓8.26)63.71 (↓5.88)70.46 (↓7.46)
PrecisionFast67.7953.5256.9837.7944.6652.15
Slow69.58 (↑1.79)54.68 (↑1.16)58.13 (↑1.15)38.09 (↑0.30)45.86 (↑1.20)53.27 (↑1.12)
F1Fast70.5866.1870.2748.5954.4162.01
Slow68.70 (↓1.88)64.60 (↓1.58)68.05 (↓2.22)46.53 (↓2.06)53.33 (↓1.08)60.24 (↓1.76)

Across all five datasets, Fast Thinking consistently achieves higher recall and F1, while Slow Thinking achieves higher precision. The average F1 decreases from 62.01 to 60.24 with Slow Thinking. These results support that the thinking paradox and recall–precision complementarity persist at a larger model scale.

Scaling MER-R1 training to 30B models is computationally demanding, requiring over four times the computational resources of our current setup. We leave larger-scale RL training for future work.

Revisions: Model sizes are clarified in Section 5 (L372) and Appendix D.1 (L963–965). The 30B results are provided in Appendix D.7 (L1074–1092, Table 11).

RQ2: SFCC Ablation and Slow–Fast Comparisons

We conducted an additional SFCC-only ablation and evaluated Fast/Slow Thinking separately for Ex2 (RD+AD) and Ex3 (MER-R1) from Table 4. We reran inference three times and averaged the results.

ModelModeOV-MERD+MER23MER24MELDIEMOCAPMOSIMOSEISIMSSIMSv2R-Mean
SFCC-onlyFast66.9660.8764.0844.9849.8684.7885.9487.2586.6257.35
Slow66.2564.3767.2646.0952.7584.2385.5487.8087.6159.34 (↑1.99)
Ex2 (RD+AD)Fast69.5065.0667.1446.1854.5883.3385.2587.2787.6160.49
Slow69.7664.6067.4447.3954.3484.2385.4186.8087.6160.71 (↑0.21)
MER-R1 (Ex3)Fast69.6063.5067.2446.7255.5685.8386.9087.8288.1060.52
Slow70.4565.5368.5548.3755.8985.9186.7786.9487.0061.76 (↑1.23)

1. Independent contribution of SFCC. Applying SFCC alone improves Slow Thinking R-Mean from 58.11 (Baseline) to 59.34 and reverses the Slow–Fast gap from −0.38 to +1.99. These results demonstrate that SFCC independently contributes to mitigating the thinking paradox.

2. Contributions of the components. RD+AD improves Slow Thinking R-Mean from 58.11 to 60.71. Adding SFCC further increases it to 61.76, while Fast Thinking remains nearly unchanged (60.49 vs. 60.52). Consequently, the Slow–Fast gap increases from +0.21 to +1.23.

These results clarify the complementary roles of the components: dual-objective disentanglement improves overall recognition performance, while SFCC further strengthens Slow Thinking and its advantage over Fast Thinking.

Revisions: The SFCC-only ablation is added to Table 4 (L432–441). Detailed Fast/Slow comparisons and statistical analyses are provided in Appendix D.8 (L1096–1128, Tables 12–13).

Thank you to the author for replying to my comments. It has resolved some of my questions. I maintain my original positive score.

Thank you for your feedback and for taking the time to review our response. We sincerely appreciate your positive assessment and constructive suggestions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.