acceptodds
Under review as a conference paper at ICLR 2027

EmotionAdapt-R1: Confidence-Calibrated Adaptive Reasoning for Speech Emotion Recognition

Abstract

Speech emotion recognition inherently involves varying ambiguity, ranging from explicit cues to subtle, context-dependent expressions. While recent SpeechLLMs leverage Chain-of-Thought for explainable emotion reasoning, they apply static strategies allocating equal computation regardless of complexity. This fixed-depth method is often inefficient for clear cases and insufficient for ambiguous ones. To address this, we propose EmotionAdapt-R1, a confidence-aware framework enabling adaptive reasoning through Uncertainty-of-Thought, where uncertainty is integrated into generation to realize a "thinking fast and slow" paradigm. Specifically, we introduce Confidence-Adaptive GRPO (CA-GRPO), a policy optimization algorithm that dynamically aligns accuracy, reasoning depth, and confidence expression. Extensive evaluations show our approach outperforms strong baselines on standard SER benchmarks, demonstrating the value of self-aware confidence for efficient, adaptive multimodal reasoning.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
SpeechLLMs, Reasoning, Adaptive Thinking

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_Q14yOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

EmotionAdapt-R1 trains a speechLLM to output a confidence score first, then reason briefly when confident and at length when uncertain. It uses SFT on LLM-generated reasoning traces, followed by GRPO with five rewards. It reports ~70% average accuracy on four SER benchmarks, better high-confidence accuracy, and fewer reasoning tokens.

Critical strengths

Well-motivated: Speech emotion is an intrically ambigious task so combining reasoning with uncertainty makes sense. Good performance and analysis: 1. Careful study of gains with different reward types. 2. Strong out-of-domain gains on SAVEE and RAVDESS.

Critical weaknesses
  1. Central claim is untested. The main argument is ambiguous inputs need deeper reasoning, but it didn't show that longer reasoning improves on low-confidence cases.

  2. The reasoning-quality evaluation is in the method’s favor. Short-mode outputs are excluded from the comparison, the LLM judges are given paralinguistic attributes from the same pipeline used to build the training traces.

The human evaluation isn’t reported in the main text.

  1. The gain over EmotionThinker is about 2 points, with no seeds or variance. The model trains on IEMOCAP and MELD while most baselines don’t.

  2. The references can be more rigorously chosen.

Questions for authors

Is there a reason why confidence is emitted before <think>, so the model decides how sure it is before reasoning?

Does forcing short reasoning on low-confidence inputs hurt?

Could you check the paper's Internal inconsistencies? The text reports a reasoning score of 4.27, but Table 1 shows 4.17. Table 3’s chosen setting (λ = 0.3) gives 68.9% accuracy, while the main results report 70.97% with the same weights.

Do we really need this reference below? Might be better to be more careful with the references used.

Markus Raab. Think smart, not hard-a review of teaching decision making in sport from an ecological rationality perspective. Physical education and sport pedagogy, 12(1):1–22, 2007.

Reviewer_H3XmOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes EmotionAdapt-R1, a confidence-aware adaptive reasoning framework for speech emotion recognition. The main idea is to adjust reasoning length according to the model's predicted confidence, producing shorter explanations for high-confidence predictions and longer explanations for uncertain cases. The proposed framework consists of two stages. First, adaptive supervised fine-tuning uses automatically extracted prosodic features and VAD-based emotion clarity scores to construct confidence-conditioned reasoning traces. Second, Confidence-Adaptive GRPO (CA-GRPO) optimizes emotion classification accuracy, reasoning quality, confidence calibration, and reasoning length through a combination of reward functions. The authors report improvements in average SER accuracy and reduced reasoning length compared with several SpeechLLM baselines.

Critical strengths

A relevant approach to adaptive reasoning in speech emotion recognition

The paper addresses the practical problem of allocating reasoning effort based on uncertainty in speech emotion recognition (SER). The proposed framework jointly optimizes classification accuracy, confidence calibration, and reasoning length through reinforcement learning.

Evaluation across multiple benchmarks

The experiments cover multiple SER benchmarks and include ablations of the proposed reward components. The reported results show improvements in classification accuracy and reasoning efficiency over the GRPO baseline.

Critical weaknesses

Insufficient evidence for confidence calibration

The calibration reward uses group-level rollout accuracy to assign target confidence intervals, but this does not necessarily establish calibrated confidence for individual predictions. The predefined intervals also lack sufficient justification. Although Figure 5 shows improved confidence–accuracy alignment, standard calibration metrics such as ECE or Brier score are not reported, leaving the degree of calibration improvement unclear.

Limited evidence that adaptive reasoning improves prediction quality

The confidence–length alignment reward explicitly encourages shorter reasoning at high confidence and longer reasoning at low confidence. Thus, the inverse correlation in Figure 4 primarily demonstrates compliance with the reward objective rather than improved reasoning. The paper does not establish whether longer reasoning improves accuracy on difficult samples, and fixed-length baselines under matched computational budgets are missing.

Unclear contribution of adaptive reasoning to classification accuracy

The improvements cannot be clearly attributed to adaptive reasoning itself. Adding the length reward without the calibration reward reduces accuracy from 63.57% to 62.55%. Moreover, the reward ablations introduce the reasoning reward alongside the calibration or length reward, preventing a clean comparison with the GRPO baseline. Fixed-length and direct-classification baselines are also needed to isolate the contribution of adaptive reasoning.

Questions for authors
  1. What justifies using group-level rollout accuracy to supervise individual prediction confidence? Can the authors report standard calibration metrics (ECE and Brier score) and confidence distributions alongside Figure 5?

  2. Does longer reasoning improve accuracy on uncertain samples? Can the authors compare against fixed-length reasoning baselines under matched computational budgets?

  3. How does the method compare with direct emotion classification without explicit reasoning? Given that the reward ablations also change the reasoning reward, can the authors isolate the contributions of calibration and length control?

  4. Why are short-reasoning outputs excluded from the comparison? Can the authors report the quantitative human evaluation results described in the appendix?

Reviewer_32rHOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes EmotionAdapt-R1, a speech emotion recognition framework that uses uncertainty as a signal to adapt reasoning length. It combines adaptive supervised fine-tuning with Confidence-Adaptive GRPO to jointly optimize prediction accuracy, confidence calibration, and reasoning length. The paper evaluates the framework on four emotion recognition datasets and analyzes the contributions of its reward components.

Critical strengths
  1. The paper proposes using uncertainty as a signal for adaptive reasoning, allocating shorter reasoning to confident predictions and longer reasoning to uncertain ones to improve emotion recognition performance and efficiency.

  2. The paper conducts component analyses of the proposed adaptive reasoning behavior and confidence calibration, including reward ablations and analyses of reasoning length and confidence–accuracy alignment.

Critical weaknesses
  1. Limited evaluation and unclear provenance of baseline results. The baseline recognition numbers in Table 1 largely match those reported in the EmotionThinker paper, with Step-Audio-R1 and EmotionAdapt-R1 appearing to be the only new model entries. The authors should clarify which results were newly obtained and which were taken from prior work, especially because Appendix B.1 states that all models were evaluated under a unified protocol. More concerningly, EmotionThinker’s IEMOCAP score is reported as 76.68%, whereas the original paper reports 77.68%, compared with 73.45% for EmotionAdapt-R1. This discrepancy requires clarification. Reusing published results can be appropriate, but the provenance and comparability of the results need to be established before describing the evaluation as “extensive.” See EmotionThinker, Table 2.
  2. Missing relevant baselines and incomplete evaluation metrics. Why is only accuracy reported, rather than also reporting F1 scores? Important related approaches are missing, including SURE: Synergistic Uncertainty-aware Reasoning for Multimodal Emotion Recognition in Conversations, published at ICASSP 2026, and AffectGPT. SURE is particularly relevant because it combines uncertainty modeling with reasoning for emotion recognition. These methods use different modalities and evaluation protocols, so their published scores are not directly comparable; the authors should discuss these differences and provide comparisons under matched conditions where feasible. Together with the limited comparative evaluation, the insufficient distinction from prior work leaves the claimed technical contribution inadequately supported and the novelty appearing incremental.
  3. Incomplete coverage of related work. The related-work discussion should provide broader coverage of audio language models for emotion recognition, given the substantial work in this area. The distinction between “speech language models” and “audio language models” should also be clarified, along with how the proposed approach relates to both.
  4. Insufficiently reported validation of the LLM-as-judge evaluation. The paper uses LLM judges to evaluate emotion reasoning traces on a five-point scale. Although Appendix B.3 describes a blinded human evaluation involving three evaluators and ten test cases per model, the human evaluation results are not reported. Neither inter-annotator agreement nor agreement between human and LLM judgments is provided. Without these results, it is difficult to assess the reliability of the reasoning-quality scores and the claimed improvements.
Questions for authors
  1. How is the emotion taxonomy mapping across datasets performed (lines 376–377)? Please provide the dataset-specific mappings, explain how categories outside the unified seven-class taxonomy are handled, and describe how the mapping was validated.

  2. Many aspects of the approach, experimental setup, and reported baseline numbers follow EmotionThinker. Can the authors clearly summarize their contribution relative to that paper, distinguishing inherited components from the proposed additions and showing which additions account for the improvements?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.