acceptodds
Under review as a conference paper at ICLR 2027

Dr.EMO: De-biased Reward Optimization for Emotionally Expressive Text-to-Speech

Abstract

Achieving natural and controllable emotional expression remains a key challenge in text-to-speech (TTS) synthesis. Reinforcement learning offers a promising approach through emotion reward optimization. However, such optimization relies on reward models that can reliably recognize emotions in synthesized speech. Since human-verified emotional labels for synthesized speech are scarce, existing methods typically employ speech emotion recognition (SER) models trained on real human speech as reward models. Yet, we identify a substantial misalignment between such models and human judgments on synthesized speech. To address this issue, we curate an open-source dataset of manually filtered synthesized speech with clear emotional expression and train a de-biased emotion reward model that better aligns with human judgments. Based on this model, we introduce Dr.EMO, a multi-objective group relative policy optimization framework that enhances emotional expressiveness while preserving general TTS capabilities. Experiments demonstrate that Dr.EMO achieves state-of-the-art performance in conveying target emotions, with over 90% of synthesized utterances exhibiting clear human-evaluated emotional expression. Our findings highlight the importance of reliable, de-biased rewards for effective emotional TTS training.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Emotional Text-to-Speech, Reinforcement Learning, Generalization

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_iYqmOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a GRPO framework for improving emotional expressiveness in TTS. The authors first adapt an SER reward model using synthetic speech to better align rewards with human judgments, and then combine emotion, ASR, and duration rewards during GRPO training. Experiments show substantially improved target emotion expression while largely preserving general TTS quality.

Critical strengths
  1. This paper identifies a practically issue in emotional TTS post training. The mismatch between SER based rewards and human judgments on synthetic speech is clearly demonstrated, and adapting the reward model to synthetic speech leads to better human alignment.
  2. The experimental validation is strong. The method is tested on two TTS backbones and compared with several reward model variants. The human evaluation and ablation results also support the main findings.
Critical weaknesses
  1. The methodological novelty is limited. The main pipeline combines reward model adaptation on synthetic speech with standard GRPO and existing ASR and duration rewards. The contribution is useful but most individual components are not new.
  2. The reward model relies on manually curated synthetic speech, which may limit scalability and generalization to broader emotions and TTS systems.
Questions for authors
  1. How much of the improvement in Qwen2.5-Omni-SER comes specifically from the synthetic data?
  2. The proposed models show lower UTMOSv2 scores. Did you conduct human evaluation of naturalness to check whether stronger emotion comes at the cost of speech quality?
  3. Could you provide the missing GRPO settings, such as the group size KK, clipping ratio, KL coefficient, and sampling strategy used for rollout generation?
  4. How sensitive is the reward model to the choice of generator, and would using synthetic speech from multiple TTS systems further improve generalization?
3 comments

We thank the reviewer for the constructive feedback and recognition of our experimental validation. We address the concerns below.

W1. Methodological Novelty

We emphasize that this work addresses a critical and pressing challenge: substantially enhancing the salience of emotional expression in TTS-generated speech. Importantly, we need to clarify that Dr.EMO is a post-training framework specifically designed to enhance emotional expressiveness in TTS models through two key stages: (i) modeling accurate emotion reward signals on TTS output distributions (Section 3), and (ii) optimizing TTS policies via GRPO using these reward signals (Section 4).

Our contributions lie in (i) providing empirical insights into how to effectively construct reward models for emotional expression, and (ii) experimentally demonstrating state-of-the-art performance in emotional TTS by combining GRPO with the reward model developed in the first stage (Section 5).

W2. Generalization and Scalability

We emphasize that Section 6 has already demonstrated the generalization capabilities of Dr.EMO from two perspectives: (i) the generalization of reward models constructed within the Dr.EMO framework to unseen TTS output distributions (Section 6.1), and (ii) the generalization of Dr.EMO-trained state-of-the-art TTS models to other synthesis tasks (Section 6.2).

Furthermore, we argue that a single reward model cannot be expected to cover the output distributions of all possible TTS systems. Instead, Dr.EMO is a post-training framework designed to enhance emotional expressiveness in individual TTS models, and we have demonstrated its effectiveness across different TTS backbones (Section 5). More importantly, human annotation is indeed necessary to obtain reliable emotional supervision that lies beyond the capabilities of existing automatic methods, even at the cost of reduced scalability. Human feedback is indispensable for evaluating subjective emotional expression and ensuring reliable reward modeling.

Q1. Contribution of Synthetic Data to Reward Modeling

As shown in Table 1 (I–II), adapting Qwen2.5-Omni with 8K human-verified synthetic utterances and 4K real utterances improves its agreement with human judgments from 36.4% to 84.1%, a substantial gain of 47.7 percentage points. Compared with the widely used Emotion2vec baseline (58.7%), our adapted Qwen2.5-Omni-SER also achieves a 25.4-point improvement.

To further isolate the contribution of synthetic data to downstream TTS optimization, we additionally conducted an ablation using a reward model adapted exclusively on 4K real speech utterances, without synthetic speech. We then trained the CosyVoice3 policy using this real-only reward model under the same GRPO framework and evaluated its emotional expressiveness using the same strict human evaluation criterion.

Reward Model Training DataHuman ESR (%)
Real-only (4K)77.8
Mixed (4K real + 8K synthetic)91.7

The real-only reward model achieves an ESR of 77.8% (778/1,000), whereas the mixed-data reward model achieves 91.7% (917/1,000), representing a 13.9-percentage-point improvement. This result demonstrates that incorporating human-verified synthetic speech into reward modeling substantially improves downstream emotional expressiveness under the evaluated training configuration.

Importantly, Table 1 quantifies the overall improvement in reward–human agreement after adaptation, while the additional ablation directly evaluates the impact of different reward-model training datasets on downstream GRPO performance. Since the mixed dataset also contains more training examples, the ablation does not fully disentangle the effect of synthetic speech from increased training data volume. Nevertheless, it provides direct empirical evidence that mixed-data reward adaptation yields substantially better emotional TTS performance than real-only adaptation.

Q2. Human Evaluation of Speech Naturalness

UTMOSv2 is a model-based metric designed to estimate human-perceived speech quality. However, its predictions may not fully capture perceptual naturalness for emotionally expressive synthesized speech, particularly when stronger emotional expression introduces prosodic characteristics that differ from those typically represented in automatic quality estimation.

To directly investigate whether improved emotional expressiveness compromises perceived speech quality, we conducted an additional human Mean Opinion Score (MOS) evaluation comparing the original CosyVoice2 and CosyVoice3 backbones with their Dr.EMO-trained counterparts.

For each system, we sampled 250 utterances, evenly distributed across the five target emotions (50 per emotion), resulting in 1,000 evaluated utterances across four systems. The evaluation focuses on human-perceived speech naturalness and overall quality, independently of whether the intended emotion is successfully expressed.

BackboneBaseline MOSDr.EMO MOSDifference
CosyVoice23.7923.812+0.020
CosyVoice33.8203.840+0.020

The results show that Dr.EMO achieves slightly higher average human MOS scores on both backbones, with an improvement of 0.020 points in each case. In contrast to the decreases observed in UTMOSv2, the human evaluations do not show a corresponding reduction in average perceived speech quality.

These findings suggest that the substantial improvements in emotional expressiveness achieved by Dr.EMO do not necessarily come at the expense of human-perceived naturalness. The observed MOS differences are small and are reported descriptively rather than as statistically significant improvements. Nevertheless, the results provide complementary human perceptual evidence that the reductions in automatic quality estimates should not be directly interpreted as equivalent degradation in perceived naturalness.

Q3. GRPO Hyperparameters and Rollout Sampling Configurations

Both CosyVoice2 and CosyVoice3 use the same core GRPO optimization settings, including a group size of K = 8, a KL regularization coefficient of 0.001, and a PPO clipping ratio of 0.2. The global batch size is 32, and the learning rate is set to 1 × 10⁻⁶. These optimization settings are shared across the two backbones and are also provided in our anonymous code repository.

Due to differences in model architectures and speech-token generation mechanisms, the two backbones adopt different rollout sampling configurations:

  • CosyVoice2: Sampling temperature = 1.0, top-p = 0.95, and maximum generation length = 512 tokens.
  • CosyVoice3: Sampling temperature = 1.0, top-p = 1.0, and a dynamic maximum generation length of min(20L, 1024), where L denotes the number of input text tokens. The shared GRPO settings maintain a consistent policy optimization procedure across both backbones, while the backbone-specific sampling configurations accommodate their respective generation mechanisms. We will explicitly document these hyperparameters and rollout settings in the revised manuscript to improve reproducibility.

Q4. Generator Sensitivity and Multi-Generator Reward Training

As demonstrated in Section 6.1, although our reward model is trained using synthetic speech exclusively from CosyVoice2, it generalizes to three unseen TTS systems (IndexTTS2, GPT-4o-TTS, and Gemini-2.5-Pro) without further adaptation. On 949 human-verified positive samples, Qwen2.5-Omni-SER achieves 81.98% emotion consistency, compared with 56.38% for Emotion2vec, demonstrating substantial cross-generator transfer.

Importantly, Dr.EMO does not require a universal reward model covering all possible TTS output distributions. Instead, it provides a post-training framework that supports reward adaptation to target synthesis distributions and subsequent TTS policy optimization through GRPO. Its effectiveness is demonstrated on both CosyVoice2 and CosyVoice3, achieving human-evaluated ESRs of 91.1% and 91.7%, respectively. While incorporating multiple synthetic generators may further improve reward robustness, our results demonstrate that single-generator reward adaptation already enables effective transfer to unseen TTS systems, without requiring multi-generator training.

Dear Reviewer iYqm,

As we are approaching the final day of the rebuttal period, we would like to thank you again for your time and constructive feedback on our paper.

We have provided detailed responses to all your concerns, including additional experiments on the contribution of synthetic data and human-evaluated speech naturalness, as well as further clarifications regarding methodological contributions, generalization, and implementation details. We hope these new results and explanations have addressed your concerns and provided stronger evidence supporting our contributions.

We understand that this is a busy period. We would greatly appreciate it if you could take another look at our responses and let us know whether any concerns remain. We would be happy to provide further clarification before the discussion period closes.

Thank you again for your time and consideration.

Best regards,
The Authors

Reviewer_xfQQOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper investigates the mismatch between existing speech emotion recognition (SER) models and human judgments of synthesized speech, and its implications for reinforcement learning of emotional TTS. The authors curate 8K human-filtered synthetic utterances, combine them with 4K real-speech utterances, and fine-tune an emotion recognition model to serve as a reward model. They then optimize two TTS backbones using GRPO with emotion, ASR, and duration rewards. Human evaluation shows substantial improvements in target-emotion success.

Critical strengths
  • Clear motivation and a coherent connection between the problem and the proposed solution. The paper examines whether the reward models used for emotional TTS actually agree with human perception, adapts the reward model using human-verified synthetic speech, and evaluates the resulting TTS outputs with human listeners. This progression makes the work easy to follow and its practical motivation convincing.
  • Substantial human involvement in both data construction and evaluation. Human filtering addresses the fact that conditioning a TTS model on an emotion does not guarantee that the generated speech expresses it. The final evaluation also goes beyond automatic SER scores: utterances are independently assessed by two annotators, with additional adjudication for disagreements. This is particularly valuable given that the reliability of automatic emotion evaluation is itself under examination.
  • The downstream results support the importance of reward-model choice. Relative to GRPO using Emotion2vec rewards, the proposed approach improves human-evaluated ESR by 11.8 and 8.0 percentage points on CosyVoice2 and CosyVoice3, respectively. The ranking reversal between Emotion2vec-based evaluation and human evaluation is especially informative: a higher automatic emotion score does not necessarily correspond to better perceived emotional expression. The contribution is primarily empirical and data-driven, and these results provide meaningful support for it.
Critical weaknesses
  • The main area for improvement is the reporting of human-evaluation reliability. The paper describes independent annotation, blinded system identities, randomized presentation, and adjudication, which are positive aspects of the protocol. However, it does not report inter-annotator agreement, the proportion of utterances requiring adjudication, or confidence intervals for ESR and the differences between systems. Since human evaluation is the central evidence for the paper’s conclusions, these statistics would help readers assess the consistency of the judgments and the robustness of the reported improvements. This is primarily a reporting limitation rather than evidence against the effectiveness of the approach.
Questions for authors
  • Could you report agreement between the initial annotators, the proportion of cases requiring adjudication, and confidence intervals for ESR and the gains over the Emotion2vec-reward baselines?

  • As an additional analysis, have you compared reward-model fine-tuning on real speech only, synthetic speech only, and their combination? This would help clarify the specific contribution of synthetic-speech adaptation relative to additional supervised emotion training, although the current downstream results already demonstrate the usefulness of the proposed combination.

3 comments

We sincerely thank the reviewer for the constructive and encouraging assessment of our work, particularly the recognition of our motivation, extensive human evaluation, and evidence linking reward-model reliability to downstream emotional expressiveness. We provide additional statistical evidence and a targeted reward-model data ablation below.

W1. Reliability of Human Evaluation

We appreciate the careful attention to annotation reliability, which provides an opportunity to make the existing evaluation protocol and its statistical support more explicit.

The concern is primarily about the reporting of reliability statistics, rather than a deficiency in the evaluation design or evidence of unreliable results. As described in Appendix C, utterances were evaluated independently by two blinded annotators under randomized presentation, and disagreements were independently adjudicated. Crucially, the human ESR is computed from the final adjudicated labels, rather than unverified judgments from a single annotator.

Our recovered initial annotation records for both Dr.EMO-trained systems further substantiate the protocol: the annotators agree on 92.20% of CosyVoice2 utterances and 90.80% of CosyVoice3 utterances before adjudication. The remaining 7.80% and 9.20% were adjudicated. We also quantify chance-corrected agreement using Cohen’s κ and report confidence intervals, as detailed in Q1. These measurements distinguish raw agreement from chance-corrected agreement without overstating either metric.

Most importantly, the primary claim is supported by a convergent pattern of results: Dr.EMO achieves human-evaluated ESRs of 91.1% and 91.7% on two independent TTS backbones, outperforming the Emotion2vec-reward GRPO variants by 11.8 and 8.0 percentage points, respectively (Table 2). The additional reliability statistics make the evidence more transparent, while the central observation of improved human-perceived emotional expression remains unchanged.

Q1. Initial Agreement, Adjudication, and ESR Uncertainty

For the two Dr.EMO systems with available original annotation records, the initial annotators agreed on 922/1,000 utterances (92.20%) for CosyVoice2 and 908/1,000 (90.80%) for CosyVoice3. The remaining 78 and 92 utterances required adjudication, corresponding to 7.80% and 9.20%. Following Appendix C, disputed samples were reviewed by additional annotators to determine final labels. We also calculate Cohen’s Kappa to account for agreement expected from the annotators’ marginal label distributions.

SystemInitial agreement [95% CI]Adjudication rate [95% CI]Cohen’s Kappa [95% CI]
CV2 + Dr.EMO92.20% [90.60%, 93.70%]7.80% [6.30%, 9.40%]0.4394 [0.3378, 0.5348]
CV3 + Dr.EMO90.80% [89.00%, 92.50%]9.20% [7.50%, 11.00%]0.4328 [0.3346, 0.5278]

These statistics are based on the two Dr.EMO systems for which initial annotation records remain available. Cohen’s Kappa (0.4394 and 0.4328) contextualizes the high raw agreement; raw agreement should not be interpreted as near-perfect chance-corrected agreement. Reliability is measured before adjudication, whereas ESR uses final adjudicated labels.

For final human-evaluated ESR, the 95% Wilson confidence intervals are 91.1% [89.17%, 92.71%] for CV2 + Dr.EMO and 91.7% [89.83%, 93.25%] for CV3 + Dr.EMO. The corresponding Emotion2vec-reward baseline ESRs are 79.3% and 83.7%, giving point-estimated gains of 11.8 and 8.0 percentage points. The utterance-level baseline annotations are unavailable, so paired confidence intervals for these gains cannot be reconstructed. We therefore distinguish the ESR confidence intervals from uncertainty in the between-system differences and do not claim statistical significance from the individual intervals alone. We will include the available statistics and their coverage in the revised manuscript.

Q2. Real-only Versus Mixed-data Reward Adaptation

We thank the reviewer for the suggestion to examine the role of synthetic speech more directly. We additionally trained a CosyVoice3 policy via GRPO using a Qwen2.5-Omni reward model adapted on 4K real-speech utterances only. Under the same strict human ESR evaluation criterion, this policy achieves 77.8% ESR (778/1,000), versus 91.7% when using our mixed-data reward model (4K real + 8K synthetic), an improvement of 13.9 percentage points.

CV3 GRPO reward-model adaptation dataHuman ESR
Real-only (4K)77.8%
Mixed (4K real + 8K synthetic)91.7%

This direct downstream ablation supports the benefit of including synthetic speech when adapting emotion rewards for TTS policy optimization. It is also consistent with the real-to-synthetic speech distribution gap discussed by Ge et al. (2026): supervision drawn only from human speech may not adequately match the policy’s synthesized output distribution.

The comparison measures downstream policy ESR, not the reward model’s own recognition accuracy. Since the mixed setting also contains more adaptation examples, it does not isolate data-domain effects from training-set size. A synthetic-only condition has not yet been evaluated, so we do not infer the incremental contribution of real speech beyond synthetic adaptation. Separately, Section 6.1 shows that the mixed-data RM achieves 81.98% consistency on human-verified positive samples from three unseen TTS systems, compared with 56.38% for Emotion2vec. This transfer result complements the downstream ablation but, because it is positive-only, does not measure rejection of unsuccessful emotional expressions on the unseen systems. We will add the new ablation and clarify these distinctions in the revision.

Additional Evidence. Human Speech Naturalness

We additionally evaluated perceived speech naturalness on 250 utterances per system, sampled evenly across the five emotions (50 utterances per emotion; 1,000 utterances across four systems). The mean MOS ratings are comparable between the original and Dr.EMO-trained backbones, with an increase of 0.020 points for each Dr.EMO variant.

BackboneBaseline MOSDr.EMO MOS
CosyVoice23.7923.812
CosyVoice33.8203.840

These descriptive human ratings provide complementary evidence that the stronger emotional expression does not come with an appreciable decrease in perceived naturalness in the evaluated samples. We do not interpret the small mean differences as evidence of statistical superiority.

Dear Reviewer xfQQ,

As we are approaching the final day of the rebuttal discussion period, we would like to sincerely thank you again for your time, constructive feedback, and encouraging assessment of our work.

We have posted a detailed response addressing your concerns, including additional experimental results, human evaluation reliability statistics, and corresponding clarifications to the manuscript.

We understand that this is a busy period, and we would be very grateful if you could take a moment to review our response and let us know whether it adequately addresses your concerns. If any questions or uncertainties remain, we would be happy to provide further clarification and discuss them before the discussion period closes.

Thank you again for your valuable feedback and consideration.

Best regards,

The Authors

Reviewer_qh6WOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper argues that SER models trained on human speech give unreliable emotion rewards for synthesized speech. The authors build a reward-adaptation set of 8K human-filtered CosyVoice2 utterances (generated from LibriTTS-test/SeedTTS-test texts with RAVDESS emotional prompts) plus 4K TESS/CREMA-D utterances, and fine-tune Emotion2vec and Qwen2.5-Omni-3B on it. On a 1K CosyVoice2 held-out set, Qwen2.5-Omni-SER reaches 84.1% "agreement" with human judgments, versus 58.7% for Emotion2vec. This model then serves as a discrete emotion reward (+3/-1) in standard multi-objective GRPO, together with an ASR reward (with a -2 override when R_asr < 0.5) and a group-median duration reward adopted from Gao et al. (2025b). On CosyVoice2-0.5B and CosyVoice3-0.5B, human-judged emotion success rate (ESR) rises from 61.5% to 91.1% and from 56.7% to 91.7%. The paper also reports a ranking reversal between Emotion2vec accuracy and human ESR, a positive-only transfer test of the reward model on three unseen TTS systems, and largely preserved WER/SIM on Seed-TTS-Eval and CV3-Eval. The optimizer is unmodified GRPO; the contribution is the adapted reward model, its data, and the empirical study.

Critical strengths
  1. The paper evaluates the end goal with humans, and this exposes a real proxy failure. The paper does not let the reward model grade itself. In Table 2, the Emotion2vec-reward CosyVoice3 policy gets 96.1% E2V-Acc but only 83.7% human ESR, whereas DR.EMO gets 82.0% E2V-Acc and 91.7% ESR; the same reversal holds on CosyVoice2 (85.8/79.3 vs. 80.8/91.1). E-SIM barely moves (95.8-96.1) while ESR ranges from 32.2% to 91.1%. Many papers report Emotion2vec accuracy and E-SIM as emotion metrics, so this is concrete and useful evidence that those metrics can rank systems wrongly.

  2. Large, consistent human-judged gains with a working multi-reward recipe. ESR improves by about 30-35 points on two backbones, and the per-emotion results (Figure 2) show gains on all four non-neutral classes (e.g., CosyVoice2 sad 48.0 -> 90.5, surprised 51.0 -> 99.0). The reward ablation in Appendix A shows why the ASR term and the duration term are needed (emotion-only reward drives WER to 78.95%; the duration term drops the prolonged-speech rate from 7.7% to 0.0%), and WER/SIM are checked on Seed-TTS-Eval and CV3-Eval. As a practical recipe for emotional TTS post-training, this is likely to be reused.

Critical weaknesses

1.The headline reward-reliability number is close to a trivial baseline. Table 1 gives only percentages. Assuming 200 utterances per emotion, the reported values of all four models fit only one set of counts: 816 of the 1,000 held-out utterances are human-positive and only 184 are negative. A reward that always says "target achieved" would therefore get 81.6% agreement, while giving GRPO no learning signal. Qwen2.5-Omni-SER (84.1%) is only 2.5 points above this. It catches 91% of successes but only about 54% of failures, so nearly half of the failed utterances still get a positive reward. Its balanced accuracy (72.6%) is close to Emotion2vec-SFT (71.4%). The claimed gain of "over 25 points" over Emotion2vec mostly comes from class imbalance.
2.Two human evaluations of the same baseline do not agree. Sections 3.1 and 5.1 use the same 200 texts, the same CosyVoice2 model and the same five emotions. Table 1 implies a baseline success rate of about 81.6%, but Table 2 reports 61.5%. Even directly from Table 1, angry has almost no failures, yet Figure 2 gives angry only 77.5%. Such a large gap means ESR depends heavily on prompt sampling or on the annotation process. No inter-annotator agreement is reported. This makes the 91% results harder to trust.
3.Novelty is limited, and the claimed source of the gain is not isolated. Ge et al. (2026), cited as the motivation, already showed that Emotion2vec fails on human-verified synthetic speech. They also fine-tuned Emotion2vec and Qwen2.5-Omni-3B on real plus human-filtered CosyVoice2 speech (including TESS and CREMA-D). RRPO (Wang et al., 2025a) already adapted an emotion reward model with human labels for CosyVoice2 RL. The duration reward comes from Gao et al. (2025b), and GRPO is unchanged. The remaining contribution is plugging this kind of reward model into GRPO. Even here, the key ablation is missing: there is no Qwen2.5-Omni reward fine-tuned on real speech only. Without it, the gain may simply come from turning a general audio LLM into an SER classifier, not from "de-biasing" with synthetic data.
4.The SOTA claim is not supported, and naturalness is not checked by humans. The CosyVoice models are prompted with acted RAVDESS audio, from the same pool used for reward data and RL. GPT-4o-TTS and Gemini get only a text instruction, and the IndexTTS2 setting is not described. In the human test, annotators see the target label and only answer yes or no, which favors exaggerated speech. UTMOSv2 drops on both backbones (3.48 to 3.21, 3.47 to 3.25), and the F0 range grows by 40% in Appendix D. The paper says the MOS drop may not reflect human perception, but gives no human naturalness rating to show this.

Questions for authors
  1. Please give the per-emotion positive/negative counts. Are they 196/128/198/168/126 positives? If so, how do you justify agreement as the reward-reliability metric, given that a constant "success" reward scores 81.6%? Please add balanced accuracy, sensitivity/specificity and, ideally, within-group ranking accuracy on GRPO rollouts.
  2. Why is the human success rate of CosyVoice2 on the same 200 texts about 81.6% in Section 3.1 but 61.5% in Table 2? Were the prompt audios, sampling seeds, annotators or protocol different? Please report inter-annotator agreement (e.g., Cohen's kappa) and the adjudication rate for every system.
  3. Do you have results with (a) Qwen2.5-Omni fine-tuned on the 4K real subset only, and (b) Emotion2vec-SFT, as GRPO rewards? If not, please narrow the claim that the gain comes from synthetic-speech de-biasing.
  4. Please explain exactly what the reward-model adaptation adds beyond Ge et al. (2026) (same backbones, similar data and filtering) and RRPO (Wang et al., 2025a).
  5. How was IndexTTS2 conditioned? Can you compare CosyVoice2/3 and DR.EMO under text-instruction control only, or give the commercial systems the same RAVDESS reference audio, so that the external comparison is fair?
8 comments

We appreciate the reviewer’s detailed analysis and recognition of the human-evaluated gains. We address the weaknesses and questions separately below.

W1. Reward Reliability and Class Imbalance

The class imbalance in Table 1 is not artificially introduced but naturally arises from the TTS output distribution. Specifically, the 1K utterances were generated by CosyVoice2 across five target emotions and subsequently evaluated by human annotators, resulting in 816 positive and 184 negative samples. This imbalance reflects the actual rollout distribution of synthesized speech under the evaluated generation conditions, rather than a deliberately constructed evaluation set. Our objective is not to develop a reward model optimized for an artificially balanced classification benchmark, but to obtain reliable reward signals aligned with human judgments on the TTS output distribution. Therefore, we adopt overall agreement as a distribution-aware measure of reward alignment.

Moreover, the reported 25.4-percentage-point improvement over Emotion2vec is evaluated on exactly the same samples and class distribution. Although the imbalance gives greater weight to positive samples, the improvement reflects a substantial increase in sensitivity from 53.2% to 90.8%, rather than any manipulation of the evaluation distribution. Meanwhile, balanced accuracy also improves from 68.2% to 72.6%. Detailed balanced accuracy, sensitivity, and specificity results for all four reward models are provided in our response to Q1. The lower specificity (54.3%) indicates that some unsuccessful expressions remain challenging to distinguish, but does not negate the improved alignment with human judgments under the target TTS distribution.

More importantly, the effectiveness of the reward model is ultimately demonstrated through downstream GRPO optimization, rather than offline agreement alone. Under the same GRPO framework, replacing Emotion2vec with Qwen2.5-Omni-SER improves human-evaluated ESR from 79.3% to 91.1% on CosyVoice2 and from 83.7% to 91.7% on CosyVoice3. These consistent gains demonstrate that adapting the reward model to the TTS output distribution provides effective optimization guidance and substantially improves human-perceived emotional expressiveness.

W2. Evaluation Criteria and Annotation Reliability

The discrepancy between the 81.6% human-positive rate in Section 3.1 and the 61.5% ESR in Table 2 arises from two different human annotation criteria applied to the same synthesis setting: a moderately relaxed criterion for reward-model development and a stricter criterion for final TTS evaluation.

Initially, we adopted a unified, strict evaluation criterion requiring clearly perceptible emotional expression, as used in our final TTS evaluation. However, we found this strict criterion unsuitable for reward modeling and early-stage GRPO optimization. Specifically, an overly strict criterion labels many utterances with weak but recognizable emotional cues as negative samples, causing the resulting SER reward model to reject such expressions. During early GRPO training, when the TTS policy struggles to generate sufficiently salient emotional expressions, a reward model trained under this strict criterion would assign the same negative emotion reward (-1) to most candidates within a rollout group. Consequently, these candidates would have little or no relative advantage from the emotion reward, limiting effective emotion-specific policy updates. To avoid this reward saturation and the resulting lack of useful emotion-specific gradients during early optimization, we adopt a moderately relaxed criterion for reward modeling, allowing the model to recognize weaker but identifiable emotional expressions and provide more informative reward signals that encourage exploration toward stronger emotional expression.

To address this issue, we adopt a moderately relaxed human evaluation criterion for RM construction, allowing the reward model to recognize and reward weaker but identifiable emotional expressions. This provides more informative reward signals during early optimization and encourages the policy to progressively explore stronger emotional expressions. In contrast, the final TTS evaluation retains the original strict criterion, requiring clearly perceptible emotional expression to assess the actual effectiveness of Dr.EMO. Consequently, the 81.6% and 61.5% results reflect different evaluation thresholds, while all TTS systems in Table 2 are evaluated under the same strict criterion. Notably, Dr.EMO achieves ESRs of 91.1% and 91.7% even under this more demanding evaluation.

The reliability of our human evaluation is further supported by independent annotation statistics. For Dr.EMO-trained CosyVoice2 and CosyVoice3, the initial inter-annotator agreement rates are 92.2% and 90.8%, with adjudication rates of 7.8% and 9.2%, respectively. The corresponding Cohen's Kappa values are 0.4394 and 0.4328. These statistics, together with the independent adjudication protocol described in Appendix C, provide transparent evidence of the annotation process. Detailed confidence intervals and annotation statistics are provided in our response to Q2.

W3. Methodological Novelty and Contribution Attribution

We emphasize that our work addresses a critical and pressing challenge in emotional TTS: improving the perceptual salience of emotional expression. We clarify that Dr.EMO is a unified post-training framework designed to enhance emotional expressiveness through two complementary stages: (i) modeling reliable emotion reward signals on the TTS output distribution, and (ii) optimizing the TTS policy through multi-objective GRPO. Our contributions lie in providing empirical insights into how to effectively construct reward models for emotional expression and demonstrating best human-evaluated emotional TTS performance among the evaluated systems by integrating the adapted reward model from Stage (i) into GRPO optimization.

Regarding Ge et al. (2026), its findings reveal that emotion reward models for synthesized speech are often fragile and exhibit limited generalization across TTS distributions. This observation directly motivates our design choice: rather than pursuing a universal RM covering all possible TTS distributions, Dr.EMO provides a targeted post-training pipeline for improving emotional expressiveness in specific TTS models, through reward adaptation followed by GRPO optimization. Furthermore, motivated by the generalization challenges identified in Ge et al., we carefully examine the transferability of our adapted RM to three unseen TTS generators in Section 6.1, achieving 81.98% human-label consistency compared with 56.38% for Emotion2vec.

**Regarding RRPO (Wang et al., 2025a), **the two works address fundamentally different challenges in reward-based emotional TTS optimization. RRPO primarily targets reward hacking, where TTS policies exploit spurious acoustic patterns to obtain artificially high rewards, and improves RM robustness through hybrid regularization. In contrast, Dr.EMO addresses a more fundamental issue: the misalignment between SER predictions and human judgments on synthesized speech, which can exist even before policy optimization. Our framework therefore focuses on constructing human-aligned reward signals on the target TTS output distribution and utilizing them to improve perceptually salient emotional expression. Rather than emphasizing robustness against adversarial reward exploitation, we investigate how reward reliability translates into effective downstream TTS optimization, supported by human evaluation across two backbones.

Importantly, our additional ablation further demonstrates the effectiveness of incorporating human-verified synthesized speech into reward model training. Under identical CosyVoice3 GRPO settings, a Qwen2.5-Omni reward model adapted using only 4K real utterances achieves 77.8% human-evaluated ESR, whereas our mixed-data RM achieves 91.7%, a 13.9-percentage-point improvement. This result demonstrates that the benefits extend beyond simply fine-tuning a general-purpose audio LLM for emotion classification. Moreover, Dr.EMO achieves ESRs of 91.1% and 91.7% across two TTS backbones, establishing the practical effectiveness of our framework for improving human-perceived emotional expressiveness.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

We thank all reviewers for their constructive feedback. We first summarize the contributions of our work, then address the main concerns shared across reviewers and highlight the additional experimental evidence.

Main Contributions

Our work addresses a critical challenge in emotional TTS: improving the perceptual salience of emotional expression in synthesized speech. We propose Dr.EMO, a post-training framework consisting of (i) constructing reliable, human-aligned emotion reward signals on TTS output distributions, and (ii) optimizing TTS policies through multi-objective GRPO. Our contributions lie in establishing empirical insights into effective emotion reward modeling and demonstrating how reliable rewards enable state-of-the-art human-evaluated emotional expressiveness. Dr.EMO achieves 91.1% and 91.7% ESR on CosyVoice2 and CosyVoice3, respectively, outperforming the corresponding Emotion2vec-reward GRPO baselines by 11.8 and 8.0 percentage points.

GR1. Reward Reliability and Human Evaluation

We provided additional analyses to substantiate the reliability of our reward model and human evaluation.

  • Reward reliability beyond class imbalance. Our adapted Qwen2.5-Omni-SER improves agreement with human judgments from 36.4% to 84.1%. We additionally reported balanced accuracy, sensitivity, and specificity for all four reward models. Compared with Emotion2vec, sensitivity increases from 53.2% to 90.8%, while balanced accuracy improves from 68.2% to 72.6%. We clarified that the class imbalance naturally arises from the evaluated TTS output distribution.

  • Evaluation criteria and annotation reliability. We clarified the moderately relaxed criterion used for RM construction, which preserves informative rewards during early GRPO optimization, and the strict criterion used for final human ESR evaluation. We further provided initial inter-annotator agreement (92.2%/90.8%), adjudication rates (7.8%/9.2%), Cohen's κ (0.4394/0.4328), and confidence intervals for the Dr.EMO-trained systems.

GR2. Additional Experimental Evidence

Following the reviewers' suggestions, we conducted additional experiments to directly investigate the contribution of synthetic data and the preservation of speech naturalness.

  • Synthetic data substantially improves downstream emotional expressiveness. We additionally trained CosyVoice3 using a reward model adapted exclusively on 4K real utterances. Under identical GRPO settings, the real-only RM achieves 77.8% human ESR, compared with 91.7% using our mixed-data RM (4K real + 8K synthetic), a substantial 13.9-percentage-point improvement.

  • Human evaluation confirms comparable speech naturalness. We conducted an additional human MOS evaluation on 1,000 utterances across four systems, balanced across five emotions. Dr.EMO achieves comparable or slightly higher MOS scores than the original backbones (CosyVoice2: 3.792 → 3.812; CosyVoice3: 3.820 → 3.840). These results provide direct perceptual evidence that the observed UTMOSv2 decreases do not correspond to an appreciable naturalness reduction in the evaluated samples.

GR3. Generalization and Summary of Clarifications

We also addressed concerns regarding generalization, methodological contributions, fair comparisons, and reproducibility.

  • Generalization. Section 6.1 demonstrates that our RM transfers to three unseen TTS generators without further adaptation, achieving 81.98% emotion consistency on human-verified positive samples, compared with 56.38% for Emotion2vec. Section 6.2 further demonstrates that Dr.EMO largely preserves zero-shot TTS, voice conversion, and cross-lingual synthesis capabilities.

  • Methodological contributions. We clarified that Dr.EMO focuses on constructing human-aligned rewards for synthesized speech and translating improved reward reliability into effective on-policy TTS optimization, distinguishing our objective from prior work on SER generalization and reward hacking.

  • Fair comparisons and reproducibility. We clarified the emotion-conditioning protocols of external baselines, confirmed identical evaluation conditions between the original CosyVoice backbones and their Dr.EMO-trained counterparts, and provided detailed GRPO hyperparameters and rollout configurations in our anonymous repository.

Together, these additional experiments and analyses strengthen the evidence for our central contribution: effective human-aligned emotion reward modeling enables substantial improvements in emotional TTS through GRPO, while maintaining comparable perceptual speech naturalness and general synthesis capabilities. We believe the rebuttal has directly addressed the reviewers' central concerns and further substantiated the technical soundness and empirical significance of Dr.EMO.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.