- Rating
- 2(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes a GRPO framework for improving emotional expressiveness in TTS. The authors first adapt an SER reward model using synthetic speech to better align rewards with human judgments, and then combine emotion, ASR, and duration rewards during GRPO training. Experiments show substantially improved target emotion expression while largely preserving general TTS quality.
Critical strengths
- This paper identifies a practically issue in emotional TTS post training. The mismatch between SER based rewards and human judgments on synthetic speech is clearly demonstrated, and adapting the reward model to synthetic speech leads to better human alignment.
- The experimental validation is strong. The method is tested on two TTS backbones and compared with several reward model variants. The human evaluation and ablation results also support the main findings.
Critical weaknesses
- The methodological novelty is limited. The main pipeline combines reward model adaptation on synthetic speech with standard GRPO and existing ASR and duration rewards. The contribution is useful but most individual components are not new.
- The reward model relies on manually curated synthetic speech, which may limit scalability and generalization to broader emotions and TTS systems.
Questions for authors
- How much of the improvement in Qwen2.5-Omni-SER comes specifically from the synthetic data?
- The proposed models show lower UTMOSv2 scores. Did you conduct human evaluation of naturalness to check whether stronger emotion comes at the cost of speech quality?
- Could you provide the missing GRPO settings, such as the group size , clipping ratio, KL coefficient, and sampling strategy used for rollout generation?
- How sensitive is the reward model to the choice of generator, and would using synthetic speech from multiple TTS systems further improve generalization?
3 comments
Q2. Human Evaluation of Speech Naturalness
UTMOSv2 is a model-based metric designed to estimate human-perceived speech quality. However, its predictions may not fully capture perceptual naturalness for emotionally expressive synthesized speech, particularly when stronger emotional expression introduces prosodic characteristics that differ from those typically represented in automatic quality estimation.
To directly investigate whether improved emotional expressiveness compromises perceived speech quality, we conducted an additional human Mean Opinion Score (MOS) evaluation comparing the original CosyVoice2 and CosyVoice3 backbones with their Dr.EMO-trained counterparts.
For each system, we sampled 250 utterances, evenly distributed across the five target emotions (50 per emotion), resulting in 1,000 evaluated utterances across four systems. The evaluation focuses on human-perceived speech naturalness and overall quality, independently of whether the intended emotion is successfully expressed.
| Backbone | Baseline MOS | Dr.EMO MOS | Difference |
|---|---|---|---|
| CosyVoice2 | 3.792 | 3.812 | +0.020 |
| CosyVoice3 | 3.820 | 3.840 | +0.020 |
The results show that Dr.EMO achieves slightly higher average human MOS scores on both backbones, with an improvement of 0.020 points in each case. In contrast to the decreases observed in UTMOSv2, the human evaluations do not show a corresponding reduction in average perceived speech quality.
These findings suggest that the substantial improvements in emotional expressiveness achieved by Dr.EMO do not necessarily come at the expense of human-perceived naturalness. The observed MOS differences are small and are reported descriptively rather than as statistically significant improvements. Nevertheless, the results provide complementary human perceptual evidence that the reductions in automatic quality estimates should not be directly interpreted as equivalent degradation in perceived naturalness.
Q3. GRPO Hyperparameters and Rollout Sampling Configurations
Both CosyVoice2 and CosyVoice3 use the same core GRPO optimization settings, including a group size of K = 8, a KL regularization coefficient of 0.001, and a PPO clipping ratio of 0.2. The global batch size is 32, and the learning rate is set to 1 × 10⁻⁶. These optimization settings are shared across the two backbones and are also provided in our anonymous code repository.
Due to differences in model architectures and speech-token generation mechanisms, the two backbones adopt different rollout sampling configurations:
- CosyVoice2: Sampling temperature = 1.0, top-p = 0.95, and maximum generation length = 512 tokens.
- CosyVoice3: Sampling temperature = 1.0, top-p = 1.0, and a dynamic maximum generation length of min(20L, 1024), where L denotes the number of input text tokens. The shared GRPO settings maintain a consistent policy optimization procedure across both backbones, while the backbone-specific sampling configurations accommodate their respective generation mechanisms. We will explicitly document these hyperparameters and rollout settings in the revised manuscript to improve reproducibility.
Q4. Generator Sensitivity and Multi-Generator Reward Training
As demonstrated in Section 6.1, although our reward model is trained using synthetic speech exclusively from CosyVoice2, it generalizes to three unseen TTS systems (IndexTTS2, GPT-4o-TTS, and Gemini-2.5-Pro) without further adaptation. On 949 human-verified positive samples, Qwen2.5-Omni-SER achieves 81.98% emotion consistency, compared with 56.38% for Emotion2vec, demonstrating substantial cross-generator transfer.
Importantly, Dr.EMO does not require a universal reward model covering all possible TTS output distributions. Instead, it provides a post-training framework that supports reward adaptation to target synthesis distributions and subsequent TTS policy optimization through GRPO. Its effectiveness is demonstrated on both CosyVoice2 and CosyVoice3, achieving human-evaluated ESRs of 91.1% and 91.7%, respectively. While incorporating multiple synthetic generators may further improve reward robustness, our results demonstrate that single-generator reward adaptation already enables effective transfer to unseen TTS systems, without requiring multi-generator training.
Dear Reviewer iYqm,
As we are approaching the final day of the rebuttal period, we would like to thank you again for your time and constructive feedback on our paper.
We have provided detailed responses to all your concerns, including additional experiments on the contribution of synthetic data and human-evaluated speech naturalness, as well as further clarifications regarding methodological contributions, generalization, and implementation details. We hope these new results and explanations have addressed your concerns and provided stronger evidence supporting our contributions.
We understand that this is a busy period. We would greatly appreciate it if you could take another look at our responses and let us know whether any concerns remain. We would be happy to provide further clarification before the discussion period closes.
Thank you again for your time and consideration.
Best regards,
The Authors
We thank the reviewer for the constructive feedback and recognition of our experimental validation. We address the concerns below.
W1. Methodological Novelty
We emphasize that this work addresses a critical and pressing challenge: substantially enhancing the salience of emotional expression in TTS-generated speech. Importantly, we need to clarify that Dr.EMO is a post-training framework specifically designed to enhance emotional expressiveness in TTS models through two key stages: (i) modeling accurate emotion reward signals on TTS output distributions (Section 3), and (ii) optimizing TTS policies via GRPO using these reward signals (Section 4).
Our contributions lie in (i) providing empirical insights into how to effectively construct reward models for emotional expression, and (ii) experimentally demonstrating state-of-the-art performance in emotional TTS by combining GRPO with the reward model developed in the first stage (Section 5).
W2. Generalization and Scalability
We emphasize that Section 6 has already demonstrated the generalization capabilities of Dr.EMO from two perspectives: (i) the generalization of reward models constructed within the Dr.EMO framework to unseen TTS output distributions (Section 6.1), and (ii) the generalization of Dr.EMO-trained state-of-the-art TTS models to other synthesis tasks (Section 6.2).
Furthermore, we argue that a single reward model cannot be expected to cover the output distributions of all possible TTS systems. Instead, Dr.EMO is a post-training framework designed to enhance emotional expressiveness in individual TTS models, and we have demonstrated its effectiveness across different TTS backbones (Section 5). More importantly, human annotation is indeed necessary to obtain reliable emotional supervision that lies beyond the capabilities of existing automatic methods, even at the cost of reduced scalability. Human feedback is indispensable for evaluating subjective emotional expression and ensuring reliable reward modeling.
Q1. Contribution of Synthetic Data to Reward Modeling
As shown in Table 1 (I–II), adapting Qwen2.5-Omni with 8K human-verified synthetic utterances and 4K real utterances improves its agreement with human judgments from 36.4% to 84.1%, a substantial gain of 47.7 percentage points. Compared with the widely used Emotion2vec baseline (58.7%), our adapted Qwen2.5-Omni-SER also achieves a 25.4-point improvement.
To further isolate the contribution of synthetic data to downstream TTS optimization, we additionally conducted an ablation using a reward model adapted exclusively on 4K real speech utterances, without synthetic speech. We then trained the CosyVoice3 policy using this real-only reward model under the same GRPO framework and evaluated its emotional expressiveness using the same strict human evaluation criterion.
The real-only reward model achieves an ESR of 77.8% (778/1,000), whereas the mixed-data reward model achieves 91.7% (917/1,000), representing a 13.9-percentage-point improvement. This result demonstrates that incorporating human-verified synthetic speech into reward modeling substantially improves downstream emotional expressiveness under the evaluated training configuration.
Importantly, Table 1 quantifies the overall improvement in reward–human agreement after adaptation, while the additional ablation directly evaluates the impact of different reward-model training datasets on downstream GRPO performance. Since the mixed dataset also contains more training examples, the ablation does not fully disentangle the effect of synthetic speech from increased training data volume. Nevertheless, it provides direct empirical evidence that mixed-data reward adaptation yields substantially better emotional TTS performance than real-only adaptation.