- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes a training-free steering method to improve emotion control in TTS. The key idea is to decompose a conventional emotion steering vector into a shared component which represents general emotionality and a residual component which encodes specific emotion information. Therefore, the two vectors could be weighted independently for more precise control without fine-tuning the TTS backbone. Experiments on IndexTTS-2 and CosyVoice2 show that the proposed method achieves more accurate emotion control while preserving speech naturalness and speaker similarity.
Critical strengths
- The method is simple but practically useful. The proposed method first separates the emotion vector into shared and specific residual components, and then controls them independently. This formulation is easy to understand and requires no retraining of the TTS backbone.
- The experimental analysis is well aligned with the main claim. Results on two different TTS backbones, together with OOD and human evaluations, make the empirical evidence reasonably convincing.
Critical weaknesses
- The methodological novelty is somewhat limited. The vector extraction procedure and overall inference framework are largely inherited from CoCoEmo. While the resulting improvement is meaningful, the method advance itself is fairly incremental.
- The functional interpretation is not fully validated. The split is defined using an unweighted centroid over only a few emotion categories, so it is unclear whether the same interpretation holds with more emotions or model families.
Questions for authors
- How sensitive is the decomposition to the choice of emotion categories used to compute the centroid? For example, would the conclusions remain similar if the emotion set were changed or expanded?
- The main experiments use and . How robust is this setting across different emotions and backbones?
Weakness 1: Methodological novelty relative to CoCoEmo
We thank the reviewer for recognizing that the improvement is meaningful and practically useful.
Reusing CoCoEmo’s pipeline for fair comparison. We adopted CoCoEmo’s extraction, layers, and injection rule deliberately, so the comparison is controlled: every difference comes from how the steering vector is composed. EmoRES itself is not tied to this pipeline. Applied to EmoSteer-TTS, which steers the flow-matching module instead of the speech LM, it improves all four emotion metrics on CosyVoice2 (results in Table 4 of revised paper).
Contribution. CoCoEmo establishes where to steer. We examine what the emotion vector contains. Each vector splits into a shared component that moves speech away from neutral and a residual that specifies the emotion. Conventional composition of CoCoEmo fixes the ratio between the two. Our EmoRES method makes it controllable.
CoCoEmo cannot match EmoRES. Increasing CoCoEmo’s steering strength α saturates the emotion metrics beyond α=3 while S-SIM and WER degrade (Fig. 1). With the injected norm held equal, EmoRES wins at every α with IndexTTS-2. On held-out IEMOCAP, ρ improves from 22.00% to 48.13%, and human Dom-hit improves by about 19 points.
Generality. Beyond CoCoEmo’s two backbones, performance gains also hold on CosyVoice3 and Qwen3-TTS (Appendix E) and across three SER evaluators (Appendix H).
Weakness 2 and Question 1: Sensitivity of the decomposition to the emotion set
In the experiments of the original manuscript, we used 4 emotions to compute the centroid. To test the sensitivity of the choice of emotion categories to compute the centroid of emotion vectors, besides the original four emotions (angry, happy, sad, surprised), we added two additional emotions, fear and disgust. Table H shows the performance difference between the set of emotions used to construct the mean vector. On the CREMA-D evaluation set (angry, happy, sad), for EmoRES, the mean values of E-SIM, TEP, ρ, and H-rate drop a little when using more emotions for vector construction. When comparing EmoRES with CoCoEmo, no matter using 4 or 6 emotions, EmoRES outperforms CoCoEmo on ρ and H-rate by a large margin, demonstrating that the conclusion remains similar if the emotion set were expanded.
Table J shows the performance of EmoRES using the 6-emotion centroid on the RAVDESS dataset (6 emotions). EmoRES also outperforms CoCoEmo on E-SIM and TEP by a large margin.
Table H. Ablation on expanding the emotion set for computing the centroid from 4 emotions to 6 emotions for EmoRES with the CosyVoice 2 backbone on the CREMA-D evaluation set. For CoCoEmo, since the steering formula of Eq. (2) is independent of the emotion centroid, the results of the original emotions do not change when new emotions are used to calculate the emotion centroid.
Table J. Expanding the emotion set for computing the centroid with 6 emotions for EmoRES with the CosyVoice 2 backbone on the RAVDESS evaluation set (6 emotions). Since RAVDESS only has one emotion per clip, ρ and H-rate do not apply to this setting.
Question 2: Robustness of the coefficient setting across emotions and backbones
Robustness across different emotions: EmoRES achieves higher mean ρ, H-Rate, TEP and E-SIM than CoCoEmo for angry-, happy- and sad-dominant IEMOCAP requests on IndexTTS-2. These consistent gains support the applicability of a common coefficient setting across the tested emotion groups, without emotion-specific tuning.
Table K. Per-emotion improvements of EmoRES compared to CoCoEmo on IEMOCAP with IndexTTS2 under the setting of λc=1 and λr=3.
Robustness across different backbones: λr=3 was selected only on CREMA-D and applied unchanged everywhere else. It improves multiple emotion-related metrics over CoCoEmo on 4 backbones with different conditioning interfaces (Tables 1,6,7), on held-out IEMOCAP, with emotional speaker prompts (Table 8), without native emotion conditioning (Table 9), and under three independent SER evaluators (Table 10).