- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes a reference-conditioned speaker-embedding editing method to enhance emotional expression in a pretrained text-to-speech (TTS) system without retraining the synthesizer. Specifically, the authors train a FiLM-based residual predictor on paired synthetic neutral and target-emotion embeddings. Given a normalized CAM++ embedding extracted from a neutral reference recording and a target emotion label, the predictor estimates a reference-dependent residual. The predicted residual is applied with a scalar editing strength to modify the speaker condition supplied to the Flow-based acoustic model of CosyVoice 3 (CV3), while keeping the speech-token plan fixed across editing conditions and the pretrained synthesis components frozen.
Critical strengths
S1. Leveraging a Pretrained TTS Model for Emotional Enhancement
This paper proposes a speaker-embedding editing method to enhance emotional expression while keeping the pretrained CV3 synthesizer frozen. By keeping the speech-token plan fixed across editing conditions and reusing the existing acoustic conditioning interface, the method requires training only a lightweight FiLM-based residual editor without updating the pretrained synthesis components. This approach is a meaningful strength, as it enables additional emotion control while leveraging the synthesis capabilities of a pretrained TTS model.
Critical weaknesses
W1.Insufficient Evidence for the Distinct Benefits of Reference-Conditioned Editing
Although FiLM achieves higher target-emotion cosine than Shared residual, further analysis is needed to establish its distinct benefits beyond numerical improvements. In particular, its advantage over Shared residual largely disappears when residual norms are matched in Table 2, leaving the contribution of reference-dependent direction prediction unclear. The paper also discusses related methods such as EmoKnob and EmoSteer-TTS, but does not provide direct experimental comparisons with these approaches [1]. Furthermore, the proposed method requires training on synthetic neutral–target embedding pairs, while the main inference evaluation is limited to neutral references. Given these constraints, further evidence is needed to clarify the practical advantages of reference-conditioned editing beyond the reported gains over native emotional instructions, and to better position it relative to existing emotion-control approaches [2,3], particularly in terms of emotional expressiveness and speaker similarity.
[1] Wang, Siyi, et al. "Cocoemo: Composable and controllable human-like emotional tts via activation steering." arXiv preprint arXiv:2602.03420 (2026). [2] Um, Se-Yun, et al. "Emotional speech synthesis with rich and granularized control." ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2020. [3] Li, Tao, et al. "Cross-speaker emotion disentangling and transfer for end-to-end speech synthesis." IEEE/ACM Transactions on Audio, Speech, and Language Processing 30 (2022): 1448-1460.
W2. Trade-off Between Emotional Enhancement and Speaker Similarity
The proposed method improves emotional expression at higher editing strengths, but this comes at the cost of reduced speaker similarity. In Table 3 and Appendix C, speaker-similarity MOS decreases by 0.448 in Chinese and 0.224 in English at α = 5. While the authors acknowledge this trade-off and do not claim to disentangle speaker identity from emotion, preserving speaker characteristics remains essential for voice cloning. Therefore, it is important to establish whether the proposed method achieves better emotional expression than existing editing approaches at comparable speaker-similarity levels. Without such a comparison, its practical advantage remains insufficiently demonstrated.
W3. Lack of Accessible Audio Demonstrations
Although the paper reports perceptual evaluation results in Table 3, no accessible audio demo page or sample links are provided in the manuscript.
Questions for authors
Q1. Perceptual Comparison with Shared Residual Editing
Table 3 reports listening results only for unedited CV3 and FiLM, whereas Table 2 also includes Shared residual and norm-matched FiLM. Could the authors clarify why these baselines were excluded from the listening evaluation?
Q2. Details of Synthetic Training Data Construction
Could the authors provide more details on the construction of the 195,301 synthetic neutral–target embedding pairs, including the number of samples per target emotion, the speech-generation model and synthesis settings, and the source and composition of the input texts?
Q3. Applicability to Non-Neutral References
The authors acknowledge that the proposed method has not been validated for non-neutral references. Could the authors further clarify the expected behavior of the current editor when applied to an already emotional reference, such as Angry → Happy? In particular, is there any preliminary evidence indicating whether the learned residuals remain meaningful outside the neutral-reference setting?
2 comments
Official Comment by Reviewer L5dS
Thank you for the additional experiments and detailed clarifications. I appreciate the authors' continued efforts to address my concerns.
The new reference-averaged control provides useful evidence that the learned reference-dependent predictions contribute to the improvements achieved by V2. However, while combining the learned and shared directions improves performance, I remain cautious about interpreting this result as evidence of strong methodological novelty or the superiority of the original FiLM approach. I will consider the additional evidence alongside the perspectives of the other reviewers before finalizing my assessment.
Thank you again for your thoughtful responses and additional efforts.
Official Comment by Reviewer L5dS
Thank you for the detailed responses and clarifications.
Regarding Q3, I understand that the proposed method has been evaluated only on neutral references and that its applicability to non-neutral references remains unvalidated. (Section 6 — Limitations: Adaptive dosing, arbitrary emotional references, arousal control and cross-backbone transfer of the trained editor remain unvalidated.) My question was not intended to suggest that the paper claims such generalization, but rather to better understand the practical scope of the approach and whether the authors had any preliminary evidence or insights to share.
I also appreciate the additional experiments reported in your responses to other reviewers, particularly the improvements achieved by V2 over dose-tuned shared editing. However, given that the original FiLM rule performs worse than dose-tuned shared editing at matched loss, I am not fully convinced that the V2 results alone sufficiently address my original concerns. It would be helpful to provide further analysis and clarification of this performance difference, as well as how the additional evidence from V2 relates to the central contributions of the original paper.
Thank you again for your thoughtful responses and additional efforts.