- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes SR-GRPO for instruction-guided zero-shot TTS. The method uses a frozen SFT model to score the same generated speech tokens with and without the instruction, using the difference in their length-normalized log-likelihoods as an instruction reward. This reward is combined with speaker-similarity and duration rewards for policy optimization. The authors also introduce a 650-sample Chinese benchmark covering six categories of instructions. SR-GRPO improves instruction-following rate, while speaker similarity decreases. Reward ablations, comparisons with classifier-free guidance and Best-of-N decoding, evaluations on external benchmarks, and experiments on CosyVoice3 provide further evidence for the approach.
Critical strengths
-
The likelihood-contrast reward is simple to implement and avoids an external instruction evaluator. The reward ablations provide useful evidence for the proposed formulation.
-
The evaluation covers multiple baselines, instruction categories, and reward configurations. The CosyVoice3 experiment provides additional evidence beyond the authors' own backbone.
-
The benchmark evaluates instruction following alongside speaker similarity and intelligibility, making the trade-off between controllability and speaker preservation explicit.
Critical weaknesses
-
The instruction reward is an indirect measure of adherence. A higher conditional likelihood relative to the instruction-free baseline does not necessarily mean that all requested attributes are satisfied. The ranking results provide some support, but the paper does not directly establish how well the reward correlates with human judgments across instruction categories.
-
The perceptual cost of reduced speaker similarity remains unclear. SIM drops from 0.756 to 0.733 after RL, but the human study evaluates instruction following only and excludes speaker timbre and general audio quality. Reference-aware human evaluations would help determine whether the gain in adherence comes at a meaningful cost to speaker identity or naturalness.
-
The robustness of some gains is uncertain. SR-GRPO improves IFR by only 0.92 percentage points over Best-of-8. The paper does not report paired confidence intervals for system differences. In addition, Best-of-N uses CPMI alone, whereas RL optimizes speaker similarity and duration as well. A comparison using the same combined rewards would better isolate the benefit of policy optimization.
Questions for authors
-
How strongly does the likelihood-contrast reward correlate with human-rated instruction adherence, particularly for compositional instructions where several attributes must be satisfied simultaneously?
-
Could you provide reference-aware human evaluations of speaker similarity and naturalness for SFT versus SR-GRPO? Is the SIM evaluation independent of the WavLM-based representation used for speaker conditioning and the timbre reward?
-
Could you report paired uncertainty estimates for the main comparisons and compare SR-GRPO against Best-of-N selection using the same combined reward signals?
5 comments
W2&Q2: Perceptual Trade-offs and Independence of Speaker Similarity Evaluation
Thank you for highlighting this limitation. We have added human evaluations of speaker similarity and overall audio quality, clarified SIM's dependence on the WavLM-based pipeline, and assessed the robustness of the speaker-similarity findings across additional encoders.
Reference-aware human evaluation
We evaluated SFT and SR-GRPO using speaker-similarity MOS (SMOS) and overall audio-quality MOS (QMOS). For SMOS, raters listened to the corresponding reference audio to assess speaker identity. All nine raters evaluated all 60 clips for each metric.
| Metric | SFT | SR-GRPO |
|---|---|---|
| SMOS | 4.01 (0.20) | 3.75 (0.25) |
| QMOS | 3.79 (0.14) | 3.83 (0.08) |
Values are means; parentheses indicate 95% confidence-interval half-widths. Higher is better.
The lower SMOS after RL is consistent with the SIM decrease from 0.756 to 0.733, suggesting a trade-off between instruction adherence and perceived speaker similarity. Mean QMOS remained similar. Details are provided in Appendix A.7.
SIM dependence and cross-encoder robustness
SIM is not fully independent of the WavLM-based representation used for speaker conditioning and the timbre reward.
To assess the robustness of the speaker-similarity findings across encoders and reduce reliance on the WavLM-based evaluator, we additionally evaluated SFT and SR-GRPO using three WeSpeaker encoders:
| Speaker encoder | SFT | SR-GRPO |
|---|---|---|
| ResNet34 | 0.8633 | 0.8473 |
| CAM++ | 0.8776 | 0.8657 |
| ECAPA1024_LM | 0.8640 | 0.8543 |
All three encoders show lower speaker similarity for SR-GRPO than for SFT, consistent with SIM and SMOS. These results and the discussion of SIM dependence are included in the appendix subsection "Additional Speaker-Encoder Evaluation."
Overall, the human ratings and cross-encoder results consistently suggest a speaker-similarity cost alongside improved adherence, while mean overall audio quality remains similar.
W3&Q3: Statistical Robustness and Reward-Matched Comparison with Best-of-N
Thank you for raising these concerns. We have added paired uncertainty estimates and combined-reward Best-of- results, and clarified the inference-cost trade-off.
1. Paired uncertainty estimates
For automatic IFR, we use all 650 matched inputs, 10,000 paired bootstrap resamples for 95% confidence intervals, and two-sided McNemar tests with Holm correction across six comparisons. Confidence intervals are pointwise.
| Baseline | SR-GRPO improvement (pp) | 95% paired CI (pp) | Holm-adjusted |
|---|---|---|---|
| CosyVoice3 | |||
| VoxCPM2 | |||
| Qwen-Audio-3.0-TTS-Plus | |||
| MiMo-V2.5-TTS | |||
| ACE-Instruct-TTS-SFT | |||
| Best-of-8, CPMI-only |
SR-GRPO significantly outperforms SFT and three external baselines after correction. Its differences from CPMI-only Best-of-8 and Qwen-Audio-3.0-TTS-Plus are not statistically significant, and we have qualified our claims accordingly. Details are provided in Appendix A.10.
For subjective scores, paired analyses on 30 matched inputs per metric yield:
| Metric | SR-GRPO minus SFT | 95% paired bootstrap CI |
|---|---|---|
| IFMOS | ||
| SMOS | ||
| QMOS |
These results support improved adherence with an average perceptual similarity cost, but no statistically detectable quality difference. Evidence for the SMOS decrease is sensitive to the test used. Details are provided in Appendix A.9.
2. Combined-reward Best-of-N
We added CPMI+timbre selection using the training weights of 0.6 and 0.3, respectively. Each reward is normalized separately by subtracting its within-input candidate-group mean and dividing by its group variance before combination.
Overall results (650 inputs). Bold indicates the best value in each row.
| Scoring | Metric | |||||
|---|---|---|---|---|---|---|
| CPMI-only | IFR (%) | 67.08 | 68.92 | 71.23 | 74.31 | 71.08 |
| CPMI-only | SIM | 0.756 | 0.739 | 0.731 | 0.720 | 0.722 |
| CPMI+timbre | IFR (%) | 67.08 | 68.46 | 70.00 | 72.61 | 70.77 |
| CPMI+timbre | SIM | 0.756 | 0.750 | 0.742 | 0.735 | 0.732 |
Both scoring schemes attain their highest overall IFR at N=8, with no further improvement at N=16. Adding timbre reward improves SIM at every N>1, but yields lower IFR. At N=8, SIM increases from 0.720 to 0.735, while IFR decreases by 1.70 percentage points.
SR-GRPO achieves 75.23% IFR, numerically exceeding CPMI-only and combined-reward Best-of-8 by 0.92 and 2.62 percentage points, respectively. Details are provided in Appendix A.5.
This comparison is partially reward-matched: the length/duration reward cannot be computed at inference time and is excluded from BoN.
3. Inference cost
Best-of-8 generates eight candidates and performs reward scoring, whereas SR-GRPO generates one candidate without reward-based selection. Assuming comparable output lengths, BoN-8 requires approximately eight times the candidate-generation work, plus CPMI scoring and, for the combined variant, timbre scoring.
Our conclusion is therefore narrower: SR-GRPO significantly improves over SFT and avoids BoN-8’s sampling and scoring overhead, but does not establish statistically significant IFR superiority over CPMI-only BoN-8.
W1&Q1: Correlation Between Likelihood-Contrast Reward and Human-Rated Instruction Adherence
We thank the reviewer for raising this important point. We agree that a higher likelihood contrast does not guarantee that every requested attribute is satisfied. To assess whether CPMI nevertheless provides a meaningful adherence signal, we examine its alignment with human judgments and external evaluation.
(1) Direct comparison with human-rated instruction adherence. Three raters performed A/B comparisons on 120 category-stratified inputs. Importantly, raters were asked to choose the output that better followed the instruction, not the output with better audio quality or greater overall appeal. These judgments therefore assess relative instruction adherence rather than general preference.
Among the 87 pairs with a clear majority preference, CPMI agrees with the human majority on 70.1% of pairs (95% CI: 59.8–79.3%), with Cohen's κ=0.404. The signed CPMI difference also correlates positively with the proportion of raters preferring A for instruction adherence (Spearman's ρ=0.374, CI: 0.180–0.539). These results indicate moderate positive alignment with human-rated instruction adherence: CPMI is informative, but imperfect.
(2) Complementary evidence from external evaluation. We do not use CPMI itself as an evaluation metric. Under IFR evaluation by Gemini-2.5-Pro, an external evaluator independent of the policy model, IFR decreases monotonically from 71.23% for Rank-0 to 63.08% for Rank-4. CPMI-based Best-of-8 selection also increases IFR from 67.08% to 74.31%. These findings complement the human results by supporting the usefulness of CPMI for selecting outputs with better instruction adherence.
Full results are provided in Appendix A.5.