- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes CycleSpeech, a framework that couples instruction-controlled speech generation with paralinguistic speech understanding through a shared, structured voice profile. Its key contribution is a reciprocal design: a forward cycle checks whether generated speech realizes the intended profile attributes, while a backward cycle reconstructs speech from an inferred profile to test whether the profile preserves speaking style. It introduces CycleGRPO, which alternates policy updates for generation and understanding using reciprocal feedback, without human preference annotations or a separately trained preference reward model. The authors also construct a 20k bilingual instruction–speech–profile dataset supporting both understanding supervision and generation verification. Validation via bilingual automatic metrics, subjective evaluations, out-of-domain tests, and controlled ablations shows improved instruction following, style accuracy, and categorical profile recovery, with cycle feedback and evolving bidirectional updates contributing to the gains.
Critical strengths
Firstly, the paper is well-written with clear structure. Moreover, it transforms one-way generation–understanding supervision into a bidirectional, verifiable loop anchored on a structured voice profile, and operationalizes this idea as a trainable and evaluable system through CycleGRPO and a companion bilingual dataset.
Critical weaknesses
Here are the two main critical weaknesses, stated concisely:
-
Potential circular validation in form. Although a fixed gold profile and independent content/style rewards partially anchor training, the forward and backward loops still share the same GEN/UND backbone, schema, and annotation distribution; the backward loop conditions GEN on profiles predicted by UND, so GEN may learn to satisfy UND’s systematic biases rather than genuine profile–speech alignment, especially since the profile reward dominates the backward objective.
-
Same-source verification bias / self-confirmation. Because UND both verifies generated speech and supplies inferred profiles for reconstruction, the loop risks self-confirmation: errors or biases in UND and the profile annotations can be reinforced rather than corrected. The ablations and OOD tests mitigate but do not fully eliminate this concern; stronger external validation, human evaluation, and cross-model/cross-annotator checks would be needed to show the gains reflect real controllability.
Questions for authors
- In the Section 3.3 which explains Joint SFT, the token-to-wav decoder is frozen. Is it trainable in the GEN-only warm-up stage? What are the relationship between "predicted wavs" and "recovered wavs", and the relationship between "predicted profile" and "recovered profile", which are mentioned in Figure 1, during the joint SFT process? What are the detailed configurations of θ_G and θ_U and are they connected in the joint SFT process?
- Why do you consider these rewards functions utilized in the reciprocal CycleGRPO?
- You mention "three annotators assessed 1,162 bilingual utterances" in Section 3.2. How do you choose this subset of 20k samples?
- Do you consider conducting default SFT experiments with your curated dataset of 20k samples on Qwen2.5-Omni and other language-audio models whose inference results you report in the table, for comparison with your proposed joint SFT?
8 comments
W2: Same-source Verification Bias and External Validation
We thank the reviewer for emphasizing the importance of independent verification. Our original submission already includes external validation using Gemini-2.5-Pro (Table 3). To further address potential self-confirmation, we conducted additional independent audio-language evaluation and blinded human assessments, neither of which relies on our trained UND verifier.
(a) Existing External Validation: Gemini-2.5-Pro (Table 3 in the Main Paper).
Our original evaluation employs Gemini-2.5-Pro, independent of our trained UND verifier, to assess instruction–speech consistency under the InstructTTSEval protocol.
| Model | Chinese Instruction Match ↑ | English Instruction Match ↑ |
|---|---|---|
| Joint SFT | 77.50 | 81.97 |
| Full CycleGRPO | 79.00 | 85.53 |
| Improvement | +1.50 pp | +3.56 pp |
Full CycleGRPO improves instruction adherence by +1.50/+3.56 pp (ZH/EN) under Gemini-2.5-Pro evaluation. These results provide external evidence of improved instruction following.
(b) Additional Independent Evaluation: AnyAudio-Judge-30B.
We further evaluated 1,037 bilingual instructions (3,111 generated utterances) using the frozen AnyAudio-Judge-30B (Li et al., EMNLP 2026), a specialized instruction–audio evaluator complementary to Gemini-2.5-Pro. This independent evaluator assesses audible instruction adherence through attribute-level Yes/No probabilities, without using gold profiles or UND predictions. Scores are averaged per instruction and across instructions on a 0–100 scale.
Table R3. Independent instruction-adherence scores (0–100).
| Model | Chinese ↑ | English ↑ | Overall ↑ |
|---|---|---|---|
| Base† | 76.09 | 76.78 | 76.38 |
| Joint SFT | 77.78 | 79.35 | 78.46 |
| w/o Cycle | 77.62 | 78.83 | 78.14 |
| Full CycleGRPO | 78.03 | 79.58 | 78.70 |
| GT Reference | 79.94 | 80.37 | 80.13 |
| Full − w/o Cycle | +0.42 | +0.75 | +0.56 |
Full CycleGRPO achieves the highest independent instruction-adherence score among the generated systems, outperforming both Joint SFT and w/o Cycle across all six language–instruction-category groups. It improves the overall score by +0.56 points over w/o Cycle and +0.24 points over Joint SFT, providing consistent external evidence of improved instruction adherence. The GT score of 80.13 serves as an empirical reference under the same evaluation protocol.
(c) Additional Blinded Human Evaluation.
We conducted 600 blinded, randomized pairwise judgments (301 Chinese; 299 English) with three annotators, covering 180 distinct instructions. Annotators compared Full CycleGRPO against Joint SFT or w/o Cycle, assessed instruction faithfulness through Win/Tie/Loss judgments, and independently rated speech naturalness.
Table R4. Human instruction-following preference and naturalness.
| Comparison | N | Win / Tie / Loss | Decisive Win Rate ↑ | Tie-adjusted Preference (95% CI) ↑ | Naturalness (Full / Baseline) ↑ |
|---|---|---|---|---|---|
| Full vs. Joint SFT | 285 | 110 / 107 / 68 | 61.80% | 57.37% [52.74, 61.94] | 4.52 / 4.48 |
| Full vs. w/o Cycle | 315 | 133 / 91 / 91 | 59.38% | 56.67% [51.19, 62.34] | 4.50 / 4.43 |
| Combined | 600 | 243 / 198 / 159 | 60.45% | 57.00% [53.04, 61.00] | 4.51 / 4.46 |
Decisive Win Rate = Wins / (Wins + Losses), excluding ties. Tie-adjusted Preference = (Wins + 0.5 × Ties) / N.
Blinded human judgments consistently favor Full CycleGRPO over both baselines, achieving 61.80%/59.38% decisive win rates against Joint SFT/w/o Cycle. Moreover, tie-adjusted preferences reach 57.37%/56.67%, with both reported 95% confidence intervals above 50%, while perceived speech naturalness remains comparable. These results provide direct human perceptual evidence of improved instruction controllability beyond internal GEN/UND feedback.
In summary, the original Gemini-2.5-Pro evaluation, additional AnyAudio-Judge-30B results, and blinded human assessments provide convergent evidence that CycleSpeech's improvements extend beyond internal UND-based verification. Together, they support improvements in externally assessed and perceptible instruction controllability. We will incorporate these additional evaluations and protocols into the revised manuscript.
Q1. Joint SFT: decoder freezing, waveform/profile terminology, and GEN–UND connectivity
Thank you for requesting these implementation details. We clarify the training stages and terminology below.
(a) Is the token-to-waveform decoder trainable during the GEN-only warm-up?
No. Both the Step-Audio-2-mini backbone and the token-to-waveform decoder remain frozen throughout Joint SFT, including the initial 2,000 GEN-only steps. Only the task-specific LoRA adapters are updated.
(b) What do “predicted/recovered wavs” and “predicted/recovered profiles” mean in Figure 1?
Figure 1(c) illustrates the full reciprocal framework, not exclusively Joint SFT:
| Term in Figure 1 | Meaning |
|---|---|
| Predicted Wavs | Speech synthesized by GEN from the original instruction, transcript, and speaker reference. |
| Recovered Profiles | Voice profiles inferred by UND from GEN-generated speech. |
| Predicted Profiles | Voice profiles inferred by UND from the original, real speech. |
| Recovered Wavs | Speech reconstructed by GEN from an instruction derived from a predicted profile. |
These terms form two reciprocal paths in Figure 1(c): (i) Predicted Wavs → Recovered Profiles, where UND infers profiles from GEN-generated speech; and (ii) Predicted Profiles → Recovered Wavs, where GEN reconstructs speech conditioned on profiles inferred by UND from the original speech.
(c) How are GEN and UND configured and connected?
Both adapters use a shared frozen Step-Audio-2-mini backbone, but have separate trainable LoRA parameters:
| Configuration | GEN (θ_G) | UND (θ_U) |
|---|---|---|
| LoRA rank | 8 | 8 |
| LoRA scaling (α) | 32 | 16 |
| Target modules | All linear layers | Attention q/k/v/o projections |
| LoRA dropout | 0.05 | 0.05 |
During Joint SFT, GEN receives teacher-forced supervision on ground-truth speech tokens, while UND learns to predict gold profiles from online GEN-generated waveforms. The generated waveforms are detached before entering UND, ensuring no cross-module gradient propagation. Thus, GEN and UND interact through generated speech while maintaining separate trainable parameters and task-specific objectives (Sec. 3.3; Appendix C.1).
We will make these distinctions explicit in the Figure 1 caption and Sec. 3.3.
We sincerely thank the reviewer for the detailed and constructive feedback. We greatly appreciate your recognition of our core contribution in transforming one-way generation–understanding supervision into a bidirectional, verifiable framework, realized through CycleGRPO and a bilingual dataset.
To address your concerns, we provide additional ablations, independent model and human evaluations, standard-SFT comparisons, and methodological clarifications, with detailed responses below.
W1: Potential Circular Validation
CycleSpeech mitigates circular validation through fixed gold-profile supervision, external acoustic feedback, and constrained alternating optimization, rather than relying solely on mutual GEN/UND predictions. We clarify the safeguards and provide supporting ablation evidence below.
(a) Training mechanism: Safeguards against error reinforcement.
Crucially, UND-predicted profiles serve as reconstruction conditions rather than supervision targets in the backward cycle. Although reconstruction quality still depends on GEN, fixed targets and frozen acoustic feedback reduce reliance on self-generated supervision.
(b) Additional ablation: Contribution of backward reconstruction feedback. To assess whether the backward objective benefits from more than gold-profile matching, we remove only the backward style reward while retaining the remaining training components.
Table R1. Effect of the Backward Style Reconstruction Reward (%).
Restoring the backward style reward consistently improves both GEN and UND performance across Chinese and English. Specifically, GEN Style Accuracy increases by +0.79/+0.12 pp (ZH/EN), while UND categorical accuracy improves by +0.24/+0.32 pp and overall score by +0.11/+0.15 pp. These consistent improvements demonstrate that backward style reconstruction provides an effective additional learning signal beyond gold-profile matching, contributing to both generation and understanding performance.
(c) Existing ablations: Benefits of Evolving Bidirectional Feedback (Table 8 in the Main Paper).
Table R2. GEN Style Accuracy under different training strategies (%).
Note: w/o Cycle removes both reciprocal feedback loops; Frozen Feedback retains both loops but uses frozen Joint-SFT counterparts for feedback; w/o UND Updates trains only GEN with UND frozen at Joint-SFT; Full alternately updates both adapters with evolving feedback.
Full CycleGRPO achieves the highest GEN Style Accuracy in both languages. While restricted-feedback variants already outperform w/o Cycle, evolving bidirectional optimization yields further gains. Notably, CycleGRPO also improves UND categorical accuracy over Frozen Feedback. This is consistent with our reciprocal learning mechanism: continually refining UND under fixed gold-profile supervision can provide more reliable feedback for subsequent GEN updates.
In summary, CycleSpeech combines fixed supervision, external acoustic feedback, and constrained optimization rather than relying on unconstrained self-confirmation. The additional ablation further validates the contribution of reconstruction-based feedback.