acceptodds
Under review as a conference paper at ICLR 2027

CycleSpeech: Reciprocal Alignment for Instruction-Controlled Speech Synthesis and Paralinguistic Understanding

Abstract

Instruction-controlled speech synthesis and paralinguistic understanding are often trained independently, leaving reciprocal feedback between the two tasks underexplored. We introduce CycleSpeech, a framework that connects generation and understanding through a shared, structured voice profile that serves as a common target for supervision and reciprocal feedback. The forward cycle assesses whether synthesized speech expresses the intended attributes by comparing recovered and target profiles. The backward cycle evaluates whether profiles inferred from real speech can guide reconstruction of the source speaking style. To support both directions, we construct a bilingual dataset of 20,046 examples pairing instructions, target speech, speaker references, and structured profiles. Building on joint supervised fine-tuning, CycleGRPO alternates policy updates using reciprocal rewards grounded in profile consistency and speaking-style reconstruction. Fixed target profiles anchor feedback from the evolving counterpart. This procedure requires neither human preference annotations nor an additional preference-trained reward model. Evaluations on Chinese and English benchmarks show improved instruction adherence and profile recovery while maintaining competitive synthesis quality. Compared with Step-Audio-2-mini, CycleSpeech improves instruction-match accuracy by 4.50 and 10.06 percentage points in Chinese and English, respectively. Controlled ablations further support the contribution of cycle feedback to generation control. These results support structured voice profiles as an interface for reciprocal training between speech generation and paralinguistic understanding. An online demo is available at https://cyclespeech.github.io.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Speech Generation, Speech Understanding, Reinforcement Learning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_xFsvOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes CycleSpeech, a framework that couples instruction-controlled speech generation with paralinguistic speech understanding through a shared, structured voice profile. Its key contribution is a reciprocal design: a forward cycle checks whether generated speech realizes the intended profile attributes, while a backward cycle reconstructs speech from an inferred profile to test whether the profile preserves speaking style. It introduces CycleGRPO, which alternates policy updates for generation and understanding using reciprocal feedback, without human preference annotations or a separately trained preference reward model. The authors also construct a 20k bilingual instruction–speech–profile dataset supporting both understanding supervision and generation verification. Validation via bilingual automatic metrics, subjective evaluations, out-of-domain tests, and controlled ablations shows improved instruction following, style accuracy, and categorical profile recovery, with cycle feedback and evolving bidirectional updates contributing to the gains.

Critical strengths

Firstly, the paper is well-written with clear structure. Moreover, it transforms one-way generation–understanding supervision into a bidirectional, verifiable loop anchored on a structured voice profile, and operationalizes this idea as a trainable and evaluable system through CycleGRPO and a companion bilingual dataset.

Critical weaknesses

Here are the two main critical weaknesses, stated concisely:

  1. Potential circular validation in form. Although a fixed gold profile and independent content/style rewards partially anchor training, the forward and backward loops still share the same GEN/UND backbone, schema, and annotation distribution; the backward loop conditions GEN on profiles predicted by UND, so GEN may learn to satisfy UND’s systematic biases rather than genuine profile–speech alignment, especially since the profile reward dominates the backward objective.

  2. Same-source verification bias / self-confirmation. Because UND both verifies generated speech and supplies inferred profiles for reconstruction, the loop risks self-confirmation: errors or biases in UND and the profile annotations can be reinforced rather than corrected. The ablations and OOD tests mitigate but do not fully eliminate this concern; stronger external validation, human evaluation, and cross-model/cross-annotator checks would be needed to show the gains reflect real controllability.

Questions for authors
  1. In the Section 3.3 which explains Joint SFT, the token-to-wav decoder is frozen. Is it trainable in the GEN-only warm-up stage? What are the relationship between "predicted wavs" and "recovered wavs", and the relationship between "predicted profile" and "recovered profile", which are mentioned in Figure 1, during the joint SFT process? What are the detailed configurations of θ_G and θ_U and are they connected in the joint SFT process?
  2. Why do you consider these rewards functions utilized in the reciprocal CycleGRPO?
  3. You mention "three annotators assessed 1,162 bilingual utterances" in Section 3.2. How do you choose this subset of 20k samples?
  4. Do you consider conducting default SFT experiments with your curated dataset of 20k samples on Qwen2.5-Omni and other language-audio models whose inference results you report in the table, for comparison with your proposed joint SFT?
8 comments

We sincerely thank the reviewer for the detailed and constructive feedback. We greatly appreciate your recognition of our core contribution in transforming one-way generation–understanding supervision into a bidirectional, verifiable framework, realized through CycleGRPO and a bilingual dataset.

To address your concerns, we provide additional ablations, independent model and human evaluations, standard-SFT comparisons, and methodological clarifications, with detailed responses below.


W1: Potential Circular Validation

CycleSpeech mitigates circular validation through fixed gold-profile supervision, external acoustic feedback, and constrained alternating optimization, rather than relying solely on mutual GEN/UND predictions. We clarify the safeguards and provide supporting ablation evidence below.

(a) Training mechanism: Safeguards against error reinforcement.

  • Fixed gold-profile anchoring: The backward objective assigns a weight of 0.60 to consistency with fixed ground-truth profiles, ensuring that UND is not supervised solely through GEN reconstruction outcomes.
  • External acoustic feedback: A frozen ParaMETA encoder compares reconstructed speech with the original real speech, providing complementary style information beyond direct profile matching.
  • Constrained alternating optimization: GEN remains inference-only during backward optimization; only UND is updated. During forward optimization, UND provides feedback while GEN is updated. Separate LoRA adapters, a frozen pretrained backbone, and KL regularization against fixed Joint-SFT policies further constrain optimization.

Crucially, UND-predicted profiles serve as reconstruction conditions rather than supervision targets in the backward cycle. Although reconstruction quality still depends on GEN, fixed targets and frozen acoustic feedback reduce reliance on self-generated supervision.

(b) Additional ablation: Contribution of backward reconstruction feedback. To assess whether the backward objective benefits from more than gold-profile matching, we remove only the backward style reward while retaining the remaining training components.

Table R1. Effect of the Backward Style Reconstruction Reward (%).

LanguageVariantGEN Style Acc. ↑UND Cat.Avg ↑UND OS ↑
Chinesew/o Backward Style59.6373.9478.25
ChineseFull CycleGRPO60.4274.1878.36
Englishw/o Backward Style60.6670.1876.69
EnglishFull CycleGRPO60.7870.5076.84

Restoring the backward style reward consistently improves both GEN and UND performance across Chinese and English. Specifically, GEN Style Accuracy increases by +0.79/+0.12 pp (ZH/EN), while UND categorical accuracy improves by +0.24/+0.32 pp and overall score by +0.11/+0.15 pp. These consistent improvements demonstrate that backward style reconstruction provides an effective additional learning signal beyond gold-profile matching, contributing to both generation and understanding performance.

(c) Existing ablations: Benefits of Evolving Bidirectional Feedback (Table 8 in the Main Paper).

Table R2. GEN Style Accuracy under different training strategies (%).

VariantChinese ↑English ↑
w/o Cycle58.9258.96
w/ Frozen Feedback59.6660.68
w/o UND Updates59.6959.53
Full CycleGRPO60.4260.78

Note: w/o Cycle removes both reciprocal feedback loops; Frozen Feedback retains both loops but uses frozen Joint-SFT counterparts for feedback; w/o UND Updates trains only GEN with UND frozen at Joint-SFT; Full alternately updates both adapters with evolving feedback.

Full CycleGRPO achieves the highest GEN Style Accuracy in both languages. While restricted-feedback variants already outperform w/o Cycle, evolving bidirectional optimization yields further gains. Notably, CycleGRPO also improves UND categorical accuracy over Frozen Feedback. This is consistent with our reciprocal learning mechanism: continually refining UND under fixed gold-profile supervision can provide more reliable feedback for subsequent GEN updates.

In summary, CycleSpeech combines fixed supervision, external acoustic feedback, and constrained optimization rather than relying on unconstrained self-confirmation. The additional ablation further validates the contribution of reconstruction-based feedback.

W2: Same-source Verification Bias and External Validation

We thank the reviewer for emphasizing the importance of independent verification. Our original submission already includes external validation using Gemini-2.5-Pro (Table 3). To further address potential self-confirmation, we conducted additional independent audio-language evaluation and blinded human assessments, neither of which relies on our trained UND verifier.

(a) Existing External Validation: Gemini-2.5-Pro (Table 3 in the Main Paper).

Our original evaluation employs Gemini-2.5-Pro, independent of our trained UND verifier, to assess instruction–speech consistency under the InstructTTSEval protocol.

ModelChinese Instruction Match ↑English Instruction Match ↑
Joint SFT77.5081.97
Full CycleGRPO79.0085.53
Improvement+1.50 pp+3.56 pp

Full CycleGRPO improves instruction adherence by +1.50/+3.56 pp (ZH/EN) under Gemini-2.5-Pro evaluation. These results provide external evidence of improved instruction following.

(b) Additional Independent Evaluation: AnyAudio-Judge-30B.

We further evaluated 1,037 bilingual instructions (3,111 generated utterances) using the frozen AnyAudio-Judge-30B (Li et al., EMNLP 2026), a specialized instruction–audio evaluator complementary to Gemini-2.5-Pro. This independent evaluator assesses audible instruction adherence through attribute-level Yes/No probabilities, without using gold profiles or UND predictions. Scores are averaged per instruction and across instructions on a 0–100 scale.

Table R3. Independent instruction-adherence scores (0–100).

ModelChinese ↑English ↑Overall ↑
Base†76.0976.7876.38
Joint SFT77.7879.3578.46
w/o Cycle77.6278.8378.14
Full CycleGRPO78.0379.5878.70
GT Reference79.9480.3780.13
Full − w/o Cycle+0.42+0.75+0.56

Full CycleGRPO achieves the highest independent instruction-adherence score among the generated systems, outperforming both Joint SFT and w/o Cycle across all six language–instruction-category groups. It improves the overall score by +0.56 points over w/o Cycle and +0.24 points over Joint SFT, providing consistent external evidence of improved instruction adherence. The GT score of 80.13 serves as an empirical reference under the same evaluation protocol.

(c) Additional Blinded Human Evaluation.

We conducted 600 blinded, randomized pairwise judgments (301 Chinese; 299 English) with three annotators, covering 180 distinct instructions. Annotators compared Full CycleGRPO against Joint SFT or w/o Cycle, assessed instruction faithfulness through Win/Tie/Loss judgments, and independently rated speech naturalness.

Table R4. Human instruction-following preference and naturalness.

ComparisonNWin / Tie / LossDecisive Win Rate ↑Tie-adjusted Preference (95% CI) ↑Naturalness (Full / Baseline) ↑
Full vs. Joint SFT285110 / 107 / 6861.80%57.37% [52.74, 61.94]4.52 / 4.48
Full vs. w/o Cycle315133 / 91 / 9159.38%56.67% [51.19, 62.34]4.50 / 4.43
Combined600243 / 198 / 15960.45%57.00% [53.04, 61.00]4.51 / 4.46

Decisive Win Rate = Wins / (Wins + Losses), excluding ties. Tie-adjusted Preference = (Wins + 0.5 × Ties) / N.

Blinded human judgments consistently favor Full CycleGRPO over both baselines, achieving 61.80%/59.38% decisive win rates against Joint SFT/w/o Cycle. Moreover, tie-adjusted preferences reach 57.37%/56.67%, with both reported 95% confidence intervals above 50%, while perceived speech naturalness remains comparable. These results provide direct human perceptual evidence of improved instruction controllability beyond internal GEN/UND feedback.

In summary, the original Gemini-2.5-Pro evaluation, additional AnyAudio-Judge-30B results, and blinded human assessments provide convergent evidence that CycleSpeech's improvements extend beyond internal UND-based verification. Together, they support improvements in externally assessed and perceptible instruction controllability. We will incorporate these additional evaluations and protocols into the revised manuscript.

Q1. Joint SFT: decoder freezing, waveform/profile terminology, and GEN–UND connectivity

Thank you for requesting these implementation details. We clarify the training stages and terminology below.

(a) Is the token-to-waveform decoder trainable during the GEN-only warm-up?

No. Both the Step-Audio-2-mini backbone and the token-to-waveform decoder remain frozen throughout Joint SFT, including the initial 2,000 GEN-only steps. Only the task-specific LoRA adapters are updated.

(b) What do “predicted/recovered wavs” and “predicted/recovered profiles” mean in Figure 1?

Figure 1(c) illustrates the full reciprocal framework, not exclusively Joint SFT:

Term in Figure 1Meaning
Predicted WavsSpeech synthesized by GEN from the original instruction, transcript, and speaker reference.
Recovered ProfilesVoice profiles inferred by UND from GEN-generated speech.
Predicted ProfilesVoice profiles inferred by UND from the original, real speech.
Recovered WavsSpeech reconstructed by GEN from an instruction derived from a predicted profile.

These terms form two reciprocal paths in Figure 1(c): (i) Predicted Wavs → Recovered Profiles, where UND infers profiles from GEN-generated speech; and (ii) Predicted Profiles → Recovered Wavs, where GEN reconstructs speech conditioned on profiles inferred by UND from the original speech.

(c) How are GEN and UND configured and connected?

Both adapters use a shared frozen Step-Audio-2-mini backbone, but have separate trainable LoRA parameters:

ConfigurationGEN (θ_G)UND (θ_U)
LoRA rank88
LoRA scaling (α)3216
Target modulesAll linear layersAttention q/k/v/o projections
LoRA dropout0.050.05

During Joint SFT, GEN receives teacher-forced supervision on ground-truth speech tokens, while UND learns to predict gold profiles from online GEN-generated waveforms. The generated waveforms are detached before entering UND, ensuring no cross-module gradient propagation. Thus, GEN and UND interact through generated speech while maintaining separate trainable parameters and task-specific objectives (Sec. 3.3; Appendix C.1).

We will make these distinctions explicit in the Figure 1 caption and Sec. 3.3.

Reviewer_BJ41OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper connects instruction-controlled speech synthesis and paralinguistic understanding through a canonical voice profile. Two LoRA adapters share a frozen Step-Audio-2-mini backbone. Joint SFT trains generation and profile recovery, followed by CycleGRPO: the understanding adapter assesses generated speech against a target profile, while the generation adapter reconstructs speech from an inferred profile to provide style feedback. Fixed profile targets and auxiliary content/quality rewards anchor the alternating updates. The work also constructs 20,046 bilingual training examples. Evaluation includes bilingual generation and understanding, out-of-domain generation, listening tests, and controls for removing cycles, freezing feedback, and updating only one direction.

Critical strengths
  1. The structured interface is a useful extension of coupled speech understanding and generation. Speech Chain learning already couples recognition and synthesis through reconstruction, and UniStyle combines style captioning with controllable synthesis. CycleSpeech contributes an explicit ten-field paralinguistic interface and alternating reward-based updates of two adapters. Fixed profile targets and external content, quality, and style signals provide meaningful anchors. The contribution is therefore a practical structured extension of coupled systems, rather than the coupling principle itself.

  2. The complete system has useful bilingual and cross-domain evidence. The 20,046-example resource, generation and understanding evaluations, out-of-domain tests, listening study, and no-cycle/frozen-feedback/single-direction controls support practical utility. Generation InstructionMatch improves from SFT to RL by 1.50 points in Chinese and 3.56 in English (Table 3). The no-cycle comparison also supports a generation-side style benefit. These gains matter to my recommendation, even though the additional improvement in understanding is much smaller than the gain from SFT.

Critical weaknesses
  1. The forward reward conflates adherence to a partial instruction with agreement with the complete source profile. Appendix B.4 constructs instructions from selected fields: the APS template omits emotion and scene, while RP uses a scene description. Nevertheless, Equation 5 and Appendix C.3 give all ten source-profile fields equal weight. An output can therefore satisfy the specified constraints yet receive a lower reward because unspecified attributes differ from the original recording. A speaker reference can supply identity information, and scene descriptions can imply some attributes, so the issue is not simply whether each field is explicitly named. The missing distinction is which values are actually determined by the conditioning and which admit legitimate variation. PromptTTS 2 explicitly models voice variation not specified by the prompt. The paper should distinguish reproducing a particular source realization from satisfying a conditional instruction; otherwise, the reward's interpretation as instruction adherence is too broad.

  2. The backward cycle can validate only information transmitted through the bridge, while the distinctive understanding-side gain remains weakly established. Appendix C.3 describes category-dependent bridging. If the implementation transmits only the selected fields illustrated by the templates in Appendix B.4, two profiles differing only in discarded fields give the same generator input, with transcript and speaker reference fixed. Reconstruction cannot distinguish correctness in those fields, and the reference/transcript independently supply identity/content. Direct profile supervision, comprising 60% of the UND reward, remains a genuine anchor; this is not evidence that all supervision fails. The empirical attribution also needs care: UND overall improves from 77.98/76.54 after SFT to 78.36/76.84 after RL (Table 4), and the no-cycle-to-full comparison is only 78.35/76.69 to 78.36/76.84 (Table 8). Generation StyleAcc does improve from 58.92/58.96 without cycles to 60.42/60.78 with the full model. The existing controls are relevant, but do not by themselves establish that reciprocal updating verifies the complete inferred profile or supplies a substantial additional understanding benefit beyond direct supervision.

Questions for authors
  1. Clarification of the conditional task: Is the profile reward masked or otherwise conditioned on attributes specified or implied by the instruction and reference? If all ten fields are intentionally scored, which conditioning information justifies each target, and is the task instruction compliance or source-realization reproduction?

  2. Clarification of the implemented bridge: Which fields are transmitted for each category, and does the implementation include information beyond the templates in Appendix B.4? Please distinguish fields checked by direct profile supervision from those receiving a discriminative reconstruction signal.

  3. Interpretation of existing evidence: Given the small no-cycle/full difference in UND overall, what do the frozen-feedback and single-direction controls establish about the additional benefit of reciprocal updating? Please identify the effect isolated by the existing controls and the uncertainty of that effect. This is a request to interpret the present evidence, not a request for a new broad benchmark.

4 comments

We sincerely appreciate your thoughtful and constructive review. We are particularly encouraged by your recognition of CycleSpeech's structured paralinguistic interface, reciprocal optimization framework, and bilingual and cross-domain evaluations. We also greatly value your careful distinction between our contributions and prior coupled speech systems, as well as your insightful observations on reward design and backward reconstruction.

In response, we provide additional targeted ablations and methodological clarifications below.


W1 & Q1. Conditional Instruction Adherence vs. Full-Profile Consistency

We agree that satisfying a partially specified instruction is not equivalent to reproducing the complete source profile, as unspecified attributes may admit multiple valid realizations. We clarify the conditioning semantics, reward design, and empirical evidence below.

(a) Clarification of Reward Design.

The original forward profile reward is not masked: Eq. 5 equally scores all ten source-profile fields. However, different conditioning inputs constrain different aspects of speech, while some attributes admit legitimate variation.

Table R1. Conditioning information and potentially underdetermined attributes (Appendix B.4).

Instruction TypeSpecified AttributesSpeaker Reference (Possible Cues)Potentially Underdetermined Attributes
APSGender, age, texture, speed, volume, accentGender, age, texture, accentEmotion, emotion detail, personality, scene
DSDSpeed, texture, volume, emotion detail, personalityGender, age, texture, accentEmotion, scene
RPSceneGender, age, texture, accentExact emotion, emotion detail, personality, speed, volume (may be implied by the scene description)

Speaker references provide speaker-level cues, while scene and style descriptions may implicitly constrain additional attributes. These inputs do not necessarily determine unique attribute values.

Why full-profile supervision? Each training example pairs an instruction, transcript, and speaker reference with a real utterance and its annotated voice profile. Although the complete profile is not uniquely determined by the conditioning inputs, it represents an annotated realization of the source utterance. Supervising all ten attributes provides richer, source-grounded paralinguistic feedback, encouraging GEN to learn vocal characteristics beyond explicitly specified constraints.

Accordingly, Eq. 5 measures source-profile consistency rather than strict instruction-only adherence. At inference, GEN remains conditioned on the instruction, transcript, and speaker reference; the gold profile is used only for training supervision. While full-profile supervision may penalize alternative valid realizations, we empirically examine this trade-off below.

(b) Additional Forward-Masked Reward Ablation.

To assess the effect of full-profile supervision, we conducted an additional ablation retaining only fields explicitly selected by the APS/DSD/RP instruction templates in the forward profile reward, while keeping the backward objective and other training settings unchanged.

Table R2. Effect of Forward Profile Reward Masking.

MetricZH FullZH MaskedEN FullEN Masked
GEN Style Acc. (%) ↑60.4260.2960.7860.92
CER / WER (%) ↓2.332.343.083.08
AnyAudio-Judge ↑78.0377.9479.5879.13
UND OS (%) ↑78.3678.3376.8476.71

Explicit-field masking does not improve independently evaluated instruction adherence in our experiments. Full CycleGRPO obtains slightly higher AnyAudio-Judge scores in Chinese (+0.09) and English (+0.45) while maintaining comparable generation style accuracy and content fidelity.

These results suggest that full-profile consistency can provide useful additional source-grounded supervision, although the small differences do not establish a statistically significant advantage and the reward is not equivalent to instruction-only adherence.

In summary, CycleSpeech distinguishes source-profile consistency during training from conditional instruction adherence at inference and evaluation. We will clarify this distinction and the conditioning assumptions in Sec. 3.4.1 and Appendix C.3.

W2 & Q2. Scope and Contribution of Backward Reconstruction Feedback

We thank the reviewer for this important observation. We agree that backward reconstruction can only provide feedback on information transmitted through the Profile-to-Instruction Bridge, rather than independently verifying the complete inferred profile.

(a) Clarification of the Backward Bridge and Verification Scope.

The backward bridge follows the category-dependent templates in Appendix B.4 (Table S4). For DSD, all 20 template variants use the same profile fields and differ only in wording.

Table R3. Profile fields transmitted through the backward bridge.

Instruction TypeTransmitted Profile Fields
APSGender, Age, Texture, Speed, Volume, Accent
DSDSpeed, Texture, Volume, Emotion Detail, Personality
RPScene

No additional profile fields are introduced beyond these templates; the transcript and speaker reference are supplied independently.

Importantly, direct profile matching (RprofR_{\mathrm{prof}}, 60%) supervises all ten fields, whereas style reconstruction (RstyR_{\mathrm{sty}}, 20%) provides complementary acoustic feedback on information conveyed through the bridge. Attributes omitted from the bridge cannot be independently verified by reconstruction.

(b) Additional Backward Style Reward Ablation.

To examine whether reconstruction provides useful supervision beyond direct profile matching, we additionally removed only the backward style reward RstyR_{\mathrm{sty}}, retaining the gold-profile reward and other training settings.

Table R4. Effect of Backward Style Reconstruction Reward (%).

LanguageVariantGEN Style Acc. ↑UND Cat.Avg ↑UND Desc.Sim ↑UND OS ↑
Chinesew/o Backward Style59.6373.9472.3178.25
ChineseFull CycleGRPO60.4274.1872.3078.36
Englishw/o Backward Style61.0870.1471.7876.62
EnglishFull CycleGRPO60.7870.5071.9476.84

Key findings. With direct profile supervision retained, the backward style reward improves UND categorical accuracy by +0.24/+0.36 pp (ZH/EN) and overall score by +0.11/+0.22 pp, respectively. These consistent but modest UND improvements suggest that backward style reconstruction provides complementary learning signals beyond direct profile matching.

Q3. Contribution of Evolving Feedback and Bidirectional Optimization

We appreciate the reviewer's careful assessment of our ablation results. Our controlled experiments examine three key components of CycleGRPO: cycle-based feedback, evolving counterpart feedback, and bidirectional optimization. We clarify the factors examined by each ablation and their empirical contributions below.

(a) Design Motivation and Isolated Factors.

Table R5. Design motivations of the controlled ablations.

AblationModification Relative to Full CycleGRPOFactor Examined
w/o CycleRemoves both cycle-feedback pathways while retaining alternating GRPO updates with non-cycle rewardsContribution of cycle-based feedback
w/ Frozen FBForward: Update GEN using frozen Joint-SFT UND feedback; Backward: Update UND using frozen Joint-SFT GEN feedbackContribution of evolving reciprocal feedback
w/o UND UpdatesUpdates GEN only, keeping UND frozen at Joint SFTBenefit of updating UND for GEN optimization
w/o GEN UpdatesUpdates UND only, keeping GEN frozen at Joint SFTBenefit of updating GEN for UND optimization
Full CycleGRPOAlternately updates both adapters using their latest counterpartsEvolving bidirectional optimization

These ablations examine complementary aspects of reciprocal optimization: w/o Cycle removes both cycle-feedback pathways, leaving GEN optimized with equally weighted content and quality rewards and UND with direct profile matching only. Frozen Feedback retains bidirectional updates but uses frozen Joint-SFT counterparts to isolate the contribution of evolving feedback, whereas single-direction controls examine whether updating the opposite adapter provides additional benefits beyond optimizing the active adapter alone.

(b) Quantitative Evidence.

Table R6. Effects of evolving feedback and bidirectional optimization (Table 8; %).

VariantZH GEN Style ↑EN GEN Style ↑ZH UND OS ↑EN UND OS ↑
w/o Cycle58.9258.9678.3576.69
w/ Frozen FB59.6660.6878.3476.72
w/o UND Updates59.6959.5377.9876.54
w/o GEN Updates60.2060.3578.2576.81
Full CycleGRPO60.4260.7878.3676.84

Key findings:

  • Cycle-based feedback: Compared with w/o Cycle, Full CycleGRPO improves GEN Style Accuracy by +1.50/+1.82 pp (ZH/EN), while UND OS gains remain modest (+0.01/+0.15 pp).
  • Evolving feedback: Compared with Frozen FB, Full improves GEN Style Accuracy by +0.76/+0.10 pp, supporting the potential benefit of using updated counterparts rather than fixed feedback models.
  • Bidirectional optimization: Compared with w/o UND Updates, Full improves GEN Style Accuracy by +0.73/+1.25 pp. Conversely, compared with w/o GEN Updates, UND OS improves by +0.11/+0.03 pp, indicating a smaller understanding-side benefit from evolving reconstruction feedback.

Statistical uncertainty. Paired 95% bootstrap confidence intervals support the GEN Style Accuracy improvements over w/o Cycle in both languages (ZH: +1.50 pp [0.04, 2.91]; EN: +1.82 pp [0.09, 3.55]). Full also achieves positive gains over Frozen Feedback and single-direction controls on their corresponding metrics, although these incremental effects remain statistically inconclusive.

Overall, these results support the effectiveness of CycleGRPO in generation-side attribute control, while suggesting additional benefits from evolving feedback and bidirectional optimization.

Dear Reviewer BJ41:

Thank you again for your positive assessment and constructive feedback on our work. We have carefully addressed your comments through additional experiments and methodological clarifications in our rebuttal and revised manuscript.

As the discussion period draws to a close, we would be very grateful if you have any further thoughts or feedback to share. We sincerely appreciate your time and consideration.

Reviewer_dSW5OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

CycleSpeech is a framework that connects instruction-controlled speech synthesis and paralinguistic speech understanding through a shared, structured voice profile containing ten attributes covering speaker characteristics, prosody, affect, and context. The system uses two LoRA adapters on a frozen Step-Audio-2-mini backbone, one for generation and the other for understanding. Following joint supervised fine-tuning, the proposed CycleGRPO algorithm alternately optimizes the two adapters through reciprocal feedback. In the forward cycle, the understanding adapter extracts profiles from generated speech to assess consistency with fixed target profiles. In the backward cycle, inferred profiles are converted into instructions for speech reconstruction, providing feedback on preservation of the source speaking style. Profile supervision and auxiliary content, quality, and style rewards anchor training, without requiring human preference annotations or a separately trained preference reward model. The authors also construct a bilingual dataset of 20,046 aligned instruction–speech–profile examples. Evaluations include Chinese and English generation and understanding tasks, out-of-domain generation, subjective listening tests, and controlled ablations. The results show improved instruction following and profile recovery over the base model, with most understanding gains arising from supervised fine-tuning and additional cycle-training benefits more evident in generation control.

Critical strengths
  1. The structured voice profile provides an explicit shared interface between speech generation and understanding. Its ten fields cover speaker characteristics, prosody, affect, and context, allowing the same representation to support understanding supervision, generation verification, and speaking-style reconstruction. This offers a practical, inspectable extension of existing generation–understanding coupling approaches while preserving natural-language instructions as the generation interface.

  2. CycleGRPO provides a concrete implementation of reciprocal training. The method translates profile consistency and speaking-style reconstruction into feedback for alternating optimization. Fixed target profiles and external content, quality, and style rewards provide training anchors, without requiring human preference annotations or a separately trained preference reward model.

  3. The accompanying dataset and diverse evaluations support practical utility, particularly for generation control. The authors construct 20,046 bilingual examples and evaluate the system through Chinese and English benchmarks, out-of-domain tests, subjective listening assessments, and controlled ablations. Compared with Joint SFT, CycleGRPO improves instruction-match accuracy by 1.50 percentage points in Chinese and 3.56 in English. Comparisons with the no-cycle variant also show improved generation style accuracy, supporting the usefulness of cycle feedback.

  4. The paper is clearly written and well organized. The presentation connects the shared profile representation, dataset construction, joint supervised fine-tuning, and reciprocal reinforcement learning in an accessible manner, making the overall design and its evaluation easy to follow.

Critical weaknesses
  1. Potential shared biases in reciprocal feedback. The generation and understanding modules share a backbone and are trained using the same profile annotation framework. Improved agreement between them may therefore partly reflect adaptation to shared biases rather than genuine alignment between speech and profile attributes. Although fixed target profiles and external auxiliary rewards provide some constraints, the paper does not sufficiently explain how reciprocal feedback avoids reinforcing existing errors in the understanding module.
Questions for authors
  1. How does the framework mitigate self-confirmation? Could the authors clarify how the proposed reciprocal feedback mechanism mitigates shared biases or self-confirmation, and what evidence independent of this feedback loop supports improvements in actual speech controllability?
2 comments

We sincerely thank the reviewer for your careful and insightful assessment of CycleSpeech. We particularly appreciate your recognition of the structured voice profile as an explicit, inspectable interface and CycleGRPO as a practical framework for reciprocal optimization. We also value your thoughtful assessment of the generation-side improvements and your important concern regarding shared biases.

In response, we clarify how CycleGRPO mitigates self-confirmation and provide additional independent evaluations of actual speech controllability.


W1 & Q1. Mitigating Self-Confirmation and Independent Evidence of Speech Controllability

CycleGRPO is designed to optimize speech–profile alignment rather than unconstrained agreement between GEN and UND. We clarify the training safeguards and provide independent empirical evidence below.

(a) Mechanisms for Mitigating Self-Confirmation.

CycleGRPO incorporates three complementary safeguards:

  1. Fixed source-profile supervision. Both forward and backward profile rewards are anchored to fixed voice profiles annotated from real speech, rather than predictions generated by the evolving counterpart. This prevents the supervision targets from drifting through mutual agreement.

  2. External acoustic constraints. Frozen ASR models, UTMOS, and ParaMETA provide complementary content fidelity, acoustic quality, and speaking-style reconstruction rewards, respectively. These signals are not derived from GEN–UND profile agreement.

  3. Constrained reciprocal optimization. GEN and UND use separate LoRA adapters over a frozen backbone, with only the active adapter updated in each alternating step. KL regularization against frozen Joint-SFT reference policies further constrains policy drift.

Together, these mechanisms anchor reciprocal training to real-speech supervision and external acoustic evidence, mitigating the risk of reinforcing errors from the evolving understanding model.

(b) Independent Evaluation of Actual Speech Controllability.

To determine whether improvements extend beyond internal GEN–UND agreement, we evaluate generated speech using independent audio judges and blinded human assessments, neither of which participates in reciprocal training.

Independent audio evaluation. The original Gemini-2.5-Pro evaluation (Table 3) shows instruction-match improvements of +1.50/+3.56 pp (ZH/EN) over Joint SFT.

We additionally evaluate 1,037 bilingual instructions using AnyAudio-Judge-30B, which assesses audible instruction adherence independently of the trained UND adapter.

Table R1. Independent instruction-adherence scores (0–100).

ModelChinese ↑English ↑Overall ↑
Base76.0976.7876.38
Joint SFT77.7879.3578.46
w/o Cycle77.6278.8378.14
Full CycleGRPO78.0379.5878.70
GT Reference79.9480.3780.13
Full − w/o Cycle+0.42+0.75+0.56

Full CycleGRPO achieves the highest instruction-adherence scores among the evaluated models in both languages, improving over w/o Cycle by +0.42/+0.75 points (ZH/EN). These gains are measured by an external audio evaluator rather than the trained UND module.

Blinded human evaluation. We further conducted 600 blinded pairwise judgments, with randomized presentation order and concealed system identities.

Table R2. Independent human evaluation of instruction faithfulness.

ComparisonJudgmentsFull Win / Tie / LossFull Preference (%)
Full vs. Joint SFT285110 / 107 / 6857.37
Full vs. w/o Cycle315133 / 91 / 9156.67

Full Preference = (Wins + 0.5 × Ties) / Total Judgments.

Human listeners also favor Full CycleGRPO over both baselines in instruction faithfulness, providing direct perceptual evidence beyond model-based agreement.

Overall, the combination of fixed source supervision, external acoustic constraints, independent audio evaluation, and blinded human preferences supports genuine improvements in audible speech controllability, rather than improvements measured solely by the reciprocal feedback loop.

Dear Reviewer dSW5:

Thank you again for your positive assessment and constructive feedback on our work. We have carefully addressed your comments through additional experiments in our rebuttal and revised manuscript.

As the discussion period draws to a close, we would be very grateful if you have any further thoughts or feedback to share. We sincerely appreciate your time and consideration.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

Dear Area Chair,

We sincerely thank you and all reviewers for their constructive feedback and recognition of CycleSpeech's contributions. In response, we conducted additional experiments and refined the manuscript to address the key concerns.

For clarity, we refer to Reviewers dSW5, BJ41, and xFsv as R1, R2, and R3, respectively, throughout this response.

1. Core Contributions

CycleSpeech advances instruction-controlled speech synthesis and paralinguistic understanding beyond one-way supervision and independently trained generation–understanding systems. It introduces a unified reciprocal learning framework built upon a shared ten-field voice profile, which serves as a structured, verifiable interface between generation and understanding. Through CycleGRPO, the two tasks are alternately optimized with evolving bidirectional feedback, anchored by fixed source annotations and complementary acoustic rewards. Together with a 20,046-example bilingual dataset, CycleSpeech provides an integrated approach to jointly advancing instruction-controlled speech generation and fine-grained paralinguistic understanding. Comprehensive evaluations across Chinese and English, including generation, paralinguistic understanding, out-of-domain generalization, subjective listening assessments, and controlled ablations, further demonstrate its effectiveness and practical utility.

2. Additional Experiments During Rebuttal

  • Independent evaluation and human assessment (Appendix F.1–F.2, Tables S7–S8; R1, R3): Added AnyAudio-Judge-30B evaluation on 1,037 bilingual instructions (3,111 generated utterances) and 600 blinded pairwise human judgments by three annotators, providing independent evidence of improved speech controllability.
  • Forward and backward reward ablations (Appendix G.1, Tables S9–S10; R2, R3): Examined explicit-field masking in the forward profile reward and the contribution of backward style reconstruction beyond direct profile supervision.
  • Cross-backbone Standard SFT (Appendix G.2, Table S11; R3): Fine-tuned Qwen2.5-Omni on the same 20,046 examples for two epochs, validating the effectiveness of our curated supervision beyond the Step-Audio-2-mini backbone.

3. Methodological Clarifications and Further Analysis

  • Reward design and verification scope (Sections 3.4.1–3.4.3; Appendix C.3, Table S6; R1–R3): Clarified the distinction between source-profile consistency and instruction adherence, the conditioning information provided by instructions and speaker references, and the exact fields transmitted through the backward bridge. We also explained the safeguards against potential self-confirmation.
  • Reciprocal-feedback ablations (Section 5.4, Table 8; R2): Clarified the factors isolated by no-cycle, frozen-feedback, and single-direction controls, including their retained rewards and distinct generation- and understanding-side effects.
  • Joint SFT and GEN–UND connectivity (Figure 1(c), Section 3.3, Appendix C.1; R3): Clarified the relationships between predicted/recovered waveforms and profiles, decoder freezing, adapter configurations, training schedules, and gradient flow.
  • Reward functions and annotation strategy (Sections 3.2 and 3.4; Table 2; R3): Explained the motivation for each reward component and the language- and attribute-balanced sampling strategy for human annotation.

We sincerely thank all reviewers again for their time and constructive engagement. We believe these targeted experiments and clarifications substantially strengthen CycleSpeech and address the concerns raised.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.