acceptodds
Under review as a conference paper at ICLR 2027

CuteTTS: Efficient and High-Quality Speech Synthesis via Autoregressive Modeling of Continuous Latents

Abstract

Zero-shot text-to-speech (TTS) now supports interactive assistants, personalized media, and accessibility tools. All TTS systems require faithful linguistic rendering, consistent speaker identity, and low-latency response. Yet compact streaming systems must preserve sufficient acoustic detail in a predictable low-rate latent sequence, while iterative diffusion sampling and classifier-free guidance multiply inference cost at every autoregressive step. To strike a balance between high-fidelity synthesis and low-latency inference, we present CuteTTS, a compact continuous-autoregressive TTS system. It combines semantically aligned causal VAE latents with patch-level autoregression, explicit speaker conditioning, and a bidirectional flow-matching head. We further introduce guidance-step distillation, which absorbs classifier-free guidance and multiple solver steps into a single interval-conditioned student. Evaluations on LibriSpeech and Seed-TTS-Eval demonstrate competitive intelligibility and speaker similarity in zero-shot voice cloning, while distillation lowers first-audio latency by 23.3% and real-time factor by 40.8% relative to the base model with comparable objective and subjective quality. These results provide a practical path toward continuous-autoregressive TTS that reconciles high-fidelity generation with the latency demands of real-time interaction.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
text-to-speech, speech synthesis, zero-shot voice cloning, continuous autoregressive modeling, flow matching, classifier-free guidance, knowledge distillation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 4 reviews, above 27% of the others.
Reviewer_6Sf8OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

CuteTTS is a compact streaming zero-shot TTS system combining a causal, semantically aligned continuous speech VAE, explicit speaker conditioning, patch-level autoregression, and a bidirectional flow-matching head. Speech is represented by 64-dimensional latents at 12.5 Hz and grouped into two-frame patches, giving 6.25 autoregressive decisions per second. A guidance-step distilled head learns the interval-averaged velocity of a classifier-free-guided teacher, jointly reducing solver steps and removing the unconditional branch. On LibriSpeech and Seed-TTS evaluation sets, the system reports competitive intelligibility and speaker similarity. A paired RTX 4090 experiment reduces mean first-audio latency from 49.0 to 37.6 ms and RTF from 0.184 to 0.109. The paper also studies representation rate, patch size, semantic alignment, speaker conditioning, and module capacity allocation.

Critical strengths
  1. A practical, measured quality/latency trade-off. The distilled system reduces diffusion-head evaluations from ten steps on two branches to four on one branch and reports a consistent end-to-end latency improvement, not only an NFE reduction. Table 2 includes mean and tail latency, and Appendix D times raw-reference loading, encoding, decoding, and host transfer without reusing request-specific cached features. This is a meaningful contribution for responsive voice cloning.
  2. Useful analysis of representation granularity and conditioning. The paper distinguishes VAE frame rate from autoregressive token rate and studies both fixed-patch and fixed-token-rate settings. Semantic alignment improves TTS intelligibility without materially changing reconstruction, while explicit speaker conditioning improves similarity under both WavLM and WeSpeaker in Table 4. These analyses explain design choices rather than presenting only an end-system leaderboard.
Critical weaknesses
  1. Limited novelty: the system is mainly a combination of existing methods, and the paper does not make clear what is new.** Every component maps directly to prior work. The causal VAE follows DAC and VoxCPM; the sigma-VAE comes from LatentLM and VibeVoice; the semantic alignment is the Semantic-VAE recipe (WavLM features, projection, cosine loss), as the paper itself states. The generator "follows the patch-based formulation of DiTAR" (Sec. 2.3): aggregation encoder with a summary token, causal LM, bidirectional local diffusion head with the previous patch as prefix, stop head, and LM-level CFG with 0.1 dropout. VoxCPM/VoxCPM2 use the same LocEnc-LM-LocDiT layout over a causal VAE, and VoxCPM2 also runs at 6.25 Hz token rate. Explicit speaker embeddings in AR TTS are standard (CosyVoice, MiniMax-Speech, Qwen3-TTS), and the encoder here is a distilled ECAPA. The only claimed method contribution, guidance-step distillation, is very close to existing work. Eq. 10 is IntMeanFlow's integral-velocity distillation (student regresses the teacher's average velocity over [t, t+dt]) with a CFG-guided teacher. Folding CFG into a one-stage few-step distillation with the guidance scale as an input is the core of Latent Consistency Models (augmented PF-ODE, w-embedding, skipping steps). MeanFlow already puts CFG into the average-velocity target for 1-NFE sampling, and CALM/Pocket TTS already distills CFG away for continuous AR TTS. The paper does not cite LCM, has no Related Work section, and never compares against any of these alternatives. In particular, the claim that guidance removal and step reduction are "learned jointly rather than as two sequential distillation stages" is not tested: there is no sequential baseline, no IntMeanFlow or LCM-style baseline, and no guidance-only or step-only ablation under the same budget. For ICLR, a well-engineered combination can be acceptable if it yields new understanding or clearly superior results. Here, the understanding is mostly confirmatory and the results are not clearly superior (see W2).

  2. The evidence does not support the "high-quality", "competitive", and "compact" claims.(a) Speaker similarity, the main advantage, is suspicious. CuteTTS reports SIM 78.9 on LibriSpeech and 76.5 on Seed-EN, above every baseline and above commonly reported ground-truth levels for these protocols (about 0.69 for LibriSpeech-PC under the F5-TTS protocol, about 0.73 for Seed-EN human speech). Yet no ground-truth row is given. The speaker encoder is distilled from a WavLM-Large SV model and the SIM evaluator is a fine-tuned WavLM-Large. Table 4 shows the risk: removing speaker conditioning drops WavLM SIM by 17.9 points but WeSpeaker SIM by only 8.0. Table 1 reports no independent SIM. (b) WER is not competitive: 2.04 on Seed-EN is worse than 9 of 13 baselines and 1.41 on Seed-ZH is worse than 9 of 11. (c) "0.2B" excludes the 127.9M VAE and the speaker encoder, while training uses 550k hours of internal data, several times more than Emilia-based baselines, and data overlap with test speakers is not discussed. (d) The subjective test is small (50 utterances, 7 raters, 400 pairs, no confidence intervals) and covers only quality and naturalness. Converted to counts, several "preferences" are not significant (for example, base vs Pocket TTS naturalness 8/32/10; distilled vs F5-TTS naturalness 14/21/15). There is no speaker-similarity MOS and no direct base-versus-distilled test, so "comparable subjective quality" is not shown. (e) Efficiency: the absolute gain is 11.4 ms, the base RTF (0.184) is worse than Pocket TTS (0.125) and F5-TTS (0.122), baselines use different engines (vLLM-Omni, TensorRT), VibeVoice-Realtime is run without voice cloning, and the speedup mixes removal of the unconditional LM pass with fewer head steps.

Questions for authors
  1. What is technically new in guidance-step distillation compared with IntMeanFlow with a CFG teacher, LCM one-stage guided distillation, MeanFlow with CFG, and CALM latent CFG distillation? Do you have any matched comparison showing that joint distillation beats sequential guidance-then-step distillation, or beats any of these methods at the same NFE?
  2. Can you report Table 1 SIM with an independent verifier (WeSpeaker, CAM++, or ECAPA) and add ground-truth rows? Is the WavLM SV teacher the same checkpoint as, or shares training data with, the SIM evaluator? Do the training data include LibriSpeech, LibriTTS, or Common Voice/DiDiSpeech speakers used in the test sets?
  3. Which numbers in Tables 1 and 2 are copied and which are re-evaluated? What inference settings were used for each baseline? Can the main baselines be timed with the same engine or with the same level of optimization?
  4. What is the full deployed parameter count and peak memory, including VAE and speaker encoder? Which text tokenizer is used, and does inference require the reference transcript?
  5. How much of the 11.4 ms gain comes from removing the unconditional LM pass versus reducing head steps? Please give a component-level breakdown and clarify "first PCM chunk on the CPU".
5 comments
AuthorsOpenReview ↗

Response to Reviewer 6Sf8 (1/5)

W1 & Q1. Limited novelty: the system is mainly a combination of existing methods, and the paper does not make clear what is new. Every component maps directly to prior work... The only claimed method contribution, guidance-step distillation, is very close to existing work. Eq. 10 is IntMeanFlow's integral-velocity distillation (student regresses the teacher's average velocity over [t, t+dt]) with a CFG-guided teacher. Folding CFG into a one-stage few-step distillation with the guidance scale as an input is the core of Latent Consistency Models (augmented PF-ODE, w-embedding, skipping steps). MeanFlow already puts CFG into the average-velocity target for 1-NFE sampling, and CALM/Pocket TTS already distills CFG away for continuous AR TTS. The paper does not cite LCM, has no Related Work section, and never compares against any of these alternatives... Question: What is technically new in guidance-step distillation compared with IntMeanFlow with a CFG teacher, LCM one-stage guided distillation, MeanFlow with CFG, and CALM latent CFG distillation? Do you have any matched comparison showing that joint distillation beats sequential guidance-then-step distillation, or beats any of these methods at the same NFE?

Response:

We appreciate the detailed literature contextualization and have added citations and discussion of Latent Consistency Models (LCM) and dots.tts alongside IntMeanFlow in Sec. 2.4 of the revised manuscript.

We do not claim individual algorithmic primitives—such as integral-velocity regression (IntMeanFlow), guidance-scale conditioning (LCM), or CFG distillation in continuous-AR TTS (CALM / Pocket TTS)—as standalone inventions. Rather, our contribution is their unified integration into a single-stage guidance–step distillation recipe for LM-level CFG in a patch-autoregressive system, backed by controlled ablations and end-to-end streaming latency measurements:

  • Matched same-teacher comparisons (Table R1 / Table 16): Under an equal budget of 100K updates from the same teacher, single-stage joint training matches the quality of two-stage Guidance → step distillation (+0.2 to +0.5 SIM (WS) across all three step budgets on LibriSpeech-PC, with comparable WER and UTMOS) while avoiding a second training stage. It also avoids the 1-step collapse of guidance-only distillation (WER 3.00 vs. 41.99) and the 1–2 step degradation of step-only and Step → guidance variants.
  • Guidance-scale conditioning & curriculum ablation (Table R2 / Table 17): Conditioning on random w∈[0,5]w \in [0, 5] preserves quality at w=2w = 2 compared with fixed-ww training while allowing adjustable guidance strength at inference from a single checkpoint.
  • Architectural payoff for LM-level CFG (Table R5 / Table 14): Because CFG operates at the causal-LM level in CuteTTS, distilling guidance away removes an entire unconditional LM prefill/decoding stream in addition to halving diffusion-head passes—directly saving 6.0–6.1 ms of first-audio latency at a fixed step count.
AuthorsOpenReview ↗

Response to Reviewer 6Sf8 (2/5)

W2(a) & Q2. Speaker similarity, the main advantage, is suspicious. CuteTTS reports SIM 78.9 on LibriSpeech and 76.5 on Seed-EN, above every baseline and above commonly reported ground-truth levels for these protocols (about 0.69 for LibriSpeech-PC under the F5-TTS protocol, about 0.73 for Seed-EN human speech). Yet no ground-truth row is given. The speaker encoder is distilled from a WavLM-Large SV model and the SIM evaluator is a fine-tuned WavLM-Large. Table 4 shows the risk: removing speaker conditioning drops WavLM SIM by 17.9 points but WeSpeaker SIM by only 8.0. Table 1 reports no independent SIM. Question: Can you report Table 1 SIM with an independent verifier (WeSpeaker, CAM++, or ECAPA) and add ground-truth rows? Is the WavLM SV teacher the same checkpoint as, or shares training data with, the SIM evaluator? Do the training data include LibriSpeech, LibriTTS, or Common Voice/DiDiSpeech speakers used in the test sets?

Response:

Thank you for raising this important point.

  1. Shared WavLM checkpoint & independent WeSpeaker verifier: The WavLM-Large speaker-verification checkpoint used as the teacher for our speaker encoder is indeed the same checkpoint used for SIM (WL) evaluation. To remove this dependency, we added WeSpeaker (SIM (WS)) as an independent verifier for all locally evaluated systems on LibriSpeech-PC, reported in Table 1 and Table R3. The additional Seed-TTS SIM (WS) results are provided below Table R3 in this response. Our cross-system similarity claims are grounded in SIM (WS).
  2. Revised similarity positioning: Under WeSpeaker, CuteTTS achieves 86.4 on LibriSpeech (close to CosyVoice 3 at 87.7 and IndexTTS2 at 87.0, and higher than the remaining seven measured baselines) and 86.2 on Seed-TTS EN (vs. 88.1 for IndexTTS2 and 87.5 for Qwen3-TTS), while trailing further on Seed-TTS ZH (88.9 vs. 92.9 and 92.2). Accordingly, we have revised the paper's claims to state that CuteTTS achieves strong speaker similarity close to leading systems on LibriSpeech and Seed-TTS EN despite its compact 0.2B generative backbone, rather than claiming the highest similarity across all sets.
  3. Ground-truth rows & values exceeding ground truth: Table 1 now includes ground-truth rows from F5-TTS (2.23 / 69 on LibriSpeech-PC, 2.06 / 73 on Seed-TTS EN, and 1.26 / 76 on Seed-TTS ZH). Notably, several baselines also exceed ground-truth SIM (WL) (e.g., VoxCPM2 achieves 74.2 on LibriSpeech). Because SIM (WL) measures cosine similarity against the reference prompt rather than a separate enrollment utterance, closely matching the prompt's acoustic channel and speaking style can yield higher similarity than a distinct ground-truth utterance; we note this explicitly so SIM (WL) above ground truth is not over-interpreted.
  4. Training-data overlap check: We checked our 550K-hour training corpus against LibriSpeech, LibriTTS, Common Voice, and DiDiSpeech at the utterance/file level and confirmed zero identical utterances. We did not verify speaker-level overlap, so we cannot guarantee speaker-level disjointness.

W2(b). WER is not competitive: 2.04 on Seed-EN is worse than 9 of 13 baselines and 1.41 on Seed-ZH is worse than 9 of 11.

Response:

We agree that CuteTTS does not achieve the lowest WER on Seed-TTS. Our intended claim is competitive intelligibility within a compact (0.2B backbone), low-latency streaming system—achieving 2.16% on LibriSpeech (lower than the WER of several larger systems, including VoxCPM2 2.81%, IndexTTS2 2.50%, and F5-TTS 2.42%), alongside 2.04% on Seed-EN and 1.41% on Seed-ZH, while maintaining 37.6–49.0 ms first-audio latency. We have tempered our wording in the Abstract and Sec. 3.2 to reflect this balance accurately.

AuthorsOpenReview ↗

Response to Reviewer 6Sf8 (3/5)

W2(c). "0.2B" excludes the 127.9M VAE and the speaker encoder, while training uses 550k hours of internal data, several times more than Emilia-based baselines, and data overlap with test speakers is not discussed.

Response:

As detailed in Q4, Appendix A.1 reports both the generative backbone size (228.6M / 231.8M) and the full deployed parameter count (382.7M / 385.8M). The utterance-level overlap check is described in our response to Q2. Sec. 4 reports the ~50K-hour controlled subset results (WER 2.70 / SIM (WL) 71.3 / SIM (WS) 82.5 on LibriSpeech), distinguishing smaller-scale controlled evidence from full-scale system benchmarks.


W2(d). The subjective test is small (50 utterances, 7 raters, 400 pairs, no confidence intervals) and covers only quality and naturalness. Converted to counts, several "preferences" are not significant (for example, base vs Pocket TTS naturalness 8/32/10; distilled vs F5-TTS naturalness 14/21/15). There is no speaker-similarity MOS and no direct base-versus-distilled test, so "comparable subjective quality" is not shown.

Response:

We have added the direct blinded base-vs-distilled comparison (Table R4 / Table 13) and report the exact win/tie/loss counts (out of 50 pairs per comparison) below:

ComparisonNaturalness (Win / Tie / Loss)Sound Quality (Win / Tie / Loss)
Base vs. MOSS-TTS-Nano23 / 21 / 6 (p=0.002p = 0.002)16 / 26 / 8 (p=0.152p = 0.152)
Base vs. F5-TTS16 / 25 / 9 (p=0.230p = 0.230)24 / 23 / 3 (p<0.001p < 0.001)
Base vs. ZipVoice20 / 21 / 9 (p=0.061p = 0.061)37 / 9 / 4 (p<0.001p < 0.001)
Base vs. Pocket TTS8 / 32 / 10 (p=0.815p = 0.815)17 / 22 / 11 (p=0.345p = 0.345)
Distilled vs. MOSS-TTS-Nano25 / 15 / 10 (p=0.017p = 0.017)16 / 24 / 10 (p=0.327p = 0.327)
Distilled vs. F5-TTS14 / 21 / 15 (p=1.000p = 1.000)21 / 21 / 8 (p=0.024p = 0.024)
Distilled vs. ZipVoice20 / 21 / 9 (p=0.061p = 0.061)32 / 15 / 3 (p<0.001p < 0.001)
Distilled vs. Pocket TTS9 / 25 / 16 (p=0.230p = 0.230)15 / 25 / 10 (p=0.424p = 0.424)

Both CuteTTS variants receive more wins than losses against all four baselines in sound quality. Under the reported uncorrected tests, the sound-quality preferences are statistically significant against F5-TTS and ZipVoice, and the naturalness preference is significant against MOSS-TTS-Nano. Speaker similarity is evaluated objectively via both WavLM and WeSpeaker rather than via subjective MOS.

In the direct base-vs-distilled test (Table R4), 64% of pairs are tied in naturalness (13 / 32 / 5, p=0.096p = 0.096) and 76% in sound quality (8 / 38 / 4, p=0.39p = 0.39), supporting our characterization that the distilled model remains perceptually close to the base model with a possible slight loss in naturalness.


W2(e). Efficiency: the absolute gain is 11.4 ms, the base RTF (0.184) is worse than Pocket TTS (0.125) and F5-TTS (0.122), baselines use different engines (vLLM-Omni, TensorRT), VibeVoice-Realtime is run without voice cloning, and the speedup mixes removal of the unconditional LM pass with fewer head steps.

Response:

Our primary efficiency goal is low first-audio latency for interactive streaming, which differs from full-utterance throughput (RTF). In Table 2, CuteTTS-distill achieves the lowest mean first-audio latency among all evaluated systems (37.6 ms) with an RTF of 0.109 (faster than Pocket TTS at 0.125 and F5-TTS at 0.122, though slower in RTF than non-autoregressive ZipVoice at 0.030). We frame Table 2 as reflecting practical end-to-end latency–throughput trade-offs across released system implementations—and now explicitly annotate VibeVoice-Realtime's non-cloning default-voice setting directly in Table 2—while isolating our method's exact latency components in Table R5.

Reviewer_WVEmOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

CuteTTS is a streaming zero-shot TTS system combining semantically aligned continuous audio latents, patch-level autoregression, explicit speaker conditioning, and a flow-matching head. Its guidance–step distillation trains a single interval-conditioned student to absorb classifier-free guidance and multiple integration steps. Experiments report competitive intelligibility and speaker similarity, improved low-step generation, and reduced inference cost. Additional ablations examine representation rate, patch size, and module capacity.

Critical strengths
  1. A measurable efficiency improvement within the same system. Under a paired protocol, distillation reduces mean first-audio latency from 49.0 to 37.6 ms and RTF from 0.184 to 0.109. Low-step experiments also show that the student is more robust than directly reducing the base model's sampling steps. The detailed timing protocol makes these improvements reasonably interpretable.

  2. Useful controlled design studies. The paper jointly studies latent rate and patch size rather than assuming better reconstruction necessarily improves TTS. It also examines module capacity, semantic alignment, and speaker conditioning. The independent WeSpeaker evaluation in the speaker-conditioning ablation is a useful check on reliance on WavLM-based similarity.

Critical weaknesses
  1. The main methodological increment is not sufficiently established against relevant alternatives. The paper acknowledges prior patch-based continuous-autoregressive architectures, semantic alignment, and guidance/few-step distillation. Its principal distillation experiment compares the student with an undistilled base at reduced sampling budgets. This establishes that distillation helps, but not the added value of the particular joint guidance–step formulation. A matched existing few-step distillation baseline, or a sequential guidance-then-step alternative on the same teacher, would make the contribution substantially clearer. Without such evidence, it is difficult to distinguish a methodological advance from an effective integration of established ideas.

  2. The external quality comparisons and quality-preservation claim have important limits. The system uses approximately 550k hours of internal speech, without a matched-data architectural comparison, so its competitive benchmark results do not isolate the proposed design's contribution. Distillation also incurs objective quality losses, including LibriSpeech WER increasing from 2.16 to 2.41 and SIM decreasing from 78.9 to 76.8. The subjective study uses 50 examples and compares each variant against other systems rather than directly comparing base versus distilled outputs; it therefore provides limited evidence that the speedup preserves perceived quality. These limitations matter when the main case for the system is its quality–efficiency balance.

Questions for authors
  1. How does guidance–step distillation compare with an existing few-step method or sequential guidance-then-step distillation on the same teacher, at matched training and inference budgets? This would directly clarify the formulation's incremental value.

  2. Is there direct base-versus-student listening evidence, with uncertainty estimates, supporting comparable perceived quality? Can the controlled architectural evidence be distinguished more explicitly from the external comparisons involving different training data?

2 comments
AuthorsOpenReview ↗

Response to Reviewer WVEm (1/2)

W1 & Q1. The main methodological increment is not sufficiently established against relevant alternatives. The paper acknowledges prior patch-based continuous-autoregressive architectures, semantic alignment, and guidance/few-step distillation. Its principal distillation experiment compares the student with an undistilled base at reduced sampling budgets. This establishes that distillation helps, but not the added value of the particular joint guidance–step formulation. A matched existing few-step distillation baseline, or a sequential guidance-then-step alternative on the same teacher, would make the contribution substantially clearer. Without such evidence, it is difficult to distinguish a methodological advance from an effective integration of established ideas. Question: How does guidance–step distillation compare with an existing few-step method or sequential guidance-then-step distillation on the same teacher, at matched training and inference budgets? This would directly clarify the formulation's incremental value.

Response:

We added the requested same-teacher comparison in Table R1 (Table 16 in the revised paper), where all student variants start from the identical frozen teacher and are trained for the same total budget of 100K updates:

  • Joint vs. sequential (Guidance → step): Single-stage joint distillation achieves quality comparable to the two-stage Guidance → step baseline without requiring a second training stage or optimizer restart. Across all three step budgets on LibriSpeech-PC, joint training yields slightly higher speaker similarity (+0.5 to +1.2 SIM (WL) and +0.2 to +0.5 SIM (WS)), while WER differs by -0.10 to +0.29 without a uniform ranking. We present this as a simpler, unified single-stage training recipe with comparable overall quality rather than claiming dominance on every metric.
  • Why both guidance and interval targets are needed: Guidance-only distillation is competitive at 4 steps (WER 2.25) but degrades severely at 1 step (WER 41.99). Conversely, step-only distillation retains 2 LM streams and degrades at 1–2 steps (WER 4.01 and 51.16), and Step → guidance also breaks down at 1–2 steps (WER 4.54 and 59.84). Combining guidance removal and interval learning in a single stage produces one checkpoint that supports 4-, 2-, and 1-step single-stream inference (WER 2.34 / 2.37 / 3.00).
AuthorsOpenReview ↗

Response to Reviewer WVEm (2/2)

W2 & Q2. The external quality comparisons and quality-preservation claim have important limits. The system uses approximately 550k hours of internal speech, without a matched-data architectural comparison, so its competitive benchmark results do not isolate the proposed design's contribution. Distillation also incurs objective quality losses, including LibriSpeech WER increasing from 2.16 to 2.41 and SIM decreasing from 78.9 to 76.8. The subjective study uses 50 examples and compares each variant against other systems rather than directly comparing base versus distilled outputs; it therefore provides limited evidence that the speedup preserves perceived quality. These limitations matter when the main case for the system is its quality–efficiency balance. Question: Is there direct base-versus-student listening evidence, with uncertainty estimates, supporting comparable perceived quality? Can the controlled architectural evidence be distinguished more explicitly from the external comparisons involving different training data?

Response:

We address each point with new experiments and clearer framing in the revised manuscript:

  1. Separating data scale from architectural evidence: In Sec. 4, we explicitly distinguish system-level benchmark comparisons (Sec. 3) from our representation and conditioning ablations trained on a ~50K-hour subset. On LibriSpeech test-clean, the default 50K-hour model achieves WER 2.70 / SIM (WL) 71.3 / SIM (WS) 82.5 (Table 4), compared with 2.16 / 78.9 / 86.4 for the 550K-hour model. This provides a smaller-data reference point alongside the controlled design studies.
  2. Objective quality–latency trade-off: Across the three test sets, distillation changes WER by -0.01 to +0.25 absolute, while reducing SIM (WL) by 2.1–2.3 points and SIM (WS) by 0.9–1.0 points, in exchange for a 23.3% reduction in mean first-audio latency (49.0 → 37.6 ms) and a 40.8% reduction in RTF (0.184 → 0.109). We frame these metrics jointly as our measured quality–efficiency trade-off.
  3. Direct base-vs-distilled listening test (Table R4 / Table 13): We conducted a blinded A/B listening evaluation directly comparing CuteTTS and CuteTTS-distill on the 50 LibriSpeech pairs under the same protocol as Appendix C. Most pairs were judged tied: 64% (32/50, 95% Wilson CI: [50.1%, 75.9%]) for naturalness and 76% (38/50, 95% CI: [62.6%, 85.7%]) for sound quality. Non-tied preferences lean toward the base model (13 vs. 5 for naturalness, two-sided sign-test p=0.096p = 0.096; 8 vs. 4 for sound quality, p=0.39p = 0.39), with neither difference reaching statistical significance. Accordingly, we describe the distilled model as perceptually close to the base model, with a possible slight loss in naturalness, alongside its latency gains.
Reviewer_7zs6OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

CuteTTS is a compact continuous-autoregressive TTS system combining a semantically aligned causal VAE, patch-level generation, and explicit speaker conditioning. It uses guidance–step distillation to reduce inference cost. The paper evaluates zero-shot voice cloning and streaming efficiency, and studies how representation rate, patch size, and module capacity affect synthesis quality.

Critical strengths
  1. The rate–patch and module-capacity analyses are informative and offer useful guidance for designing compact continuous-AR TTS systems.
  2. The system achieves a good quality–efficiency balance, supported by objective metrics, listening preferences, and paired latency measurements.
Critical weaknesses

Major

The novelty of the main distillation method is unclear. The earlier dots.tts technical report (June 2026, v1, Section 2.6.3, Eqs. 13–14 [1]) already learns interval-averaged velocities from a CFG-guided teacher to reduce sampling steps and remove the unconditional branch. Eqs. 10–11 here use essentially the same mechanism, but this work is neither cited nor discussed. Guidance-strength conditioning and the interval-sampling schedule differ, but the experiments do not establish what these changes add.

[1] https://arxiv.org/html/2606.07080v1#S2.SS6.SSS3

Minor

  1. Some conclusions from the analysis are broader than the evidence supports. In Table 8, VAE size varies from 49.2M to 232.7M across frame rates, so capacity changes alongside temporal resolution. The capacity study in Table 12 covers only two configurations at 50 Hz.
  2. The “0.2B” parameter count needs clarification. The 228.6M in Table 6 excludes the 127.9M VAE in Table 5 and the separate speaker encoder. Since compactness is central to the paper, the main comparison should clarify whether parameter counts follow a consistent convention across systems.
Questions for authors

What does the proposed distillation add beyond CFG-aware MeanFlow in dots.tts?

3 comments
AuthorsOpenReview ↗

Response to Reviewer 7zs6 (1/2)

Major Concern & Question. The novelty of the main distillation method is unclear. The earlier dots.tts technical report (June 2026, v1, Section 2.6.3, Eqs. 13–14) already learns interval-averaged velocities from a CFG-guided teacher to reduce sampling steps and remove the unconditional branch. Eqs. 10–11 here use essentially the same mechanism, but this work is neither cited nor discussed. Guidance-strength conditioning and the interval-sampling schedule differ, but the experiments do not establish what these changes add. Question: What does the proposed distillation add beyond CFG-aware MeanFlow in dots.tts?

Response:

Thank you for bringing dots.tts to our attention. We developed our method independently and were unaware of this technical report at the time of submission. Having reviewed Section 2.6.3, we recognize that both formulations train an interval-conditioned student to regress the finite-difference average velocity of a CFG-guided teacher trajectory (our Eqs. 10–11). We have added a citation and discussion of dots.tts in Sec. 2.4 of the revised manuscript.

The methodological and architectural differences between the two recipes are summarized below:

Aspectdots.tts (MF)CuteTTS (Ours)
Guidance scaleFixed in the teacher target (γ=1.2\gamma = 1.2)Sampled uniformly from [0,5][0, 5] during training and passed to the student as an input
Where CFG operatesAR flow-matching headCausal LM level; distillation eliminates both the unconditional LM stream and unconditional head branch
Interval samplingContinuous, log-normal samplingDiscrete grid intervals matching 4-, 2-, and 1-step inference with a curriculum
Evaluated budgets2–4 NFEs4, 2, and 1 NFEs from a single checkpoint

To directly establish what these specific design choices add, we conducted the 2×22 \times 2 ablation in Table R2 (Table 17 in the revised paper) comparing random vs. fixed ww, with and without the interval curriculum:

  • Effect of random-ww conditioning: At w=2w = 2 and 4 steps on LibriSpeech-PC, random-ww and fixed-ww students achieve comparable accuracy (WER 2.34 vs. 2.18, SIM (WS) 85.4 vs. 85.4, UTMOS 4.069 vs. 4.061). Thus, random-ww training provides inference-time controllability over guidance strength from a single checkpoint (as illustrated across w∈[1,5]w \in [1, 5] in Figure 4) without a meaningful quality penalty at the default operating point.
  • Effect of the interval curriculum: Removing the curriculum yields similar final quality at w=2w = 2 (WER 2.22 vs. 2.34 at 4 steps), confirming that the curriculum is a practical training schedule choice rather than the primary driver of final accuracy.
  • System-level contribution: Because CFG acts at the causal-LM level in our architecture, distillation removes an entire unconditional causal-LM forward stream in addition to reducing diffusion-head evaluations, saving 6.0–6.1 ms of first-audio latency by removing the unconditional LM and head path (Table R5).
AuthorsOpenReview ↗

Response to Reviewer 7zs6 (2/2)

Minor 1. Some conclusions from the analysis are broader than the evidence supports. In Table 8, VAE size varies from 49.2M to 232.7M across frame rates, so capacity changes alongside temporal resolution. The capacity study in Table 12 covers only two configurations at 50 Hz.

Response:

We agree and have narrowed the wording of both claims in the revised manuscript (Sec. 4.1, Appendix B.1, Appendix B.3):

  • VAE frame-rate ablation (Table 8): Changing the final stride alters VAE parameter count from 232.7M (6.25 Hz) to 49.2M (50 Hz). Crucially, reconstruction quality improves monotonically as frame rate increases even though parameter count decreases; therefore, the reconstruction trend cannot be attributed to larger VAE capacity. In the subsequent rate–patch TTS experiments (Tables 9–10), the causal backbone and diffusion head are held fixed. We now explicitly note the non-parameter-matched VAE sizes in Appendix B.1.
  • Module-wise capacity study (Table 12): TTS50,1\text{TTS}_{50,1} and TTS50,8\text{TTS}_{50,8} were selected to contrast two extreme regimes at a fixed 50-Hz representation: fine-grained 1-frame patches (50 tokens/s) vs. coarse-grained 8-frame patches (6.25 tokens/s). In Sec. 4.1 and Appendix B.3, we now explicitly scope this finding to the tested 50-Hz configurations rather than generalizing to all frame rates.

Minor 2. The “0.2B” parameter count needs clarification. The 228.6M in Table 6 excludes the 127.9M VAE in Table 5 and the separate speaker encoder. Since compactness is central to the paper, the main comparison should clarify whether parameter counts follow a consistent convention across systems.

Response:

We have clarified this convention in the Abstract, Introduction, and Appendix A.1. Specifically, the 0.2B figure denotes the generative TTS backbone (228.6M for CuteTTS and 231.8M for CuteTTS-distill), while the complete deployed systems—including the 127.9M causal VAE and 26.2M speaker encoder—contain 382.7M and 385.8M parameters, respectively. Baseline parameter counts in Table 1 follow the conventions reported in their respective papers.

Thanks, that clears up some of my questions. I’m still not convinced by the point about where CFG operates. Before distillation, the base model requires a second LM stream for the unconditional branch, whereas dots.tts applies CFG at the head without that extra LM computation. Removing that extra LM stream saves time, but I’m not sure the base model needed it in the first place.

I liked the rate and patch analysis, but overall the advance still feels too small for ICLR. I’ll keep my score as is.

Reviewer_i1wXOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes CuteTTS, a continuous AR TTS model designed to reduce streaming latency while preserving speech quality and speaker similarity. It generates patches of continuous latents from a semantically aligned causal σ\sigma-VAE, using speaker embedding and a flow-matching head. It also jointly distills classifier-free guidance and multiple sampling steps to reduce first audio latency relative to the baseline model.

Critical strengths

S1. This paper reduces first-audio latency through guidance–step distillation, which removes the need for a separate CFG branch and supports 1/2/4-step inference with a single student.

S2. The model achieves competitive speaker similarity with only 0.2B parameters. The distilled model also largely preserves speaker similarity while reducing inference latency.

Critical weaknesses

W1. My main concern is the limited novelty compared to previous work. The overall model follows the continuous patch-based diffusion autoregressive architecture of DiTAR. σ\sigma-VAE and semantic alignment are also based on existing methods, and it is unclear what is new in these components. Also, explicit speaker conditioning, which is presented as one of the main contributions, is already widely used in TTS systems. More importantly, it is unclear how the proposed guidance-step distillation differs from the existing IntMeanFlow method, and the paper does not provide a direct comparison.

W2. CuteTTS is trained on around 550K hours of internal data, which is much larger than the training data used by many baseline models. This makes it difficult to show whether the high speaker similarity comes from the proposed architecture or from the larger training dataset and speaker diversity. The paper also does not provide a demo page with audio samples, so I could not directly listen to the generated speech or compare it with the baselines.

W3. In Table 4, using speaker embeddings only in the diffusion head achieves almost the same similarity as the default model, while showing a lower WER. It is unclear what benefit comes from adding speaker embeddings to the LM backbone as well. More analysis is needed to show whether these results are consistent across other evaluation sets or when using the full training dataset.

Questions for authors

Q1. For the "Causal LM Backbone," did the authors train a randomly initialized Transformer from scratch, without using a pre-trained LLM?

Q2. In Figure 4, CuteTTS-distill shows much higher UTMOS scores than CuteTTS. What causes this difference, and is the improvement also reflected in subjective listening evaluations?

Q3. Could the authors clarify where the baseline results come from and how the model sizes were counted? I think the reported DiTAR results may come from the 1B model rather than the 0.6B model.

Q4. The paper does not report details about the training setup, such as the training GPU type, number of GPUs, and training time. There is also limited information about the internal 550K-hour dataset. Could the authors provide more details about the text tokenizer, initial LR, training hardware, and total training time?

3 comments
AuthorsOpenReview ↗

Response to Reviewer i1wX (1/3)

W1. My main concern is the limited novelty compared to previous work. The overall model follows the continuous patch-based diffusion autoregressive architecture of DiTAR. σ\sigma-VAE and semantic alignment are also based on existing methods, and it is unclear what is new in these components. Also, explicit speaker conditioning, which is presented as one of the main contributions, is already widely used in TTS systems. More importantly, it is unclear how the proposed guidance-step distillation differs from the existing IntMeanFlow method, and the paper does not provide a direct comparison.

Response:

Thank you for giving us the opportunity to clarify our contributions. Rather than claiming novelty for every individual architectural building block, CuteTTS integrates a compact streaming architecture with systematic design-space analyses and a unified single-stage distillation recipe tailored to LM-level CFG. Specifically, our contributions are:

  1. A compact, low-latency continuous-AR TTS system: Using a 0.2B generative backbone, CuteTTS-distill achieves the lowest mean first-audio latency among all measured systems (37.6 ms) while attaining LibriSpeech WER and speaker similarity comparable to substantially larger models (Table R3).
  2. Controlled design-space studies: We systematically study VAE frame rate, patch size, semantic alignment, speaker conditioning injection sites, and module-wise depth allocation under matched data and training budgets (Sec. 4, Appendix B).
  3. Single-stage guidance–step distillation for LM-level CFG: We evaluate our joint distillation recipe against guidance-only, step-only, and two-stage sequential baselines on the exact same frozen teacher (Table R1).

Regarding IntMeanFlow, Eq. 10 indeed employs the same finite-difference integral-velocity target applied to a CFG-guided teacher, which we explicitly acknowledge in Sec. 2.4 of the revised manuscript. Our system-level distinction lies in applying this formulation to LM-level CFG in a patch-autoregressive architecture—where distillation eliminates both the unconditional diffusion branch and the unconditional causal-LM stream—while conditioning the student on the guidance scale ww so that a single checkpoint supports multiple guidance strengths and step budgets.

Table R1 (Table 16 in the revised appendix) provides the requested comparison under an equal budget of 100K total updates:

  • Joint vs. sequential (Guidance → step): Single-stage joint training matches the overall quality of the two-stage pipeline, achieving higher SIM (WL) (+0.5 to +1.2) and SIM (WS) (+0.2 to +0.5) across all three step budgets on LibriSpeech-PC, with small differences in WER and UTMOS.
  • Role of interval distillation: Guidance-only distillation performs well at 4 steps (WER 2.25) but collapses at 1 step (WER 41.99), whereas our joint recipe maintains robust intelligibility down to 1 step (WER 3.00). Thus, the practical contribution is consolidating a two-stage distillation pipeline into a single training stage while preserving low-step inference capability.

W2. CuteTTS is trained on around 550K hours of internal data, which is much larger than the training data used by many baseline models. This makes it difficult to show whether the high speaker similarity comes from the proposed architecture or from the larger training dataset and speaker diversity. The paper also does not provide a demo page with audio samples, so I could not directly listen to the generated speech or compare it with the baselines.

Response:

We distinguish between system-level benchmark comparisons (Sec. 3) and controlled architectural ablations (Sec. 4). The representation and conditioning ablations in Sec. 4 use a ~50K-hour subset with matched training settings. On LibriSpeech test-clean, the default configuration trained on this 50K-hour subset achieves WER 2.70 / SIM (WL) 71.3 / SIM (WS) 82.5 (Table 4), compared with 2.16 / 78.9 / 86.4 for the 550K-hour model. These results provide a direct smaller-scale reference point and demonstrate that our compact architecture scales effectively when trained on larger data. We have clarified this distinction between system-level comparisons and controlled architectural evidence in Sec. 4.

AuthorsOpenReview ↗

Response to Reviewer i1wX (2/3)

W3. In Table 4, using speaker embeddings only in the diffusion head achieves almost the same similarity as the default model, while showing a lower WER. It is unclear what benefit comes from adding speaker embeddings to the LM backbone as well. More analysis is needed to show whether these results are consistent across other evaluation sets or when using the full training dataset.

Response:

Table 4 confirms that explicit speaker conditioning is essential for zero-shot cloning—removing it drops SIM (WS) by 8.0 points (82.5 → 74.5)—and pinpoints the diffusion head (via AdaLN) as the primary injection site. Specifically, diff. head spk matches the default dual-site model in speaker similarity (SIM (WL): 71.2 vs. 71.3; SIM (WS): 82.5 vs. 82.5) while achieving slightly lower WER (2.52 vs. 2.70). In Sec. 4.3 of the revised manuscript, we have revised our interpretation accordingly: we no longer attribute an additional similarity gain to LM-side speaker conditioning, treating it as optional. Because the full-scale 550K-hour model had already been trained with the default dual-site configuration prior to this ablation, we retain that checkpoint for the main benchmarks.


Q1. For the "Causal LM Backbone," did the authors train a randomly initialized Transformer from scratch, without using a pre-trained LLM?

Response:

Yes. The causal LM backbone is randomly initialized and trained entirely from scratch without any pre-trained LLM weights. It uses a Qwen3-style decoder block architecture with 7 layers, hidden dimension 1,024, 16 attention / 8 KV heads, and FFN dimension 3,072 (Table 6).


Q2. In Figure 4, CuteTTS-distill shows much higher UTMOS scores than CuteTTS. What causes this difference, and is the improvement also reflected in subjective listening evaluations?

Response:

Figure 4 evaluates both models under constrained step budgets (4, 2, and 1 NFE). Because the base model is designed for 10 NFEs per branch whereas the student is explicitly trained to match the guided teacher trajectory in fewer steps, the large UTMOS gap at 1–2 NFEs reflects the base model's numerical integration degradation rather than an improvement over the base model's default setting.

Table R1 makes both comparisons explicit on a uniform time grid:

  • At default operating points (10-step base vs. 4-step distilled): UTMOS is nearly identical—4.100 vs. 4.069 on LibriSpeech, 3.766 vs. 3.751 on Seed-TTS EN, and 2.911 vs. 2.903 on Seed-TTS ZH. This aligns with our blinded A/B listening test (Table R4), where 76% of pairs are rated equal in sound quality (8 base wins / 38 ties / 4 distilled wins, p=0.39p = 0.39).
  • At matched low step counts on LibriSpeech: The distilled student substantially outperforms the base model (4.069 vs. 3.974 at 4 steps, 3.856 vs. 3.130 at 2 steps, and 3.144 vs. 1.329 at 1 step).

We have updated the Figure 4 caption to clearly distinguish these low-step comparisons from the default operating points.


Q3. Could the authors clarify where the baseline results come from and how the model sizes were counted? I think the reported DiTAR results may come from the 1B model rather than the 0.6B model.

Response:

Thank you for catching the DiTAR distinction. We have separated the two DiTAR configurations in Table 1. The result sources are summarized below, and the full deployed parameter counts for CuteTTS are reported in Appendix A.1.

SourceSystems
Evaluated by us on all reported setsQwen3-TTS, MOSS-TTS-Nano, IndexTTS2, Pocket TTS, CuteTTS, CuteTTS-distill
LibriSpeech evaluated by us; Seed-TTS from official reportsMOSS-TTS, FireRedTTS-2, ZipVoice, CosyVoice 3, VoxCPM2
All metrics taken from official reportsF5-TTS, VibeVoice, VibeVoice-Realtime, DiTAR
DiTAR ConfigurationParamsLibriSpeech (WER / SIM (WL))Seed-TTS ENSeed-TTS ZH
DiTAR (Emilia)0.6B2.39 / 67.0––
DiTAR (280K hours)1B–1.69 / 73.51.02 / 75.3

Baseline parameter counts follow their respective reports. For CuteTTS, 228.6M (231.8M distilled) counts the generative backbone (patch encoder, causal LM, and diffusion head). Including the audio VAE (127.9M) and speaker encoder (26.2M), the full deployed totals are 382.7M and 385.8M, respectively (detailed in Appendix A.1).

AuthorsOpenReview ↗

Response to Reviewer i1wX (3/3)

Q4. The paper does not report details about the training setup, such as the training GPU type, number of GPUs, and training time. There is also limited information about the internal 550K-hour dataset. Could the authors provide more details about the text tokenizer, initial LR, training hardware, and total training time?

Response:

The TTS backbone was trained on 8 NVIDIA H100 GPUs for approximately 7 days, using AdamW with a peak learning rate of 5×10−45 \times 10^{-4}, 5K linear warmup steps, and cosine decay. The 550K-hour multilingual dataset was curated from publicly available internet speech corpora and processed through an automated filtering and ASR transcription pipeline. We use a BPE text tokenizer with a 16,384-token vocabulary trained on the TTS training transcripts. The optimizer, learning-rate schedule, and tokenizer details are provided in Appendix A.1–A.2.

Comments on the paper

3 comments
AuthorsOpenReview ↗

General Response to All Reviewers

General Response

We thank all reviewers for their thorough and constructive feedback. In response to your comments, we have added controlled same-teacher distillation comparisons, an independent speaker-similarity verifier, a direct base-vs-distilled blinded listening test, and a fine-grained latency decomposition. These additions clarify the core contribution of CuteTTS: a compact streaming continuous-autoregressive TTS system supported by systematic design-space analyses and a single-stage guidance–step distillation recipe for low-latency inference.

1. Distillation comparisons (Tables R1, R2; Tables 16, 17 in the revised paper). Our guidance–step distillation unifies classifier-free guidance (CFG) removal and interval-wise integration within a single diffusion head for LM-level guided patch generation. Trained from the same teacher with an equal budget of 100K updates, our single-stage joint training achieves synthesis quality comparable to two-stage Guidance → step distillation: SIM (WS) is 0.2–0.5 points higher across all three step budgets on LibriSpeech-PC, with small differences in WER and UTMOS. Whereas guidance-only distillation remains competitive at 4 steps but collapses at 1 step, incorporating the interval-velocity target preserves robustness at 1–2 steps. Furthermore, training with random guidance scales w∈[0,5]w \in [0, 5] enables a single checkpoint to support adjustable inference-time guidance without degrading performance at the default w=2w = 2 setting. The interval sampling curriculum serves as a training schedule choice and does not show a consistent final-quality advantage at this operating point.

2. Speaker similarity evaluation (Table R3; Table 1 in the revised paper). Because our WavLM-Large evaluator shares its checkpoint with the teacher of our speaker encoder, we added WeSpeaker as an independent verifier across all locally evaluated systems. With a 0.2B generative backbone, CuteTTS achieves a SIM (WS) of 86.4 on LibriSpeech and 86.2 on Seed-TTS EN—approaching the strongest measured systems on these benchmarks (87.7 and 88.1). Accordingly, we frame our result as strong speaker similarity for a compact system rather than the highest across all benchmarks, and we transparently report the wider gap on Seed-TTS ZH.

3. Direct base-vs-distilled listening comparison (Table R4; Table 13 in the revised paper). In our blinded A/B listening test on 50 LibriSpeech pairs, raters judged the majority of samples to be tied: 64% for naturalness and 76% for sound quality. Non-tied preferences lean toward the base model (13 vs. 5 for naturalness and 8 vs. 4 for sound quality), with neither difference reaching statistical significance (p=0.096p = 0.096 and p=0.39p = 0.39, two-sided sign test). Combined with the objective metrics, these findings demonstrate a practical quality–latency trade-off: the distilled student remains perceptually close to the base teacher, with minor objective differences and a possible slight reduction in naturalness.

4. Latency decomposition (Table R5; Tables 14, 15 in the revised paper). Removing the unconditional LM and head path saves 6.0–6.1 ms, while reducing diffusion-head steps from 10 to 4 saves 7.6–7.7 ms. These nearly additive gains explain how both components of our distillation recipe directly reduce first-audio latency. In the cross-system benchmark (Table 2), CuteTTS-distill achieves the lowest mean first-audio latency among all evaluated systems (37.6 ms).

5. Parameter count clarification (Appendix A.1). The 0.2B figure refers specifically to the generative TTS backbone (228.6M parameters for the base model and 231.8M for the distilled model). We retain this backbone count in the main comparison table while explicitly reporting the full deployed parameter counts—382.7M and 385.8M, including the 127.9M causal VAE and 26.2M speaker encoder—in Appendix A.1.

The revised manuscript preserves the paper's central focus on compact, low-latency streaming TTS and incorporates the new experiments. Further clarifications are provided in the responses below.

AuthorsOpenReview ↗

Additional Results (1/2): Tables R1–R3

Table R1. Controlled distillation comparisons (Table 16 in the revised paper). Evaluated on LibriSpeech-PC test-clean (1,127 utterances) with w=2w = 2 and Euler integration on a uniform time grid. Cells report WER (%) / SIM (WL) / SIM (WS) / UTMOS. Streams denotes the number of causal-LM streams; diffusion-head evaluations per patch equal steps×streams\text{steps} \times \text{streams}.

MethodStreams10 steps4 steps2 steps1 step
Base (teacher, undistilled)22.25 / 78.6 / 86.3 / 4.1002.24 / 78.5 / 86.4 / 3.9742.74 / 73.8 / 84.0 / 3.13048.71 / 23.7 / 61.8 / 1.329
Guidance-only12.35 / 76.7 / 85.3 / 4.1352.25 / 76.7 / 85.5 / 3.9762.57 / 72.1 / 83.1 / 3.20641.99 / 29.1 / 62.7 / 1.439
Step-only2–2.37 / 77.8 / 85.9 / 4.0404.01 / 67.9 / 81.8 / 2.75451.16 / 31.2 / 65.5 / 1.316
Guidance → step1–2.30 / 76.4 / 85.2 / 4.0822.47 / 76.0 / 84.9 / 3.8722.71 / 71.8 / 82.4 / 3.212
Step → guidance1–2.44 / 74.4 / 84.1 / 4.0504.54 / 63.3 / 79.1 / 2.69259.84 / 28.5 / 64.0 / 1.302
Joint (ours)1–2.34 / 76.9 / 85.4 / 4.0692.37 / 76.6 / 85.2 / 3.8563.00 / 73.0 / 82.9 / 3.144

Table R2. Ablation on guidance-scale sampling and interval curriculum (2×22 \times 2, Table 17 in the revised paper). Same evaluation protocol and cell format as Table R1. All four variants are joint students trained for 100K updates with guidance-scale conditioning; fixed-ww students always receive w=2w = 2 during training. Without the curriculum, intervals for 4-, 2-, and 1-step targets are sampled with fixed probabilities 0.45 / 0.30 / 0.25 from the start, matching the expected exposure under the curriculum.

VariantTraining wwCurriculum4 steps2 steps1 step
A (= Joint)random, [0, 5]yes2.34 / 76.9 / 85.4 / 4.0692.37 / 76.6 / 85.2 / 3.8563.00 / 73.0 / 82.9 / 3.144
Bfixed, 2yes2.18 / 76.7 / 85.4 / 4.0612.28 / 76.4 / 85.1 / 3.8302.73 / 72.4 / 82.6 / 3.165
Crandom, [0, 5]no2.22 / 76.8 / 85.5 / 4.0462.60 / 76.6 / 85.1 / 3.8382.68 / 72.6 / 82.8 / 3.139
Dfixed, 2no2.28 / 76.8 / 85.4 / 4.0582.47 / 76.5 / 84.9 / 3.8362.86 / 72.2 / 82.4 / 3.145

Table R3. Speaker similarity and intelligibility on LibriSpeech-PC test-clean (1,127 utterances; subset of Table 1 in the revised paper). Locally evaluated systems are sorted by independent WeSpeaker similarity SIM (WS), with ground-truth values (reported by F5-TTS) shown first. WL and WS denote WavLM-Large and WeSpeaker, respectively. Baseline parameter counts follow their official reports; CuteTTS counts denote the generative backbone (full deployed counts are 382.7M for base and 385.8M for distilled).

ModelParamsWER (%)SIM (WL)SIM (WS)
Ground truth (reported by F5-TTS)–2.2369–
CosyVoice 30.5B1.9969.787.7
IndexTTS21.5B2.5070.187.0
CuteTTS0.2B2.1678.986.4
VoxCPM22B2.8174.285.9
Qwen3-TTS1.7B1.9470.685.8
CuteTTS-distill0.2B2.4176.885.5
ZipVoice0.1B2.0567.485.2
MOSS-TTS8B1.8867.684.0
FireRedTTS-21.5B3.8364.381.1
MOSS-TTS-Nano0.1B4.1048.472.2
Pocket TTS0.1B1.5949.171.6

Seed-TTS results (WER / SIM (WL) / SIM (WS)):

  • Seed-TTS EN: IndexTTS2 2.21 / 70.9 / 88.1, Qwen3-TTS 1.61 / 71.7 / 87.5, CuteTTS 2.04 / 76.5 / 86.2, CuteTTS-distill 2.03 / 74.2 / 85.2 (Ground truth reported by F5-TTS: 2.06 / 73 / –).
  • Seed-TTS ZH: IndexTTS2 1.07 / 76.5 / 92.9, Qwen3-TTS 0.98 / 77.1 / 92.2, CuteTTS 1.41 / 77.8 / 88.9, CuteTTS-distill 1.47 / 75.6 / 87.9 (Ground truth reported by F5-TTS: 1.26 / 76 / –).
AuthorsOpenReview ↗

Additional Results (2/2): Tables R4–R5

Table R4. Blinded A/B comparison of base vs. distilled models (Table 13 in the revised paper). Evaluated on 50 LibriSpeech test-clean pairs using default inference settings under the protocol in Appendix C. Counts are Base wins / Ties / Distilled wins.

DimensionBase wins / Ties / Distilled wins
Naturalness13 / 32 / 5
Sound quality8 / 38 / 4

Table R5. Controlled latency-factor decomposition (Table 14 in the revised paper). Measured on one NVIDIA RTX 4090 over the fixed 50 requests and per-item seeds (protocol in Appendix D). Configurations C and E run the base weights without CFG as computational cost controls rather than quality-matched models.

ConfigWeightsLM streamsSteps per branchMean first-audio latency (ms)RTFPeak allocated memory (MiB)
A (base default)base21049.840.1901,932
Bbase2442.120.1441,931
Cbase11043.750.1511,931
Ebase1436.160.1071,932
D (distilled default)distilled1435.670.1101,943

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.