- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
CuteTTS is a compact streaming zero-shot TTS system combining a causal, semantically aligned continuous speech VAE, explicit speaker conditioning, patch-level autoregression, and a bidirectional flow-matching head. Speech is represented by 64-dimensional latents at 12.5 Hz and grouped into two-frame patches, giving 6.25 autoregressive decisions per second. A guidance-step distilled head learns the interval-averaged velocity of a classifier-free-guided teacher, jointly reducing solver steps and removing the unconditional branch. On LibriSpeech and Seed-TTS evaluation sets, the system reports competitive intelligibility and speaker similarity. A paired RTX 4090 experiment reduces mean first-audio latency from 49.0 to 37.6 ms and RTF from 0.184 to 0.109. The paper also studies representation rate, patch size, semantic alignment, speaker conditioning, and module capacity allocation.
Critical strengths
- A practical, measured quality/latency trade-off. The distilled system reduces diffusion-head evaluations from ten steps on two branches to four on one branch and reports a consistent end-to-end latency improvement, not only an NFE reduction. Table 2 includes mean and tail latency, and Appendix D times raw-reference loading, encoding, decoding, and host transfer without reusing request-specific cached features. This is a meaningful contribution for responsive voice cloning.
- Useful analysis of representation granularity and conditioning. The paper distinguishes VAE frame rate from autoregressive token rate and studies both fixed-patch and fixed-token-rate settings. Semantic alignment improves TTS intelligibility without materially changing reconstruction, while explicit speaker conditioning improves similarity under both WavLM and WeSpeaker in Table 4. These analyses explain design choices rather than presenting only an end-system leaderboard.
Critical weaknesses
-
Limited novelty: the system is mainly a combination of existing methods, and the paper does not make clear what is new.** Every component maps directly to prior work. The causal VAE follows DAC and VoxCPM; the sigma-VAE comes from LatentLM and VibeVoice; the semantic alignment is the Semantic-VAE recipe (WavLM features, projection, cosine loss), as the paper itself states. The generator "follows the patch-based formulation of DiTAR" (Sec. 2.3): aggregation encoder with a summary token, causal LM, bidirectional local diffusion head with the previous patch as prefix, stop head, and LM-level CFG with 0.1 dropout. VoxCPM/VoxCPM2 use the same LocEnc-LM-LocDiT layout over a causal VAE, and VoxCPM2 also runs at 6.25 Hz token rate. Explicit speaker embeddings in AR TTS are standard (CosyVoice, MiniMax-Speech, Qwen3-TTS), and the encoder here is a distilled ECAPA. The only claimed method contribution, guidance-step distillation, is very close to existing work. Eq. 10 is IntMeanFlow's integral-velocity distillation (student regresses the teacher's average velocity over [t, t+dt]) with a CFG-guided teacher. Folding CFG into a one-stage few-step distillation with the guidance scale as an input is the core of Latent Consistency Models (augmented PF-ODE, w-embedding, skipping steps). MeanFlow already puts CFG into the average-velocity target for 1-NFE sampling, and CALM/Pocket TTS already distills CFG away for continuous AR TTS. The paper does not cite LCM, has no Related Work section, and never compares against any of these alternatives. In particular, the claim that guidance removal and step reduction are "learned jointly rather than as two sequential distillation stages" is not tested: there is no sequential baseline, no IntMeanFlow or LCM-style baseline, and no guidance-only or step-only ablation under the same budget. For ICLR, a well-engineered combination can be acceptable if it yields new understanding or clearly superior results. Here, the understanding is mostly confirmatory and the results are not clearly superior (see W2).
-
The evidence does not support the "high-quality", "competitive", and "compact" claims.(a) Speaker similarity, the main advantage, is suspicious. CuteTTS reports SIM 78.9 on LibriSpeech and 76.5 on Seed-EN, above every baseline and above commonly reported ground-truth levels for these protocols (about 0.69 for LibriSpeech-PC under the F5-TTS protocol, about 0.73 for Seed-EN human speech). Yet no ground-truth row is given. The speaker encoder is distilled from a WavLM-Large SV model and the SIM evaluator is a fine-tuned WavLM-Large. Table 4 shows the risk: removing speaker conditioning drops WavLM SIM by 17.9 points but WeSpeaker SIM by only 8.0. Table 1 reports no independent SIM. (b) WER is not competitive: 2.04 on Seed-EN is worse than 9 of 13 baselines and 1.41 on Seed-ZH is worse than 9 of 11. (c) "0.2B" excludes the 127.9M VAE and the speaker encoder, while training uses 550k hours of internal data, several times more than Emilia-based baselines, and data overlap with test speakers is not discussed. (d) The subjective test is small (50 utterances, 7 raters, 400 pairs, no confidence intervals) and covers only quality and naturalness. Converted to counts, several "preferences" are not significant (for example, base vs Pocket TTS naturalness 8/32/10; distilled vs F5-TTS naturalness 14/21/15). There is no speaker-similarity MOS and no direct base-versus-distilled test, so "comparable subjective quality" is not shown. (e) Efficiency: the absolute gain is 11.4 ms, the base RTF (0.184) is worse than Pocket TTS (0.125) and F5-TTS (0.122), baselines use different engines (vLLM-Omni, TensorRT), VibeVoice-Realtime is run without voice cloning, and the speedup mixes removal of the unconditional LM pass with fewer head steps.
Questions for authors
- What is technically new in guidance-step distillation compared with IntMeanFlow with a CFG teacher, LCM one-stage guided distillation, MeanFlow with CFG, and CALM latent CFG distillation? Do you have any matched comparison showing that joint distillation beats sequential guidance-then-step distillation, or beats any of these methods at the same NFE?
- Can you report Table 1 SIM with an independent verifier (WeSpeaker, CAM++, or ECAPA) and add ground-truth rows? Is the WavLM SV teacher the same checkpoint as, or shares training data with, the SIM evaluator? Do the training data include LibriSpeech, LibriTTS, or Common Voice/DiDiSpeech speakers used in the test sets?
- Which numbers in Tables 1 and 2 are copied and which are re-evaluated? What inference settings were used for each baseline? Can the main baselines be timed with the same engine or with the same level of optimization?
- What is the full deployed parameter count and peak memory, including VAE and speaker encoder? Which text tokenizer is used, and does inference require the reference transcript?
- How much of the 11.4 ms gain comes from removing the unconditional LM pass versus reducing head steps? Please give a component-level breakdown and clarify "first PCM chunk on the CPU".
5 comments
Response to Reviewer 6Sf8 (2/5)
W2(a) & Q2. Speaker similarity, the main advantage, is suspicious. CuteTTS reports SIM 78.9 on LibriSpeech and 76.5 on Seed-EN, above every baseline and above commonly reported ground-truth levels for these protocols (about 0.69 for LibriSpeech-PC under the F5-TTS protocol, about 0.73 for Seed-EN human speech). Yet no ground-truth row is given. The speaker encoder is distilled from a WavLM-Large SV model and the SIM evaluator is a fine-tuned WavLM-Large. Table 4 shows the risk: removing speaker conditioning drops WavLM SIM by 17.9 points but WeSpeaker SIM by only 8.0. Table 1 reports no independent SIM. Question: Can you report Table 1 SIM with an independent verifier (WeSpeaker, CAM++, or ECAPA) and add ground-truth rows? Is the WavLM SV teacher the same checkpoint as, or shares training data with, the SIM evaluator? Do the training data include LibriSpeech, LibriTTS, or Common Voice/DiDiSpeech speakers used in the test sets?
Response:
Thank you for raising this important point.
- Shared WavLM checkpoint & independent WeSpeaker verifier: The WavLM-Large speaker-verification checkpoint used as the teacher for our speaker encoder is indeed the same checkpoint used for
SIM (WL)evaluation. To remove this dependency, we added WeSpeaker (SIM (WS)) as an independent verifier for all locally evaluated systems on LibriSpeech-PC, reported in Table 1 and Table R3. The additional Seed-TTSSIM (WS)results are provided below Table R3 in this response. Our cross-system similarity claims are grounded inSIM (WS). - Revised similarity positioning: Under WeSpeaker, CuteTTS achieves 86.4 on LibriSpeech (close to CosyVoice 3 at 87.7 and IndexTTS2 at 87.0, and higher than the remaining seven measured baselines) and 86.2 on Seed-TTS EN (vs. 88.1 for IndexTTS2 and 87.5 for Qwen3-TTS), while trailing further on Seed-TTS ZH (
88.9vs.92.9and92.2). Accordingly, we have revised the paper's claims to state that CuteTTS achieves strong speaker similarity close to leading systems on LibriSpeech and Seed-TTS EN despite its compact 0.2B generative backbone, rather than claiming the highest similarity across all sets. - Ground-truth rows & values exceeding ground truth: Table 1 now includes ground-truth rows from F5-TTS (
2.23 / 69on LibriSpeech-PC,2.06 / 73on Seed-TTS EN, and1.26 / 76on Seed-TTS ZH). Notably, several baselines also exceed ground-truthSIM (WL)(e.g., VoxCPM2 achieves 74.2 on LibriSpeech). BecauseSIM (WL)measures cosine similarity against the reference prompt rather than a separate enrollment utterance, closely matching the prompt's acoustic channel and speaking style can yield higher similarity than a distinct ground-truth utterance; we note this explicitly soSIM (WL)above ground truth is not over-interpreted. - Training-data overlap check: We checked our 550K-hour training corpus against LibriSpeech, LibriTTS, Common Voice, and DiDiSpeech at the utterance/file level and confirmed zero identical utterances. We did not verify speaker-level overlap, so we cannot guarantee speaker-level disjointness.
W2(b). WER is not competitive: 2.04 on Seed-EN is worse than 9 of 13 baselines and 1.41 on Seed-ZH is worse than 9 of 11.
Response:
We agree that CuteTTS does not achieve the lowest WER on Seed-TTS. Our intended claim is competitive intelligibility within a compact (0.2B backbone), low-latency streaming system—achieving 2.16% on LibriSpeech (lower than the WER of several larger systems, including VoxCPM2 2.81%, IndexTTS2 2.50%, and F5-TTS 2.42%), alongside 2.04% on Seed-EN and 1.41% on Seed-ZH, while maintaining 37.6–49.0 ms first-audio latency. We have tempered our wording in the Abstract and Sec. 3.2 to reflect this balance accurately.
Response to Reviewer 6Sf8 (3/5)
W2(c). "0.2B" excludes the 127.9M VAE and the speaker encoder, while training uses 550k hours of internal data, several times more than Emilia-based baselines, and data overlap with test speakers is not discussed.
Response:
As detailed in Q4, Appendix A.1 reports both the generative backbone size (228.6M / 231.8M) and the full deployed parameter count (382.7M / 385.8M). The utterance-level overlap check is described in our response to Q2. Sec. 4 reports the ~50K-hour controlled subset results (WER 2.70 / SIM (WL) 71.3 / SIM (WS) 82.5 on LibriSpeech), distinguishing smaller-scale controlled evidence from full-scale system benchmarks.
W2(d). The subjective test is small (50 utterances, 7 raters, 400 pairs, no confidence intervals) and covers only quality and naturalness. Converted to counts, several "preferences" are not significant (for example, base vs Pocket TTS naturalness 8/32/10; distilled vs F5-TTS naturalness 14/21/15). There is no speaker-similarity MOS and no direct base-versus-distilled test, so "comparable subjective quality" is not shown.
Response:
We have added the direct blinded base-vs-distilled comparison (Table R4 / Table 13) and report the exact win/tie/loss counts (out of 50 pairs per comparison) below:
| Comparison | Naturalness (Win / Tie / Loss) | Sound Quality (Win / Tie / Loss) |
|---|---|---|
| Base vs. MOSS-TTS-Nano | 23 / 21 / 6 () | 16 / 26 / 8 () |
| Base vs. F5-TTS | 16 / 25 / 9 () | 24 / 23 / 3 () |
| Base vs. ZipVoice | 20 / 21 / 9 () | 37 / 9 / 4 () |
| Base vs. Pocket TTS | 8 / 32 / 10 () | 17 / 22 / 11 () |
| Distilled vs. MOSS-TTS-Nano | 25 / 15 / 10 () | 16 / 24 / 10 () |
| Distilled vs. F5-TTS | 14 / 21 / 15 () | 21 / 21 / 8 () |
| Distilled vs. ZipVoice | 20 / 21 / 9 () | 32 / 15 / 3 () |
| Distilled vs. Pocket TTS | 9 / 25 / 16 () | 15 / 25 / 10 () |
Both CuteTTS variants receive more wins than losses against all four baselines in sound quality. Under the reported uncorrected tests, the sound-quality preferences are statistically significant against F5-TTS and ZipVoice, and the naturalness preference is significant against MOSS-TTS-Nano. Speaker similarity is evaluated objectively via both WavLM and WeSpeaker rather than via subjective MOS.
In the direct base-vs-distilled test (Table R4), 64% of pairs are tied in naturalness (13 / 32 / 5, ) and 76% in sound quality (8 / 38 / 4, ), supporting our characterization that the distilled model remains perceptually close to the base model with a possible slight loss in naturalness.
W2(e). Efficiency: the absolute gain is 11.4 ms, the base RTF (0.184) is worse than Pocket TTS (0.125) and F5-TTS (0.122), baselines use different engines (vLLM-Omni, TensorRT), VibeVoice-Realtime is run without voice cloning, and the speedup mixes removal of the unconditional LM pass with fewer head steps.
Response:
Our primary efficiency goal is low first-audio latency for interactive streaming, which differs from full-utterance throughput (RTF). In Table 2, CuteTTS-distill achieves the lowest mean first-audio latency among all evaluated systems (37.6 ms) with an RTF of 0.109 (faster than Pocket TTS at 0.125 and F5-TTS at 0.122, though slower in RTF than non-autoregressive ZipVoice at 0.030). We frame Table 2 as reflecting practical end-to-end latency–throughput trade-offs across released system implementations—and now explicitly annotate VibeVoice-Realtime's non-cloning default-voice setting directly in Table 2—while isolating our method's exact latency components in Table R5.
Response to Reviewer 6Sf8 (1/5)
Response:
We appreciate the detailed literature contextualization and have added citations and discussion of Latent Consistency Models (LCM) and
dots.ttsalongside IntMeanFlow in Sec. 2.4 of the revised manuscript.We do not claim individual algorithmic primitives—such as integral-velocity regression (IntMeanFlow), guidance-scale conditioning (LCM), or CFG distillation in continuous-AR TTS (CALM / Pocket TTS)—as standalone inventions. Rather, our contribution is their unified integration into a single-stage guidance–step distillation recipe for LM-level CFG in a patch-autoregressive system, backed by controlled ablations and end-to-end streaming latency measurements:
Guidance → stepdistillation (+0.2to+0.5SIM (WS) across all three step budgets on LibriSpeech-PC, with comparable WER and UTMOS) while avoiding a second training stage. It also avoids the 1-step collapse of guidance-only distillation (WER 3.00 vs. 41.99) and the 1–2 step degradation of step-only andStep → guidancevariants.