- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This work proposes CaDiS, a framework that adapts pretrained offline diffusion speech models for streaming TTS and VC. The method combines block-causal adaptation, consistency distillation to reduce generation to four denoising steps, and distribution matching to recover quality. It also incorporates task-specific designs to support text–speech alignment and speaker consistency during streaming generation.
Critical strengths
- The proposed method is tested on two different tasks, TTS and VC. The ability to handle both tasks shows the practical value of adapting an offline diffusion model for streaming
- Human evaluation shows strong English voice conversion quality. Further assessment covers multiple languages, expressive anime voice synthesis, long-form audio generation, and formal listening studies.
Critical weaknesses
- The proposed method is a combination of a three-stage recipe from Casual Forcing, using RAPR from earlier work and keeping the offline teacher's normalized positional encoding. It is difficult to understand convolution beyond these previous methods.
- Given the architecture the main question is whether the speech-specific adaptatons add value.
- The RAPR ablation does not isolate the contribution of positional re-encoding.
- CaDiS shows competitive performance on different tasks; however, I feel the comparisons under matched operating conditions are missing. Although the authors discuss the trade-offs between quality and latency, CaDiS’s VC first-packet latency is 575 ms, while X-VC’s is 285 ms (Table 4). Further, the English WER improves from 3.41% to 3.09%, but speaker similarity drops from 0.748 to 0.670, while perceptual quality improves significantly (Table 2). These results show a useful trade-off. Still, comparing the systems at the same latency would better highlight the practical advantage.
Questions for authors
- Can you provide CI for the main listening study? It would also help distinguish clear improvements from small differences in mean scores.
5 comments
Response to Reviewer s5WV (2)
CW3: We are grateful to the reviewer for pointing this out. We agree that changing the history-window length does not isolate the effect of positional re-encoding: it also changes the acoustic context available to the model.
Our comparison keeps the speaker reference, retained-history length, chunk size, and sampling budget fixed, and changes only whether the retained history and current chunk are positionally re-anchored. Both settings use a 1 s history window, 400 ms generator/ContentVec chunks, and four sampling steps. The monotonic control also undergoes Stage-3 distribution matching refinement.
Short-form English VC.
| System | WER (%) ↓ | WeSpeaker SIM ↑ | UTMOS ↑ |
|---|---|---|---|
| Monotonic, 1 s history | 2.90 | 0.674 | 3.94 |
| RAPR, 1 s history | 2.83 | 0.675 | 3.93 |
The first table shows the short-form evaluation on SeedTTS-VC. The two settings have close performance on all metrics. The paired WER/SIM confidence intervals cross zero, so these results do not establish a short-form advantage or equivalence.
Long-form English VC: 20 paired sources, mean duration 124.2 s, without sentence resets. Details in Section 4.1 and Appendix.
| System | Full-clip WER (%) ↓ | Early target SIM ↑ | Late target SIM ↑ | Late − early SIM |
|---|---|---|---|---|
| Monotonic, 1 s history | 15.39 | 0.619 | 0.125 | −0.494 |
| RAPR, 1 s history | 9.18 | 0.699 | 0.696 | −0.003 |
However, the long-form evaluation in the second table, evaluated on 20 VC sources, shows that without re-anchoring, target-speaker similarity drops much more quickly than with RAPR, even after Stage-3 refinement training using the monotonic-position scheme. These experiments support the use of RAPR's bounded positional re-anchoring for long-form VC. Early/late SIM is the mean WeSpeaker similarity over non-overlapping 10-s windows whose centers fall within the first/last 30 s of each clip.
CW4: We agree that the submitted results show a useful trade-off rather than uniform superiority. Thank you for highlighting this. Although our model is trained on fixed 400 ms chunk size, it is capable of streaming in variable chunk sizes. We have used our streaming VC checkpoint with smaller generator and ContentVec chunks to approach the latency of X-VC and other baseline systems.
We evaluate 400 ms, 200 ms, and 100 ms chunks using the same checkpoint, four steps, and 1 s RAPR history. The results are shown below.
| Generator chunk size | WER (%) ↓ | WeSpeaker SIM ↑ | UTMOS ↑ | FPL (ms) ↓ | RTF ↓ |
|---|---|---|---|---|---|
| 100 ms | 3.42 | 0.671 | 3.72 | 272 | 1.10 |
| 200 ms | 3.27 | 0.675 | 3.85 | 373 | 0.59 |
| 400 ms | 2.85 | 0.675 | 3.93 | 576 | 0.33 |
For 100 ms chunks, the FPL approaches X-VC's 285 ms result, but the RTF increases above 1, so this setting does not establish sustained real-time conversion at matched latency. Different chunk sizes show a quality–latency trade-off: smaller chunks reduce FPL, while WER and UTMOS worsen and speaker similarity changes relatively little. For practical use, the 200 ms setting offers lower startup latency with RTF < 1, at some quality cost. We thank the reviewer for suggesting this more practically meaningful comparison.
Response to Reviewer s5WV (3)
Q1: We agree that confidence intervals, particularly for between-system differences, are needed to distinguish clear improvements from small differences in mean scores. We now provide 95% CIs for the main English listening-study means and paired differences, using 20,000 assignment-cluster bootstrap resamples.
For VC versus X-VC, the paired QMOS difference is +0.8869 [0.7143, 1.0625], while the SMOS difference is −0.0603 [−0.2270, 0.1034], from our listening study data. These results support a meaningful quality–similarity trade-off: the QMOS interval is entirely above zero, while the paired SMOS difference is not resolved at the reported pointwise 95% level.
The tables below provide the main English listening-study means and 95% confidence intervals, keeping the reported means unchanged.
Main Table 1: TTS means and 95% CIs
| System | NMOS [95% CI] | QMOS [95% CI] | SMOS [95% CI] |
|---|---|---|---|
| F5-TTS | 3.78 [3.67, 3.90] | 4.01 [3.89, 4.12] | 3.79 [3.67, 3.92] |
| IndexTTS-2.5 | 3.62 [3.50, 3.73] | 3.80 [3.68, 3.92] | 3.68 [3.54, 3.82] |
| S1-mini | 3.65 [3.51, 3.77] | 4.15 [4.04, 4.26] | 3.34 [3.19, 3.49] |
| Qwen3-TTS | 3.91 [3.80, 4.01] | 4.09 [3.98, 4.19] | 3.69 [3.57, 3.81] |
| VoXtream2 | 3.62 [3.50, 3.73] | 3.92 [3.81, 4.03] | 3.78 [3.65, 3.91] |
| CosyVoice3 | 3.51 [3.38, 3.64] | 4.00 [3.89, 4.12] | 3.68 [3.54, 3.82] |
| Faster-IndexTTS-2 | 3.75 [3.64, 3.86] | 3.82 [3.70, 3.95] | 3.80 [3.67, 3.93] |
| Ours offline | 3.90 [3.80, 4.01] | 4.17 [4.06, 4.27] | 3.85 [3.74, 3.97] |
| CaDiS TTS (1000-step) | 3.99 [3.86, 4.12] | 4.17 [4.04, 4.29] | 3.79 [3.65, 3.94] |
Main Table 2: VC means and 95% CIs
| System | NMOS [95% CI] | QMOS [95% CI] | SMOS [95% CI] | Rhy [95% CI] | Exp [95% CI] |
|---|---|---|---|---|---|
| Vevo | 3.96 [3.82, 4.10] | 3.95 [3.82, 4.08] | 3.67 [3.52, 3.82] | 3.95 [3.78, 4.11] | 3.71 [3.56, 3.86] |
| Seed-VC | 3.32 [3.16, 3.47] | 2.99 [2.80, 3.17] | 2.90 [2.74, 3.06] | 3.65 [3.48, 3.83] | 3.43 [3.25, 3.61] |
| X-VC | 3.40 [3.23, 3.57] | 2.84 [2.66, 3.02] | 3.16 [3.01, 3.32] | 3.85 [3.67, 4.02] | 3.64 [3.47, 3.80] |
| MeanVC-2 | 3.58 [3.44, 3.72] | 3.26 [3.08, 3.43] | 2.37 [2.22, 2.52] | 3.73 [3.55, 3.90] | 3.52 [3.36, 3.69] |
| Ours offline | 3.92 [3.78, 4.06] | 3.87 [3.73, 4.01] | 3.01 [2.85, 3.17] | 3.95 [3.78, 4.12] | 3.81 [3.66, 3.95] |
| CaDiS VC | 3.79 [3.65, 3.93] | 3.73 [3.58, 3.87] | 3.10 [2.95, 3.26] | 4.00 [3.84, 4.15] | 3.82 [3.67, 3.96] |
We sincerely thank the reviewer for providing a comprehensive review of our paper. We have added all the experiments discussed in this rebuttal to the Appendix of the paper, with blue text color showing the revision from submission version. We will consider moving some of the experiments to the main content upon acceptance.
Response to Reviewer s5WV
We thank the reviewer for the thoughtful and constructive review, and for recognizing the practical value of studying both TTS and VC and the quality of our English VC results. We agree that the distinction from prior work, the evidence for the speech-specific designs, and the quality–latency comparison should be made clearer. We address these points below.
CW1: Thank you for pointing this out. We agree that the three-stage recipe and RAPR build on prior work, and that the TTS positional encoding is retained from the offline teacher rather than introduced as a new encoding.
We want to emphasize that our contribution is adapting the general distillation framework Causal Forcing++ with speech-specific design and its supporting experimental results, rather than a new general-purpose distillation algorithm. We study this conversion for two different conditioning structures: asynchronous text–speech alignment in TTS and frame-synchronous content conditioning in VC.
For long-form VC, the RAPR configuration provides a concrete example of this adaptation. The original StreamAvatar paper uses a small reference sink with a relatively long history window in order to achieve better long-form generation capabilities, we use latent frames from a 10-second speaker reference together with a short rolling history to enable consistent performance in streaming generation. This configuration retains a target-speaker anchor while limiting the amount of generated acoustic history carried forward during streaming.
We have revised these sections in the paper accordingly. We appreciate the reviewer's suggestion to separate these inherited components from the speech-specific adaptation and its supporting evidence more explicitly.
CW2: We agree that the speech-specific design choices should be supported by targeted experiments, rather than only by the performance of the complete system. The stage-wise results in Table 5 characterize the overall conversion pipeline, but do not by themselves establish the contribution of every task-specific choice.
For TTS, we compared retaining the offline teacher’s normalized positional encoding with an alternative design using RoPE and explicit sentence-level length conditioning. We trained an additional streaming TTS model using the same three-stage pipeline, replacing normalized positional encoding with RoPE and providing the total number of acoustic latent frames (
num_frames) through adaptive layer normalization. During training, this number is derived from the training audio; during inference, it is provided by the duration estimator. The results on SeedTTS-eval English are shown below.num_framesCaDiS achieves lower WER, slightly higher speaker similarity, and slightly lower FPL and RTF, whereas the RoPE variant achieves higher UTMOS. These results show a trade-off rather than uniform superiority of either design. They suggest that, for full-text-ready streaming TTS, explicitly conditioning the model on the total sequence length is a viable alternative to incorporating this information through normalized positional encoding. We have therefore clarified that normalized positional encoding is retained from the offline teacher, rather than introduced as a new encoding or claimed to be necessary or uniformly superior to RoPE.