- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
Continuo is a multilingual resource for controllable long-form speech generation, built from 229,388 hours of source-level-deduplicated audio across 41 languages. Its pipeline creates overlapping short-utterance, single-speaker long-form, and multi-speaker dialogue views while retaining continuous source audio, pauses, and speaker-turn structure. Subsets contain 4.77 million instruction-annotated examples and 933 thousand examples with nonverbal-vocalization tags. The paper also introduces a human-verified Chinese/English testset with 118 recordings totaling 23.3 hours, durations up to 30 minutes, and reference audio for up to four speakers. Autoregressive and masked non-autoregressive models with 0.6B sequence backbones demonstrate utility across long-form synthesis, instruction following, and nonverbal generation. Expressive-control evaluations use shorter-than-five-minute subsets, separately from the full-duration synthesis tests.
Critical strengths
-
A substantial and useful data contribution with a well-motivated long-form representation. Preserving continuous audio, original pauses, and speaker transitions addresses information that independent short utterances lose. The corpus combines substantial long-duration coverage with multilingual, dialogue, instruction, and nonverbal supervision. The paper clearly distinguishes the 229K unique source hours from the 398K hours summed across overlapping views, and does not imply that all examples have expressive annotations. The accompanying 118-case benchmark adds a useful combination of long recordings, up to four speakers, and per-speaker references. Its source/duplicate-group isolation and human verification are explicitly described. The linked testset is publicly accessible with audio and structured fields, and the processing repository is available. These are tangible resources; the full training-manifest release is still planned for acceptance (Appendix H).
-
Broad downstream validation supports the resource's practical value. Evaluating both AR and NAR architectures is more convincing than demonstrating only one training recipe. Continuo-NAR achieves 5.27% Chinese CER and 9.93% English WER on TTSD-Eval, while Continuo-AR obtains competitive short-form instruction-following results. The paper also reports weaknesses, including NAR degradation at longer durations, rather than suggesting universal superiority. The whole-sequence versus chunked comparison is particularly informative: content error and perceived naturalness can move in opposite directions (Appendix G.1/Table 20). This helps explain why long-form evaluation needs perceptual continuity checks alongside transcription and speaker metrics.
Critical weaknesses
-
End-to-end annotation quality on the retained training corpus remains insufficiently quantified. Appendix B provides useful component benchmarks, a language-routing audit, and confidence-gating diagnostics, but these do not directly estimate the final corpus's joint transcription, speaker-assignment, temporal-style, and NV annotation errors. For example, the gated emotion recognizer still has WF1 of 0.66-0.77 on the reported validation sets, and captions for recordings longer than 30 seconds are generated from extracted attributes without listening to the waveform. These are reasonable engineering choices, but their downstream annotation fidelity needs a representative audit of the actual retained examples, including longer dialogues and less-represented languages. Human verification of the 118 evaluation cases does not establish the quality of the training annotations. This uncertainty limits how confidently others can use the expressive labels and interpret the dataset's coverage claims.
-
The proposed benchmark's speaker scores need explicit coverage accounting. Appendix F.2 obtains speaker-scored segments through ASR/script matching, requiring at least 50% exact anchors for forced alignment and excluding segments shorter than 0.5 seconds. Equation (4) then averages SIM and ACC over the resulting segments. As written, this can make speaker performance conditional on successful content alignment: omitted, badly transcribed, or otherwise unalignable portions may not contribute to the speaker score. Evaluating every case does not by itself ensure that comparable portions of each system's output are scored. The paper should report the scored fraction of audio/script/turns for every system and explain how empty or failed alignments are handled. This is a clarification and validation concern, not evidence that the reported rankings are incorrect.
On balance, I lean toward weak acceptance. The scale, preservation of long-form structure, released evaluation resource, and validation under two modeling paradigms constitute a useful contribution. The concerns above constrain the strength of the quality and speaker-consistency conclusions, but appear addressable with targeted audits and fuller evaluation reporting.
Post-rebuttal update (19 Nov 2026):
The annotation audit, speaker-score coverage tables and manual-alignment check substantially address my main concerns, and the added training schedule improves reproducibility. I am keeping my rating at 3 (weak acceptance). The final-data audit is limited to Chinese and English, and the expressive labels still contain noticeable errors; these limitations should remain explicit. The remaining reporting details do not change my overall assessment. This is my final recommendation after the rebuttal, and I have no further questions.
Questions for authors
-
Final-data quality: Can you report a stratified manual audit of the retained training examples, covering transcription accuracy, speaker attribution, temporal instruction fidelity, and NV event type/placement? Please separate the final-data results from component benchmarks, and include longer dialogues and languages beyond English/Chinese where the corresponding annotations are available.
-
Speaker-score coverage: What proportions of generated duration, script tokens, and speaker turns enter SIM/ACC for each system? How are slices below the alignment threshold and cases with no valid segments handled? Does the comparison remain similar when coverage is reported alongside the scores and checked on a manually aligned subset?
-
Training reproducibility: Please specify the actual optimizer steps or epochs and effective audio/token exposure at each curriculum stage, together with the sampling ratios across overlapping views and annotated subsets. Appendix E gives learning rates and duration/token limits, but these additional quantities are needed to reproduce the data exposure and assess how the three views are used. Please identify the exact versioned manifests/configurations corresponding to the reported runs.
6 comments
Response to Reviewer 9P4X (Part 1/3)
Dear Reviewer 9P4X,
Thank you for the positive assessment and the precise, actionable questions. We ran each analysis you proposed and uploaded a revised manuscript, with all revisions marked in blue.
(1) Final-data quality audit (Response to Weakness 1 & Question 1)
“Question 1: Can you report a stratified manual audit of the retained training examples, covering transcription accuracy, speaker attribution, temporal instruction fidelity, and NV event type/placement? Please separate the final-data results from component benchmarks …”
Please refer to Table 4 (Line 369–384, Pages 7–8) and Appendix B.6, Tables 25–26 (Line 1268–1292, Page 24).
We agree that component benchmarks do not measure label quality on Continuo itself, and add a human audit:
- Transcription accuracy and speaker attribution. The testset pool (50.2 h) was drawn from the collected corpus, labeled by the same pipeline as the training data, and then held out. We compare the unedited pipeline labels with the annotators' corrected labels over the full pool, so the comparison estimates training-label quality. In Chinese / English, it holds 262 / 175 short segments, 17 / 24 long-form recordings and 90 / 58 dialogues (Table R1).
- Style instructions. We randomly sampled 300 training instructions (150 Chinese, 150 English audio). Ten annotators each checked 30 of them. An instruction receives credit only if every stated attribute, including emotion and each time interval, matches the audio. One wrong interval in a minutes-long description therefore fails the whole instruction, so the proportions in Table R2 are a conservative lower bound.
- Nonverbal vocalization (NV) tags. We audited 300 training items (150 Chinese, 150 English audio). Ten annotators each checked 30 of them. We compute NV accuracy for each item and average over items (Table R3).
Table R1. Transcripts and speaker labels of 50.2h annotation pools.
| Chinese CER ↓ | English WER ↓ | Chinese speaker acc. ↑ | English speaker acc. ↑ | |
|---|---|---|---|---|
| Short | 0.99% | 1.48% | 97.30% | 97.30% |
| Long-form | 1.12% | 0.94% | 100% | 100% |
| Dialogue | 1.52% | 1.11% | 94.40% | 93.40% |
Table R2. Human audit of 300 randomly sampled training instructions.
| Chinese style acc. ↑ | English style acc. ↑ | |
|---|---|---|
| Short | 89.60% | 90.40% |
| Long-form | 83.60% | 81.20% |
| Dialogue | 78.80% | 78.20% |
Table R3. Mean per-item NV label accuracy in a human audit of 300 training items.
| Training-data subset | Chinese NV accuracy ↑ | English NV accuracy ↑ |
|---|---|---|
| Short-form | 85.30% | 86.70% |
| Long-form | 80.60% | 82.70% |
| Dialogue | 79.70% | 74.30% |
(2) Languages beyond English and Chinese (Response to Question 1)
“… and include longer dialogues and languages beyond English/Chinese where the corresponding annotations are available.”
Please refer to Appendix B.2, Table 17 (Line 1120–1133, Page 21) and Appendix I (Line 2042–2045, Page 38).
Style and NV annotations exist only for Chinese and English, so those two audits cover every annotated language. For transcripts, we lack qualified annotators in other languages and cannot offer a final-data audit. We state this in Limitations.
As a component benchmark, Table R4 gives the error rates of our transcription model, Qwen3-ASR-1.7B, on the four largest of these languages, which together hold 72.1% of non-Chinese/English hours. It is on par with Whisper-large-v3 on FLEURS, each being better on two languages, and clearly better on Common Voice for Japanese and Korean. Both benchmarks are read speech, so these numbers do not measure accuracy on in-the-wild audio.
Table R4. ASR error rates (%) for four additional languages.
| Benchmark | System | de | ja | ko | es |
|---|---|---|---|---|---|
| FLEURS | Qwen3-ASR-1.7B (ours) | 3.92 | 5.20 | 2.57 | 3.36 |
| FLEURS | Whisper-large-v3 | 4.30 | 4.80 | 2.60 | 3.12 |
| Common Voice | Qwen3-ASR-1.7B (ours) | 5.85 | 11.64 | 5.88 | 4.65 |
| Common Voice | Whisper-large-v3 | – | 15.20 | 11.90 | – |
Response to Reviewer 9P4X (Part 2/3)
(3) Speaker-score coverage (Response to Weakness 2 & Question 2)
“Question 2: What proportions of generated duration, script tokens, and speaker turns enter SIM/ACC for each system? How are slices below the alignment threshold and cases with no valid segments handled? Does the comparison remain similar when coverage is reported alongside the scores and checked on a manually aligned subset?”
Please refer to Appendix F.2, Tables 32–34 (Line 1708–1762, Pages 32–33).
(i) Coverage. Table R5 gives the coverage on CONTINUO-testset, now also in Appendix F.2.
Table R5. Scoring coverage of the SIM/ACC protocol on CONTINUO-testset (Chinese / English).
| System | Duration (%) | Script tokens (%) | Turns (%) | ACC ↑ |
|---|---|---|---|---|
| GT | 82.0 / 77.0 | 91.3 / 88.2 | 83.2 / 82.9 | 99.33 / 98.38 |
| Higgs Audio v2 | 21.6 / 34.6 | 3.4 / 3.9 | 4.7 / 5.0 | 75.00 / 72.84 |
| VibeVoice | 80.1 / 78.5 | 87.7 / 85.7 | 83.1 / 81.4 | 93.78 / 95.98 |
| MOSS-TTSD | 73.2 / 69.9 | 91.9 / 82.7 | 83.8 / 72.0 | 99.28 / 99.22 |
| Continuo-AR | 70.5 / 74.6 | 92.3 / 90.7 | 84.7 / 84.0 | 98.57 / 95.05 |
| Continuo-NAR | 71.3 / 72.5 | 78.1 / 80.2 | 82.1 / 79.6 | 97.38 / 96.46 |
- The protocol has a ceiling. Even ground-truth recordings reach only 77–82% of duration, 88–91% of script tokens and 83% of turns, so coverage should be read against GT.
- Most strong systems are scored on comparable portions. VibeVoice, MOSS-TTSD and Continuo-AR are within 4 points of GT on tokens and turns, with one exception: in English, MOSS-TTSD covers 82.7% of tokens and 72.0% of turns. Continuo-NAR covers fewer tokens (78–80%) at near-GT turn coverage, consistent with its higher error rates on this set.
- Very low coverage marks failed generation. Higgs Audio v2 has only 3–5% of tokens and turns scored, matching its 75–81% CER/WER, and it ranks last under either accounting.
(ii) Unscored portions. We do not impute unscored outputs; they are counted as uncovered. Excluding Higgs Audio v2, this happens for three reasons: chunks with under 50% exact ASR matches are not force-aligned (0.0–7.7% of script tokens), segments shorter than 0.5 s are excluded from speaker scoring (3.4–9.3%), and some tokens receive no time alignment (2.0–8.7%). A case with no valid segments at all is excluded from the SIM and ACC averages.
(iii) Manual alignment. We randomly selected 10 Chinese and 10 English cases, aligned every system's output manually and recomputed SIM and ACC. Each cell gives automatic / manual alignment.
Table R6. SIM and ACC under automatic / manual alignment on CONTINUO-testset.
| System | Chinese SIM ↑ | Chinese ACC ↑ | English SIM ↑ | English ACC ↑ |
|---|---|---|---|---|
| GT | 0.865 / 0.866 | 98.9 / 98.4 | 0.850 / 0.850 | 99.4 / 99.5 |
| Higgs Audio v2 | 0.736 / 0.728 | 69.6 / 70.0 | 0.668 / 0.652 | 70.8 / 65.5 |
| VibeVoice | 0.796 / 0.793 | 91.8 / 91.5 | 0.736 / 0.734 | 89.6 / 89.5 |
| MOSS-TTSD | 0.861 / 0.859 | 99.1 / 99.2 | 0.818 / 0.817 | 99.8 / 99.6 |
| Continuo-AR | 0.830 / 0.830 | 98.6 / 98.7 | 0.735 / 0.732 | 93.0 / 91.8 |
| Continuo-NAR | 0.831 / 0.831 | 96.8 / 96.3 | 0.783 / 0.782 | 92.3 / 92.5 |
Except for Higgs Audio v2, which has few scored segments, manual alignment changes SIM by at most 0.003 and ACC by at most 1.2 points. Among the synthesis systems, both Chinese rankings and the English SIM ranking are identical under the two alignments. In English, only Continuo-AR and Continuo-NAR swap ACC order, with a gap below 1 point. Automatic alignment therefore gives the same comparison as manual alignment.
Note on AI tool use during my review
Following the PCs' request, I'm adding a note about how AI tools were used during my reviewing process. I discussed this paper with GPT-6-Astra while working on my review. I used it as a sounding board: I shared my own reading notes — mainly the gap between the component-level benchmarks in Appendix B and the end-to-end annotation quality of the retained training corpus, and the speaker-score coverage accounting around Eq. (4) / Appendix F.2 — and asked it to check whether my criticisms were technically sound and whether I had overlooked obvious counterarguments. It also helped me polish the wording of the review text. My assessment of the paper, the rating, and all specific claims in the review are my own: I verified every cited detail (the corpus statistics, the WF1 ranges, the alignment thresholds, the released testset and repository links, etc.) directly against the manuscript. The AI discussion helped me articulate the coverage-accounting concern more precisely, but it did not change my overall assessment. My questions for the authors in the review stand as written, and I look forward to their responses.