- Rating
- 2
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper presents DuplexTTS, a framework for synthesizing and screening full-duplex conversational speech through native generation and rule-based composition. The paper construct a bilingual speech corpus and assess its quality and utility through human evaluation and downstream full-duplex model training.
Critical strengths
-
Complementary synthesis routes with systematic quality control. Native and rule-based synthesis produce corpora with complementary downstream strengths in interruption response and continuation through non-interrupting speech. Final-audio screening and additional source-utterance checks strengthen this mixed-data pipeline by detecting missing or near-silent utterances that can escape dialogue-level checks.
-
Evidence of downstream utility supported by multi-level evaluation. The authors assess the generated data through interaction statistics and human listening, and further evaluate its utility through Moshi adaptation. Mixed-data training increases macro expected-behavior accuracy on Full-Duplex-Bench from 26.43% to 43.03%, providing evidence of practical value beyond perceptual quality alone.
Critical weaknesses
-
Insufficient comparison with existing data-construction approaches. The paper lacks both a structured comparison of its capabilities against related pipelines and a downstream comparison with Moshi fine-tuned on a suitable alternative full-duplex corpus. Improvement over the unadapted Moshi checkpoint alone cannot distinguish the value of this particular framework from the benefit of additional task-relevant training. Without these comparisons, the methodological differentiation and practical advantage of the proposed synthesis and curation choices remain insufficiently established.
-
The downstream comparisons confound data composition with data quantity and training budget. The mixed condition uses 1,425.61 hours of training data and is evaluated at update 9,100, whereas rule-only uses 167.97 hours and is evaluated at update 700. Consequently, the stronger mixed-data result cannot establish whether combining the two synthesis routes provides benefits beyond using more data and optimization. Matched-budget comparisons are needed to substantiate the incremental value of the mixed-data design.
Questions for authors
-
Would it be possible to add a table comparing DuplexTTS with Lychee-FD[1], DuplexGen[2], ConversationalVoice[3], DuplexChat[4], and SmoothConv & DuplexConv[5] in terms of data size, data format, construction methods, and quality checks?
-
How does Moshi trained on DuplexTTS compare with the same model trained on at least one suitable existing dataset, using the same data size and training budget? Alternatively, how does it compare with a suitable released model on the same benchmark?
-
What are the numbers and proportions of normal turns, backchannels, interruptions, and side talk in the rule-based training set?
-
Does mixed-data training still outperform rule-only training after applying the interruption-timing correction from Appendix B.3, when data size and training budget are matched?
[1] Liu Z, Zhang X, Li Y, et al. Hierarchical Acoustic-Semantic Modeling: Modality Separation and Semantic Coherence for Full-Duplex SLMs[C]//Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2026: 9264-9280. [2] Kim T, Kim K, Woo S H, et al. DuplexGen: Adaptive Synthesis of Human-AI Turn-Taking Dialogues[J]. arXiv preprint arXiv:2607.26178, 2026. [3] He R Y, Cao B, Xu C, et al. ConversationalVoice: Full-Duplex Speech Data from Real Conversations through Source-Faithful Reconstruction and Conversation-Grounded Expansion[J]. arXiv preprint arXiv:2609.08147, 2026. [4] Nakata W, Saito Y, Saruwatari H. DuplexChat: Constructing Speaker-Separated Full-Duplex Dialogue Speech at Scale for Spoken Dialogue Language Modeling[J]. arXiv preprint arXiv:2607.04941, 2026. [5] Wang C, Shao M, He C, et al. SmoothConv and DuplexConv: Complementary Mandarin Multi-Party Conversational Speech Corpora for Speech Interaction[J]. arXiv preprint arXiv:2610.11150, 2026.
7 comments
Supplementary tables for Q1–Q4 (inline)
Dear Reviewer u4Wh,
For the reviewer's convenience, we provide the tables supporting our earlier reply directly in this thread, each followed by a brief summary. All values match the revised PDF.
Q1 — the requested comparison (Table 1 of the revision). Sizes use each paper's own counting unit; n.r. = not reported. Columns: separate per-speaker channels, word timestamps, controlled event placement, source-utterance checks, dialogue screening, downstream full-duplex training evaluation. Source-verified notes for every cell are in Appendix E.6.
| Pipeline | Construction | Size (own unit) | Sep. | TS | Event | Src. | Screen | FD eval. |
|---|---|---|---|---|---|---|---|---|
| Lychee-FD [1] | agents + TTS | ~140K dial. | ✔️ | n.r. | ✔️ | – | ✔️ | ✔️ |
| DuplexGen [2] | TTS + sched. placement | 1,623 h | ✔️ | – | ✔️ | – | ✔️ | ✔️ |
| ConversationalVoice [3] | real-audio reconstr. | n.r. | ✔️ | ✔️ | – | ✔️ | ✔️ | – |
| DuplexChat [4] | podcast reconstruction | 415K h (URLs only) | ✔️ | – | – | – | ✔️ | n.r. |
| SmoothConv & DuplexConv [5] | spontaneous rec. | 2,100 h | ✔️ | ✔️ | – | – | ✔️ | – |
| DuplexTTS (ours) | native gen. + rule comp. | 1,835 h | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ |
Existing pipelines render scripted exchanges through TTS cascades or source real conversations, while DuplexTTS (ours) combines native generation, where a single speech model produces both interacting channels, with rule-based composition over verified source utterances.
Q2 — released-model comparison, per task (reference rows of Table 5).
| Task | Lychee-FD (released) | Original Moshi (re-eval.) | Matched DuplexTTS arms (range) |
|---|---|---|---|
| Interruption | 55.50 | 58.50 | 30.67–59.00 |
| Backchannel | 30.61 | 8.16 | 19.73–29.25 |
| Talking-to-other | 8.00 | 21.00 | 30.33–53.67 |
| Background | 14.00 | 10.00 | 24.00–49.00 |
| Macro | 27.03 | 24.42 | 33.81–42.65 |
Lychee-FD[1] improves over the original checkpoint mainly through backchannel continuation; every matched DuplexTTS arm reaches a higher macro. As noted earlier, Lychee-FD differs in architecture and training history, and its unknown judgments remain in the denominator.
Q3 — event composition of all rule training subsets (Table 21 of the revision). Unique retained source events; count and share of each row's total. Per-window occurrence counts shift each share by at most 2.4 points.
| Training subset | Normal | Backchannel | Interruption | Side talk | Total |
|---|---|---|---|---|---|
| Original rule-only | 32,691 (48.8%) | 13,083 (19.5%) | 11,431 (17.1%) | 9,746 (14.6%) | 66,951 |
| Rule part of original mixed | 97,364 (48.6%) | 39,348 (19.6%) | 34,371 (17.2%) | 29,338 (14.6%) | 200,421 |
| Matched rule-only, original | 22,622 (47.7%) | 10,740 (22.7%) | 7,134 (15.1%) | 6,886 (14.5%) | 47,382 |
| Matched rule-only, corrected | 22,711 (47.7%) | 10,834 (22.8%) | 7,162 (15.0%) | 6,899 (14.5%) | 47,606 |
| Rule part of matched mixed, original | 11,422 (47.9%) | 5,400 (22.6%) | 3,570 (15.0%) | 3,458 (14.5%) | 23,850 |
| Rule part of matched mixed, corrected | 11,460 (47.9%) | 5,441 (22.7%) | 3,586 (15.0%) | 3,462 (14.5%) | 23,949 |
The four event types keep stable shares across all rule subsets, so the matched arms train on the same event mixture as the original rule corpus.
Supplementary tables for Q1–Q4 (inline) (2)
Q4 (and W2) — the five prespecified contrasts (Table 20 of the revision). Percentage points; 95% case-bootstrap intervals; three paired seeds. The interaction row subtracts the two correction rows: a positive value means the timing correction costs rule-only training more than mixed training.
| Contrast | Equal-task macro | Interruption |
|---|---|---|
| Mixed − Rule (original timing) | −8.70 [−12.5, −4.8] | +20.33 [+14.3, +26.2] |
| Mixed − Rule (corrected timing) | −3.73 [−7.2, −0.4] | +24.50 [+19.2, +30.0] |
| Correction effect on Rule-only | −5.11 [−8.0, −2.2] | −8.00 [−12.7, −3.3] |
| Correction effect on Mixed | −0.14 [−2.8, +2.4] | −3.83 [−8.3, +0.7] |
| Correction × composition | +4.97 [+1.1, +8.7] | +4.17 [−2.0, +10.3] |
At matched budget, the macro advantage of mixing does not hold under either timing condition, while its interruption advantage persists and grows after the correction, which costs rule-only training more than mixed training.
We hope these tables are helpful, and we would be glad to provide further detail during the discussion period.
Best regards, The Authors
[1] Liu Z, Zhang X, Li Y, et al. Hierarchical Acoustic-Semantic Modeling: Modality Separation and Semantic Coherence for Full-Duplex SLMs[C]//Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2026: 9264-9280. [2] Kim T, Kim K, Woo S H, et al. DuplexGen: Adaptive Synthesis of Human-AI Turn-Taking Dialogues[J]. arXiv preprint arXiv:2607.26178, 2026. [3] He R Y, Cao B, Xu C, et al. ConversationalVoice: Full-Duplex Speech Data from Real Conversations through Source-Faithful Reconstruction and Conversation-Grounded Expansion[J]. arXiv preprint arXiv:2609.08147, 2026. [4] Nakata W, Saito Y, Saruwatari H. DuplexChat: Constructing Speaker-Separated Full-Duplex Dialogue Speech at Scale for Spoken Dialogue Language Modeling[J]. arXiv preprint arXiv:2607.04941, 2026. [5] Wang C, Shao M, He C, et al. SmoothConv and DuplexConv: Complementary Mandarin Multi-Party Conversational Speech Corpora for Speech Interaction[J]. arXiv preprint arXiv:2610.11150, 2026.
Response to Reviewer u4Wh
Dear Reviewer u4Wh,
Thank you for the detailed review, the five references, and the two concrete experimental requests — both now carried out.
W1 & Q1 (structured comparison). The revision adds the requested table as Table 1 (Section 2, p. 2), organized along the dimensions the review asked for: data size (each paper's own counting unit), data format, construction method, and quality checks, plus downstream training evaluation, for all five suggested pipelines and DuplexTTS. Every cell is verified against the primary papers and official releases (notes in Appendix E.6); unreported properties are marked n.r. The comparison shows where DuplexTTS differs: existing pipelines render scripted exchanges through TTS cascades or source real conversations, while DuplexTTS combines native generation, where a single speech model produces both interacting channels, with rule-based composition over verified source utterances. We discuss the downstream comparison under Q2.
W2 & Q4 (training budget, and mixing after the timing correction). We agree with this characterization of the original downstream comparison; the global response reports the matched-budget experiment, and the revision scopes the full-pool mixed result as recipe-level. The same experiment also addresses Q4. Timing correction reduces the rule-only macro score by 5.11 pp but has little effect on mixed training (−0.14 pp). After correction, mixed training still has a lower macro score than rule-only training (−3.73 pp [−7.2, −0.4]), but performs better on interruption response (+24.50 pp [+19.2, +30.0]). (Table 5, p. 9; contrasts in Table 20, Appendix E.1.)
Q2 (released-model comparison). Following the review's alternative, we evaluated the released Lychee-FD model on the identical 498-case protocol: it reaches 27.03 macro, versus 24.42 for the original checkpoint and 33.81–42.65 for every matched DuplexTTS-trained arm. Lychee-FD differs in architecture and training history, and its frequent unknown judgments (e.g., 68 of 98 backchannel cases) remain in the denominator. We also considered training the same backbone on an external corpus at equal budget: the DuplexGen release lacks word timestamps and would need a new alignment stage, which did not fit the discussion window. (Table 5, p. 9; Appendix E.2.)
Q3 (event composition). The revision reports counts over the actual training manifests: for the original rule-only subset, normal 48.8%, backchannel 19.5%, interruption 17.1%, side talk 14.6% of 66,951 unique events, with all six rule subsets in Appendix E.5 (Table 21); per-window occurrence counts shift each share by at most 2.4 points.
We hope these additions provide a useful basis for discussion, and we would be glad to provide further detail during the discussion period.
Best regards,
The Authors