- Rating
- 3
- Confidence
- 2(−1)
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes Bridge-Duplex, a dual-stream full-duplex S2S framework that couples a persistent semantic stream with a frame-wise generative stream via selective bridge attention and training-time next-latent prediction, significantly improving multi-turn semantic consistency efficiently.
Critical strengths
The motivation is clear and well supported by the preliminary K/V pruning experiment, which shows that the semantic and generative streams have different context dependencies. The experimental evaluation is quite thorough, testing the approach across two different backbones (F-Actor and a LoRA-adapted Moshi) and including a 3B baseline to account for model size. The performance gains on MTR-DB and Behavior-SD are solid, especially regarding dialogue quality and multi-turn consistency.
Critical weaknesses
While the framework excels in full-duplex conversational interaction, the evaluation is largely confined to dialogue chit-chat datasets. It lacks validation on broader, general-purpose speech LLM benchmarks to demonstrate whether the base model's general reasoning and factual knowledge capabilities are well-preserved under this full-duplex formulation.
Questions for authors
Overall, I have no other major concerns with the paper. My only question is whether the authors could consider evaluating on general speech language model benchmarks such as VoiceBench as well as speech understanding benchmarks such as MMAU. The method clearly performs well in full duplex settings, but it would be good to verify that performance on standard reasoning and general speech understanding does not degrade.
5 comments
Author Response 2: Detailed Response
Comment 1 (W1, Q1): General Speech Evaluation beyond Dialogue Benchmarks
We have added evaluations of spoken reasoning with Big Bench Audio (BBA) [R1], knowledge-based and open-ended question answering with selected VoiceBench tasks [R2], and speech understanding with MMAU-Speech [R3]. Table R1 summarizes these complementary evaluations.
We include both pretrained Moshi and a Moshi baseline fine-tuned on conversational speech data to distinguish pretrained capability retention from performance following conversational adaptation. Text-only models serve as reference points rather than modality-matched baselines. For MMAU-Speech, we synthesize textual prompts using TTS for speech-input models, while text-only LLMs receive ASR transcriptions of the audio alongside the original text prompts. Unlike our dialogue benchmarks, MMAU-Speech focuses on fine-grained speech understanding and reasoning tasks that are less aligned with our conversational speech-focused adaptation.
Table R1. Results on BBA, selected VoiceBench tasks, and MMAU-Speech. Higher scores are better within each column. BBA categories, OpenBookQA, and MMAU-Speech report accuracy (%); AlpacaEval and CommonEval report LLM-judged response-quality scores on a 1–5 scale.
| Model | Input | Formal Fallacies (BBA) | Navigate (BBA) | Object Counting (BBA) | Web of Lies (BBA) | OpenBookQA (VoiceBench) | AlpacaEval (VoiceBench) | CommonEval (VoiceBench) | MMAU-Speech (MMAU) |
|---|---|---|---|---|---|---|---|---|---|
| Helium-6B | Text | 25.6 % | 51.6 % | 28.4 % | 15.2 % | 41.10 % | 3.21 | 1.74 | 26.7 % |
| Llama-3.2-1B-Instruct | Text | 26.4 % | 30.0 % | 35.6 % | 20.8 % | 53.19 % | 3.91 | 2.79 | 32.7 % |
| Moshi (Pretrained) | Audio | 40.0 % | 12.0 % | 15.2 % | 28.4 % | 17.36 % | 3.07 | 1.34 | 0.00 % |
| Moshi (Conversation-adapted) | Audio | 0.0 % | 3.6 % | 0.8 % | 0.4 % | 5.49 % | 2.02 | 1.09 | 3.60 % |
| Selective Bridge-Duplex (Moshi) | Audio | 18.4 % | 19.2 % | 15.6 % | 4.0 % | 18.2 % | 2.06 | 1.74 | 6.91 % |
| F-Actor (Conversation-adapted) | Audio | 0.0 % | 2.4 % | 0.0 % | 1.6 % | 0.00 % | 1.39 | 1.01 | 1.50 % |
| Selective Bridge-Duplex (F-Actor) | Audio | 2.0 % | 12.4 % | 0.8 % | 12.4 % | 12.8 % | 2.81 | 1.85 | 3.60 % |
As shown in Table R1, Bridge-Duplex achieves higher scores than the corresponding conversation-adapted baselines on the reported tasks across both backbones. However, conversational fine-tuning of the Moshi baseline is also associated with reduced general reasoning and question-answering performance on several evaluated tasks. To contextualize this pattern, Table R2 summarizes the adaptation data, multi-turn semantic performance, and average BBA accuracy of the Moshi-based models.
Table R2. Training-data composition and conversational and spoken-reasoning performance of Moshi-based models.
| Model | Task-specific adaptation data | DQ / CC ↑ (Behavior-SD) | DQ / CC ↑ (MTR-DB) | BBA Avg. Accuracy ↑ |
|---|---|---|---|---|
| Moshi (Pretrained) | — (no additional adaptation) | 2.91 / 3.49 | 1.99 / 2.64 | 23.90 % |
| Moshi (Conversation-adapted) | Behavior-SD (conversational speech) | 3.72 / 4.40 | 2.07 / 2.60 | 1.20 % |
| Bridge-Duplex (Moshi backbone) | Behavior-SD (conversational speech) | 3.82 / 4.59 | 3.01 / 4.34 | 14.30 % |
Taken together, Tables R1 and R2 show that conversational adaptation improves the Moshi baseline’s conversational performance while reducing its performance on several general reasoning and question-answering tasks. Under the same adaptation data, Bridge-Duplex achieves stronger results than this adapted baseline in both multi-turn dialogue and the reported general-capability evaluations. However, the remaining gap to pretrained Moshi on BBA indicates that these improvements do not imply uniform preservation of pretrained capabilities. These findings motivate future work on conversational adaptation strategies that better retain pretrained reasoning and question-answering capabilities while improving multi-turn dialogue.
[R1] M. Hill-Smith and G. Cameron, “Evaluating Audio Reasoning with Big Bench Audio,” Artificial Analysis Blog, 2024. (https://artificialanalysis.ai/methodology/speech-to-speech-benchmarking#speech-reasoning)
[R2] Chen, Yiming, et al. "Voicebench: Benchmarking llm-based voice assistants." Transactions of the Association for Computational Linguistics 14 (2026): 378-398.
[R3] Sakshi, Sakshi, et al. "Mmau: A massive multi-task audio understanding and reasoning benchmark." International Conference on Learning Representations. Vol. 2025. 2025.
We would like to follow up on our rebuttal
Dear Reviewer kUqX,
We sincerely thank you once again for your valuable time and effort in reviewing our manuscript.
We deeply appreciate your insightful comments and we have worked to address your concerns through additional experiments and clarifications. Following your suggestion, we have provided additional evaluations on Big Bench Audio, selected VoiceBench tasks, and an adapted MMAU-Speech setting in our rebuttal.
We hope these comments and improvements can be taken into account when deciding on the final assessment for paper. We remain open to further discussion and are happy to provide additional clarifications if needed.
Thank you once again for your constructive feedback and the opportunity to improve our work.
sincerely,
The Authors.
Author Response 1: Summary
We thank the reviewer for the positive assessment and valuable suggestion. To address the concern about general speech capabilities beyond conversational dialogue, we provide additional evidence on the following aspect:
We provide the evaluation details and quantitative results below.