- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper decouples the streaming audio-visual interaction model from the asynchronous reasoning and tool-use layers. A Writer-Director pipeline was employed to construct timestamped training dialogues, encompassing scenarios such as interruptions and latency in reasoning feedback. Experimental results demonstrate the model's superior performance in turn-taking and conversational reasoning, while ablation studies further examine the contribution and substitutability of the reasoning layer.
Critical strengths
- DuplexOmni achieves 72.6% ToR on Full DuplexBench, compared with 36.3% for the strongest listed baseline, with 0.506 s response latency. The interaction-only variant retains 65.2%, supporting the value of the trained interaction model independently of the external reasoner.
- The Writer-Director pipeline provides a concrete approach to constructing continuous-interaction data. Its explicit representation of interruptions, shared silence, reasoning requests, and delayed feedback addresses an important training challenge for asynchronous voice systems.
Critical weaknesses
The central claim of effective reasoning and tool coordination during evolving interaction is not directly evaluated. Sections 4.4–4.5 rely on separate turn-taking and spoken-question benchmarks, while appendix contains training examples rather than held-out coordination results. First-audio-packet latency also does not measure useful task completion. A focused evaluation of interrupted reasoning or tool tasks could resolve this concern during rebuttal if suitable results already exist.
Questions for authors
Please report whether the system correctly incorporates changed user conditions during background reasoning or tool execution, including final-task correctness and time to a useful answer.
Summary of Discussion and Revisions
We thank the AC and the three reviewers for their time and constructive feedback. We are glad that, during the discussion, all three reviewers indicated that their main concerns have been addressed. Below we summarize the main results added during the rebuttal and where the changes appear in the revised PDF.
To assess coordination between the interaction layer and the thinking layer on held-out data (AkUf, nV7n, x2rH), we evaluated DuplexOmni on spoken GSM8K across 400 English and 400 Chinese test problems, excluding those that appear verbatim in our released training data. While the thinking layer works, the user changes a number or adds a missing condition. DuplexOmni reaches 85.0–92.2% final accuracy when the thinking layer gets the exact text of what the user said, and 77.6–87.3% when it gets only the speech. At most 1.8% of final answers keep the outdated value, and the correct answer is spoken a median of 8.7–11.8 s after the user stops speaking.
We also ran controlled comparisons with the same backbone, trained weights, and thinking layer (nV7n). Removing cancel/restart lowers accuracy in all eight cells by 6.9–26.1 points. With the thinking layer getting only the speech, single injection of the thinking result reduces accuracy in all four cells by 4.1–10.5 points. When thinking results arrive late and in pieces, DuplexOmni is 1.1–26.8 points more accurate than single injection and speaks the correct answer 3.5–7.6 s earlier.
On tool use and continuous interaction (AkUf, x2rH), the interaction layer relays results that must be spoken verbatim with 96.5% (Chinese) / 95.3% (English) field-level accuracy, and 95–96% of interjections during thinking are answered within 2 s.
Turning to Daily-Omni (AkUf), evaluating under the same streaming conditions brings DuplexOmni's base model down from 70.9% offline to 55.9%, and MiniCPM-o 4.5 from 80.2% (reported offline) to 60.2% in its official full-duplex mode. Time slicing alone costs the base model 1.6 points.
Revisions in the PDF
Table 1 caption and Section 4.1: the setting of each Daily-Omni number is stated.
Appendix J: the held-out evaluation above (Tables 9–11), including the controlled comparisons, delayed results, interjections, and tool-result relay.
Appendix E: the Daily-Omni comparison under offline and streaming settings (Table 4).
Appendix B: discussion of MoshiRAG and the concurrent Gander and Realtime-Venus.
Appendix H: labeled as training examples.
Appendix A: a human evaluation of interaction naturalness and a direct ablation of Writer–Director supervision are listed as future work.
We are grateful to the reviewers for helping us improve the paper.