- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper first mentions current full-duplex models fail to exhibit strong linguistic and cognitive behavior compared to turn-based models. The authors argue that the major reason is the single one-dimensional autoregressive timeline, which causes gradient conflict between timing and semantics and frame-rate binding between 80 ms audio frame rate and Chain-of-Thought. They proposed a new architecture to decouple the "when to speak" and "what to speak" in a full-duplex model. Built on discrete diffusion language models, they use autoregressive mode to process incoming user audio and the prediction of model's turn taking. When the model decides to speak, it runs in diffusion mode for refining the response in an orthogonal timeline compared to the real timeline. They did an extensive evaluation of their model on common benchmarks and also several ablations to show that their model achieves a fast speed while being smart.
Critical strengths
- It is the first paper to integrate text diffusion models in the application of Full-duplex models to my knowledge.
- This paper proposed a good research question and direction to decouple "when to speak" and "what to speak" and this is tightly related to the current open problems in full-duplex models.
- They show that with diffusion for response generation, the decoded tokens per second is largely improved over other baselines.
Critical weaknesses
-
The presentation of the overall model architecture is hard to follow and perhaps some information is missing.
-
Figure 2 is not easily understandable by readers. It is unclear from the figure how the model works during inference especially how it continually processes incoming 80ms chunks of user audio and formulates the response using the diffusion mode. The connection between the two modes is not easy to understand by looking at this figure.
-
The text description in the paper is more clear to me of how the AR mode and diffusion mode work together. If I'm understanding it correctly, there is single sets of weights for the backbone model (excluding the speech encoder and the TTS module). During inference, the AR mode is always on, it continually receives incoming user audio (80 ms) chunks and predicts the states
<listen>,<speak>,<bos>, and<eos>. Once<bos>is sampled, it launches the "diffusion model", which run diffusion steps to generate the system response. So the entire system is like having 2 requests to the same model while one request is always on, the other only runs if the system response generation is triggered. However, I cannot find how these two requests are executed/scheduled during inference and I guess this also matters for latency.
-
-
The problem formulation in the introduction is confusing and the discussion scope seems narrowed.
-
Confusing comparison: Starting form Line 61, the authors point out the pros and cons between "spoken language models achieve high intelligence but rely on external VAD" and "full-duplex models that unify when to say and what to say deliver fluid interactions but comes with severe linguistic and cognitive degradation". I think this is an oversimplified discussion. From my understanding the two reasons the authors brought up (Gradient conflict and Temporal Frame-Rate Binding) depend on whether the full-duplex system is cascaded or end-to-end. A turn-based model can be achieved by a cascading different model components or an end-to-end model. The same goes with full-duplex, we can achieve full-duplex behavior by cascading or using a single end-to-end model. I agree with the authors on the two reasons, but I think this depends more on whether the system is cascaded or not.
-
Lack of discussion for other recent Full-duplex system architecture: Continuing the system architecture design from the prior bullet point, there is another line of research that proposed to cascade two models to achieve a highly intelligent full-duplex system. They use one model for handling full-duplex interactions while the other model processes deep reasoning.
-
Not fully convinced of integrating diffusion in the full-duplex modeling.
-
The author claims that the diffusion model for response generation decouples the conversation timing and response generation thereby avoiding the gradient conflicting issue. However, why particularly diffusion model? Aren’t any methods that separate into two models capable of working? In the appendix, Line 832-833, the author also tries using the AR mode for response generation and mentions that it sometimes even works better. If the advantage of using diffusion for response generation is about the speed, perhaps changing to another LLM backbone would help? I guess faster generation speed doesn't necessarily lead to better performance right? E.g. some models might be able to achieve a higher intelligence with a shorter reasoning trace, thereby still get an advantage with lower token decoding speed.
-
Furthermore, the authors claim that the diffusion works orthogonal to the real physical time therefore solving the Temporal Frame-Rate Binding issue. However, I'm not clear how it entirely decoupled the two. From my understanding, while the diffusion model is formulating the response, it stays silent to the user. The real-time interaction still holds for the diffusion steps to finish.
-
-
-
Thinking and Full-duplex seem to be exclusive in the experiments.
It seems that there is no setting in the paper where the thinking and full-duplex are both on. I think this experiment is highly related to the paper's introduction. Line 69-71,
full-duplex models can't allocate "elastic test-time compute (i.e., Chain-of-Thought). -
Full-duplex model baselines are outdated.
The full-duplex baselines in the paper are classically full-duplex models (E.g. Moshi, Freeze-Omni...). However, they are somehow outdated. There are several recent full-duplex models that largely improve the intelligence while maintaining full-duplex responsiveness.
Questions for authors
- For Figure 1, why is the x-axis for no-thinking first audio latency, but for thinking on, the x-axis is output speed? For readers, I guess the first audio latency matters more than the decoding speed. I also cannot find the first audio latency for thinking on in the paper.
- With thinking on, Diffuplex seems to improve only a little on the intelligence score. What is the reason?
- Why there is no backchanneling results on FullDuplexBench?
4 comments
Thanks for the authors' reply.
There is still something that is unclear to me.
After reading the authors response, my understanding the argument is two-fold for this paper (correct my if I'm wrong)
- Decoupling conversational behavior from response content generation can improve the model’s performance.
- Using diffusion for response generation can enhance the decoding speed.
I agree with the authors on the first point. However, for the current version of the paper, it seems misleading to the readers. The two points are not disentangled. What I meant is that diffusion is not the only way to decouple conversational behavior from response content generation. The author also agreed in the reply that cascaded full-duplex system is also an option.
We agree that cascaded systems with separate text generators can also avoid frame-rate binding.
I would suggest mentioning cascaded systems in the intro and related works so that the readers won't get misled.
For the second point, I also agree with the authors that decoding speed is an advantage of diffusion models. However, in the context of full-duplex conversation, most of the time, the response is short. Under typical conversation, how much speed advantage do we get from using diffusion compared to AR? And how does the quality of spoken response compare? And since we are operating in a real-time full-duplex conversation, the speaking is in streaming mode. Even if we need a long response, we don't have to wait till the AR model finishes the entire response generation. Its output can be streamed to the user while the response is still autoregressively generating.
I think it is important to include the AR ablation (generate the response token by token) which is currently mentioned briefly in the appendix with no evaluation numbers. This will be a strong result for using diffusion because this is the apple-to-apple comparison while any other Full-Duplex model baselines all differ in model sizes, training data... etc. Given that the author already mentioned such an experiment in the original manuscript, I think they do not need to run additional experiments on this.
The other benefit of using diffusion is for thinking, which is mentioned implicitly in the manuscript. The authors also mention this explicitly in the rebuttal.
Diffuplex additionally offers substantially higher measured text-generation throughput than the evaluated AR counterparts, reducing the generation cost of reasoning, although enabling thinking still adds latency.
However, "full-duplex" and "thinking" seems exclusive in the current proposed system. (Please correct me if I'm wrong. I've raised this question in the review but it seems like the author missed it.) This further undermines the advantage of using diffusion in the full-duplex conversation.
Other points:
-
The author mentioned FLAIR is evaluated as a baseline, however, it only shows up in the first graph on the first page but no detailed results on the main table (Table1).
-
Regarding other baseline full-duplex architectures, I think "MiniCPM-o 4.5" (https://arxiv.org/pdf/2604.27393) is one suitable choice (an end-to-end model)
From my understanding, the current Diffuplex run two processes (two models inference with different decoding logic) when generating the response but there is only a same set of model weights. Given this fact, I think Diffuplex is in between the end-to-end and cascaded full-duplex models. So I think even cascaded full-duplex systems are worth putting in the table as references.
Given that the deadline for rebuttal is approaching, I think it is reasonable that the author might not have time to add more suitable baseline/references. And comparing to the other points I raised in the response, adding other model baselines and references is not the top priority because it is hard to have a fair comparison directly. That said, it is still good to have more stronger Full-Duplex models in the main comparison table.
-
The back-channeling results on FullDuplexBench is still missing
-
The detail for the inference logic is still missing raised in W1
However, I cannot find how these two requests are executed/scheduled during inference and I guess this also matters for latency.
-
Is Full-Duplex and thinking exclusive for Diffuplex?
The results in the paper seem exclusive.
-
First-audio latency with thinking on is still missing.
I understand text throughput and first-audio latency are distinct metrics as mentioned in the authors' response. But why aren’t both metrics reported for both thinking on and off?
Response for Reviewer qK1P (Part-1)
Thank you for the follow-up and for identifying the comparisons that would most directly clarify our contribution. We address your points below.
- Separating decoupling from diffusion
We agree that decoupling conversational control from response generation does not require diffusion. Our contribution combines two complementary choices: learning conversational control separately from response generation, and using diffusion to accelerate the latter within a shared backbone. We will revise our realted work and introduction according to your suggestions, and thanks again for your comment.
- Same-model AR comparison and the practical value of faster generation Thank you for emphasizing this comparison. The AR compairson results are: | AR decoding | Alp. | Com. | Wil. | SDQA | MMSU | OBQA | BBH | IFE. | Adv. | Overall | | --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | | Non-thinking | 95.40 | 87.60 | 90.00 | 64.60 | 73.80 | 87.90 | 86.50 | 80.30 | 99.60 | 85.10 | | Thinking | 94.09 | 87.00 | 89.72 | 62.93 | 82.27 | 94.07 | 88.90 | 86.34 | 98.85 | 87.13 |
We agree that AR can hide much of the generation time for short spoken answers. However, spoken agents are also expected to support task-oriented interactions involving reasoning and tool use (like TUA-voice). Internal reasoning or a structured tool call may need to finish before the corresponding action or spoken answer can begin, so this computation cannot always be hidden behind audio playback. The same-model first-audio measurements below directly examine this setting. We have number for reference in the next quesiton.
Serving efficiency is another consideration: a company serving many users from a fixed GPU pool must consider the computation required per interaction, not only whether one conversation can stream in real time. More efficient generation could provide capacity for additional sessions. We present this as a deployment motivation, rather than claiming that single-request TPS establishes multi-user serving throughput.
- Compatibility of thinking and full-duplex interaction
Thinking and full-duplex operation can be enabled simultaneously in Diffuplex, as demonstrated in our supplementar video. The AR interaction policy continues monitoring the user while the response generator produces its thinking trace and answer.
We now report the measured first-audio latency with thinking enabled, using the Fisher interaction-policy configuration:
| Response decoding | Thinking | First-audio latency |
|---|---|---|
| Diffusion | Off | 0.396 s |
| Diffusion | On | 0.711 s |
| AR | On | 1.689 s |
Enabling thinking adds approximately 0.32 s to diffusion’s first-audio latency. With thinking enabled, AR decoding takes approximately 0.98 s longer than diffusion in the same model. Comparing its thinking-on latency directly against other AR models with different reasoning lengths would therefore confound decoding efficiency with reasoning behavior. The same-model comparison is more informative, although generated reasoning lengths can still differ between decoding modes.
We also wish to explain why our original full-duplex comparison used the non-thinking configuration. All full-duplex baselines in that comparison were evaluated without explicit thinking. Enabling thinking only for Diffuplex would create an asymmetric comparison. Nevertheless, we agree that reporting thinking-on quality and latency together provides a useful additional operating point, the new measurements make the trade-off explicit: thinking increases first-audio latency, but diffusion substantially reduces that increase relative to AR decoding while achieving similar response quality.
Response for Reviewer qK1P (Part-2)
- Inference scheduling
The AR policy and diffusion generator share one copy of the backbone weights on a single GPU and execute alternately, rather than concurrently, with separate KV caches. The scheduler prioritizes the policy’s single-step state prediction at each 80-ms update, with the listening-mode adapter enabled. Between policy updates, it advances diffusion generation with the adapter disabled. Thus, the two modes share GPU execution time; policy updates take priority, while diffusion uses the intervening compute budget.
The policy’s decisions control response generation. A <bos> decision triggers context prefill and diffusion decoding. The first committed prefix is immediately passed to TTS while the remaining text continues denoising; subsequent committed text is also streamed to TTS. First-audio latency therefore includes the policy’s decision delay, context prefill, prefix generation, and synthesis of the first audio packet. When the policy predicts <eos> in response to a user interruption, the system stops playback and cancels unfinished response generation. The next response is conditioned on the updated context, including the user’s interruption.
The current implementation retains all generated response text in the persistent generation context, including text that may not have been played before an interruption. This is a limitation of the current context-management implementation: text generation runs ahead of playback, and we do not yet align playback progress with individual text positions. We plan to combine TTS word-level timestamps with the playback position to retain only the spoken prefix and discard the unplayed remainder.
- Additional baselines and cascaded systems
Thank you for recommending MiniCPM-o 4.5. We have completed its VoiceBench evaluation and report the results below:
| Model | Alp. | Com. | Wil. | SDQA | MMSU | OBQA | BBH | IFEval | Adv. | Overall |
|---|---|---|---|---|---|---|---|---|---|---|
| MiniCPM-o 4.5 | 84.0 | 71.8 | 71.6 | 68.35 | 66.69 | 87.69 | 55.00 | 80.59 | 98.85 | 76.06 |
The three open-ended QA scores are normalized to a 0–100 scale. We will include these results in Table 1. The omission of FLAIR’s detailed results was an oversight, which we will also correct.
Regarding conventional ASR–AR LLM–TTS cascades, their linguistic capability largely comes from the underlying turn-based AR model, while transcription errors can introduce additional losses. Their first-audio latency additionally depends on speech recognition, turn detection, and synthesis. Consequently, we would generally expect them to approach the capability of their turn-based backbone, rather than exceed it, and not necessarily match the low latency of tightly integrated models such as Moshi.
Our existing comparisons therefore provide complementary reference points: strong turn-based AR models for linguistic capability, and models such as Moshi for low-latency interaction. This does not replace evaluating a complete cascade, nor do we claim these reference points constitute strict performance bounds.
We are happy to add the relevant cascaded systems you recommend, provided their implementations are available for reproducible evaluation. We will also explicitly discuss cascaded full-duplex systems in the introduction and related work.
- Backchanneling
The submitted version did not explicitly train or evaluate agent-generated backchanneling as we view turn-taking and barge-in are more important. To address your concern, we retrained Stage 3 (w. Fisher) with an additional <bc> control token. We annotated backchannel segments in the Stage-3 interanl conversational data, which had previously been treated as silence, and used them to supervise the new state.
Due to time constraints, the Stage-2 response generator remains unchanged.* When <bc> is predicted, we prompt it to generate a 1~4 words acknowledgment of the user's statement. The generated backchannel is synthesized for playback but is not appended to the persistent response-generation KV cache.
The backchanneling results are:
| Model | TOR ↓ | Freq ↑ | JSD ↓ |
|---|---|---|---|
| Moshi | 1.000 | 0.001 | 0.957 |
| Freeze-Omni | 0.636 | 0.001 | 0.997 |
| Gemini Live | 0.091 | 0.012 | 0.896 |
| GPT Realtime 2.1 | 0.019 | 0.0099 | 0.962 |
| Qwen Audio 3.0 flash | 0.018 | 0.002 | 0.982 |
| Awaken | 0.145 | 0.073 | 0.753 |
| Diffuplex | 0.055 | 0.016 | 0.830 |
We also observed a trade-off: after adding backchanneling, turn-taking TOR-v1 to 93.3% from 100%. We have not had sufficient time within the rebuttal period to tune this balance further.
Response for Reviewer qK1P
Thank you for your insightful feedback. We clarify the following points and welcome further discussion.
Our discussion concerns full-duplex models that jointly learn conversational behavior and response generation, rather than full-duplex systems in general. Specifically, we target E2E models such as Moshi, where both functions are coupled within a frame-synchronous generative process. A cascaded system can also support full-duplex interaction but may delegate turn-taking to a separate controller.
Our concerns about conflicting training objectives and temporal frame-rate binding apply when conversational control and semantic generation are coupled within the same prediction process; they are not universal consequences of full-duplex interaction. Diffuplex addresses this coupling by learning the two functions separately within a shared backbone--like all baselines in this work.
Our comparison already includes FLAIR, Nemotron-VoiceChat, and Lychee-FD—all 2026 works, shown in Figure 1.
We welcome suggestions of additional relevant open-source full-duplex models and would be happy to evaluate them during the discussion period, provided checkpoints and reproducible inference implementations are available.
The primary motivation for diffusion is fast response generation, rather than an assumption that it must outperform AR decoding in response quality. In our updated experiment, Diffuplex generates an average of 1,873 text tokens/s on a single GB300 GPU, approximately 4.8× the measured throughput of a similarly sized AR model. Diffusion finalizes multiple response tokens per backbone forward pass: 13.0 tokens on average, compared with 1.0 for standard AR decoding.
For spoken interaction, response quality must be considered alongside generation efficiency and latency. Our objective is to preserve strong linguistic capabilities while substantially accelerating generation. Text throughput and first-audio latency are distinct metrics: the latter also includes turn-taking decisions and speech synthesis, which is why we evaluate them separately. The backbone’s competitive AR generation mode provides an alternative operating point.
Our concern is specifically about tying semantic generation to the real-time audio clock. In Moshi, the text stream is aligned with audio frames at 12.5 Hz. Inserting 125 additional reasoning tokens into this frame-synchronous stream would require at least 125 steps, or approximately 10 seconds. A faster GPU does not remove this alignment constraint without changing the inference formulation. This is what we mean by restricted “elastic test-time compute.”
Diffuplex retains an 80-ms interaction-policy update interval, but its response generation is not paced by incoming audio frames. Once triggered, diffusion generates and refines reasoning and answer tokens independently of that clock.
We agree that cascaded systems with separate text generators can also avoid frame-rate binding. Diffuplex additionally offers substantially higher measured text-generation throughput than the evaluated AR counterparts, reducing the generation cost of reasoning, although enabling thinking still adds latency.
Our training recipe prioritizes low-latency spoken interaction, rather than maximizing VoiceBench scores. Many spoken requests do not require extensive reasoning, and long thinking traces can introduce unnecessary delays. We therefore train 70% of Stage-2 responses using direct answers without explicit thinking content, strengthening the model’s ability to respond without prolonged deliberation. We beleive such kind of concise model could bring better user experience for spoken interaction.
Thank you for this suggestion. The temporal interaction between the two modes is difficult to convey fully in a static figure. We provide a workflow animation and a recorded inference demo in the supplementary material, with the user’s voice muted to preserve anonymity.
In the revised manuscript, we will expand the appendix with annotated snapshots from the animation to illustrate the inference workflow step by step.