- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
Authors propose DiffuPlex, the method for full-duplex dialogue model. Rather than the agent listening to the user and responding autoregressively, whenever the user fills in one frame, it predicts future user and assistant frames within a fixed block, and if the user input diverges from this prediction, it revises only the subsequent part.
Critical strengths
- The idea of predicting future user and assistant frames within a block and revising only the subsequent portion when the observed user input diverges from the prediction is interesting.
- The paper examines runtime alongside conversational behaviors such as turn-taking and backchanneling, rather than reporting runtime results alone. Although the interpretation and practical significance of these results need further clarification, considering both aspects is a positive feature.
Critical weaknesses
-
In lines 43–44, you say “update relatively coarse chunks,” but it is unclear what this is being compared with and roughly how many Hz “coarse” refers to. It would be better to mention even a simple number.
-
The ratio between successful predictions and cases requiring revision seems likely to vary depending on whether the user is having a normal conversation and on the evaluation dataset. However, the 60-clip runtime subset mentioned in the caption of Table 2 is not sufficiently characterized in terms of turn-taking, pause handling, interruption, and backchanneling.
-
The reduction in Time and increase in speed offline probably assume running as fast as possible rather than actually conversing with a person, and I question the necessity of this result. In highly constrained situations such as on-device settings, I understand doing this in preparation for cases where computation cannot finish within 80 ms. However, since the GPU used is an RTX A6000, the benefit obtained by reducing computation by that amount seems very marginal. It seems necessary to strengthen the argument for this need, for example, by showing that it helps when running on a CPU or NPU. In particular, looking at the Wake behavior column in Table 2, the p99 time is actually slower for DiffuPlex, and RL-Seamless also appears to have no cases exceeding 80 ms. If so, I wonder whether there is no advantage in terms of the timing aspect mentioned in the introduction.
-
To be honest, I had difficulty understanding Figure 2. First, it would have been easier to understand that the blue, green, yellow, and purple elements in (a) are codec streams if cb0, cb1, etc. were shown on the left, as in Figure 4. Also, it would be easier to understand if the “Interaction unfolds” panel showed what role the stored predictions U1 and U2 play during revision.
-
You say that Figure 3 shows efficiency–interaction, but it is unclear whether turn-taking latency and backchannel frequency are really metrics related to efficiency rather than metrics showing the model’s accuracy. For turn-taking latency, to put it plainly, couldn’t it be achieved by fine-tuning PersonaPlex on data with fast turn-taking, and couldn’t backchannel frequency be achieved by fine-tuning on data with frequent backchanneling? Also, in Figure 3(c), it is unclear what the y-axis means. Even if that were understood, the score differences among AR, Silero, the two Smart Turn models, and DiffuPlex are small, so I question whether they are significant.
-
I am not an expert in diffusion LLMs, but from the perspective of someone in the speech field, I am not sure whether DiffuPlex’s methodology really uses diffusion. Rather, it uses a BERT-style masking–prediction method, and considering that it is distilled from an autoregressive model, I wonder whether it would be more appropriate to categorize it as speculative decoding.
-
Since Full-Duplex-Bench v1.0 used GPT-4o as the Interruption Judge, results measured using GPT-4o also seem necessary to make the same comparison with other papers.
Questions for authors
- My understanding is that, rather than the agent listening to the user and responding autoregressively, whenever the user fills in one frame, it predicts future user and assistant frames within a fixed block. If the user input diverges from this prediction, it revises only the subsequent part. Is draft correctness judged based on whether it diverges, rather than on an exact match/mismatch?
- What is the ratio between cases where prediction succeeds and actually reduces the time and cases where revision is ultimately needed? Could you provide statistics on turn-taking, pause handling, interruption, and backchanneling, or a few actual data examples, for the 60-clip runtime subset mentioned in the caption of Table 2?
- If revision is ultimately needed, does it take more time than autoregressive generation? It would be easier to understand if you showed the case where everything is done autoregressively (worst case) and the case where everything succeeds through diffusion (best case) on some of the evaluation datasets.
- Is latency actually a problem for models like PersonaPlex? Is the actual response time slow enough to be inconvenient? Given the Wake behavior results in Table 2, what advantage does DiffuPlex provide in terms of the timing aspect mentioned in the introduction?
- How are turn-taking latency and backchannel frequency related to efficiency rather than accuracy? What does the y-axis in Figure 3(c) mean, and are the small score differences among AR, Silero, the two Smart Turn models, and DiffuPlex significant?
- If there is a paper from the diffusion LLM literature that you have substantially drawn on, I would appreciate it if you mentioned it in the methodology, for example, “we adopted ~ et al.”
- Could you also provide the results in Table 1 measured using GPT-4o as the Interruption Judge, as in Full-Duplex-Bench v1.0?
Since I did not use AI while reviewing, the answers to my questions may be somewhere in the appendix. If so, please let me know by mentioning the section numbers, and I will consider changing my score.
12 comments
W2, Q2 & Q3. Runtime-subset coverage, draft use, and revision costs.
Thank you for these questions. We apologize that the runtime subset was not sufficiently explained. We provide its composition, draft-use statistics, and a full-benchmark timing check below.
-
The 60-clip subset and draft use.
For controlled timing, we ran systems sequentially on one A6000, with no other experimental jobs on the server. We used 60 clips to finish these isolated measurements sooner: 15 per interaction category, including 7 Candor and 8 synthetic pause clips. We selected clips using the AR outputs, aiming to keep assistant speech/silence proportions similar to the full benchmark. Appendix E.3 now explains this selection.
Below, is frames advanced per backbone call. Revision (%) shows how often new user input triggers the next backbone call before all retained frames are used. Each clip's final interval is excluded from this percentage; includes all frames and calls.
Task Clips LISTEN c LISTEN revision (%) SPEAK c SPEAK revision (%) Pause (Candor) 7 3.07 65.4 3.29 70.1 Pause (Synthetic) 8 2.92 58.5 3.06 64.6 Backchannel 15 2.56 59.8 2.77 67.4 Turn taking 15 2.02 28.5 2.19 34.1 Interruption 15 1.38 4.3 1.70 10.0 Overall 60 1.98 29.8 2.26 37.6 LISTEN/SPEAK denote DiffuPlex-LISTEN/DiffuPlex-SPEAK. This table is now in Appendix I.1, Table 19. The original Figure 7 analysis uses all 727 clips, with revision rates of 23.6%/32.1%.
Multi-frame use and revision can occur in the same interval. Among non-final intervals in the subset, 23.8%/37.9% advance multiple frames. Figure 9 in Appendix I.3 already illustrates this: 3 of 12 retained frames are consumed before the remaining 9 are revised. Figures 8 and 11 show consecutive 16-frame advances and repeated one-frame advances, respectively; further audio examples are on the demo page.
-
Table 2 remeasured on all 727 clips.
To check whether the subset affects the conclusions, we additionally measured the full benchmark. Times are cumulative seconds over 727 clips, not the 60 in Table 2; parentheses give speedups relative to RL-Seamless. Streaming wall is elapsed loop time; E2E GPU includes the codec; Core LM covers language-model computation.
Model c ↑ Streaming wall ↓ E2E GPU ↓ Core LM ↓ Moshi 1.00 6424.3 (1.170×) 6420.0 (1.170×) 5526.0 (1.195×) PersonaPlex 1.00 7500.8 (1.002×) 7496.3 (1.002×) 6604.7 (1.000×) RL-Seamless 1.00 7514.1 (1.000×) 7509.6 (1.000×) 6605.8 (1.000×) DiffuPlex-LISTEN 1.91 5314.3 (1.414×) 5298.9 (1.417×) 4192.3 (1.576×) DiffuPlex-SPEAK 2.23 4755.5 (1.580×) 4740.9 (1.584×) 3684.9 (1.793×) Model Relative Core LM time/wake Wake-frame p99 ↓ Wakes >80ms ↓ Moshi 0.84× - - PersonaPlex 1.00× - - RL-Seamless 1.00× 52.60ms 0% DiffuPlex-LISTEN 1.21× 66.61ms 0% DiffuPlex-SPEAK 1.24× 66.68ms 0% Relative time/wake is normalized to RL-Seamless; p99 concerns frames invoking the backbone. The full benchmark also shows runtime savings, with all measured wakes below 80ms.
-
Does revision cancel the savings?
On the timed 60-clip subset, intervals consuming multiple frames before revision have these mean costs:
Model Frames consumed Processing time Estimated AR time DiffuPlex-LISTEN 4.81 85.83ms 224.69ms DiffuPlex-SPEAK 4.69 85.24ms 219.19ms These are cumulative processing times across several frames, not single-frame latencies. They include the initial prediction, including subsequently discarded frames, and all frames consumed before the next wake. The AR estimate uses the same clip's mean AR frame time for the same number of frames. Later revision wakes are included in the total runtimes above.
We also ran forced timing extremes over all 727 clips. Forcing revision at every check gives wall-clock speedups of 0.799×/0.786× for DiffuPlex-LISTEN/DiffuPlex-SPEAK, so immediate revision can be slower than AR. Disabling confidence filtering and revision gives 2.203×/4.601×; LISTEN still stops at non-PAD states. These are timing stress tests, not quality-preserving settings. Revision after several consumed frames can still save computation, while immediate revision can erase the benefit.
Thank you for prompting us to clarify the subset and test whether the timing benefits persist over the complete benchmark.
Q1. When does the backbone wake, and how is draft consistency assessed?
Thank you for asking us to clarify this procedure. User frames arrive every 80ms, but the backbone does not predict a new block at every arrival. At each wake, it predicts several future user and assistant frames, retains an initial assistant prefix for playback, and stores the user semantic-token distributions and assistant text-token probabilities needed for revision. It predicts a new block only when the retained frames run out or incoming user input triggers revision.
The check uses probabilities rather than an exact token match. When a user frame arrives, we look up the probability assigned to its semantic token in the stored user prediction. We combine this with a term based on the probabilities of the assistant's selected text token and its alternatives. Revision becomes more likely when the user input was unexpected and the assistant was uncertain about its response. No additional backbone pass is needed for this check. If the score reaches , the unplayed future is discarded and re-predicted at the next frame; received user input and played assistant audio remain unchanged. The rule and derivation were already provided in Section 3.3, Eq.4, and Appendix D.3. We have clarified what is stored and how the per-frame check differs from a backbone wake.
W1. What does "relatively coarse chunks" mean?
Thank you for pointing out this ambiguity. We intended to compare interaction updates with the 80ms frame interval (12.5Hz) used by Moshi and PersonaPlex. SyncLLM uses 160-240ms chunks (approximately 4.17-6.25Hz), while MiniCPM-o 4.5 reports selecting 1s chunks (1Hz). We have revised the Introduction to give these numbers and distinguish these chunk-based models from models whose user and assistant streams advance together at a finer frame-level granularity. Single-sequence serialization does not itself imply coarse updates.
W6 & Q6. Relationship to masked prediction, diffusion, and speculative decoding.
Thank you for encouraging us to clarify these connections. Masked prediction is shared by BERT-style objectives and masked diffusion: MDLM (Sahoo et al., 2024), for example, derives its objective as a mixture of masked-language-modeling losses. The specific construction we directly adopt is the clean-context training layout of Block Diffusion (Arriola et al., 2025, Section 3.2), already cited in Section 3.1. It processes clean and masked sequences together so that masked future blocks condition on the preceding realized interaction (Appendix C.3, Figure 4).
DiffuPlex adapts this layout to rolling full-duplex prediction. Masks reflect which future inputs are unavailable at each lookahead. Each wake predicts multiple future positions in one temporal-backbone pass, with causal attention and distillation from the frozen AR backbone. A confident prefix is retained, and future prediction repeats as new user input arrives. Related methods include causal masked prediction with confidence-based prefix commitment in FLUID (Ma et al., 2026), one-step masked drafting in TiDAR (Liu et al., 2025), and frozen-AR distillation with gated LoRA in Uno (Sahoo et al., 2026). Section 3.1 now explicitly identifies the adopted training layout, and Section 2 discusses these related methods.
Distillation describes how the predictor is trained; speculative decoding involves verifying proposed tokens during generation. DiffuPlex performs no AR verification or rejection sampling and does not claim exact preservation of the AR output distribution. Instead, Eq.4 checks predictions against newly arriving user input and revises only the unplayed future. This addresses the full-duplex setting, where future user speech is unavailable when the draft is generated.
W4. Clarifying Figure 2 and the role of stored predictions.
Thank you for these suggestions. We apologize that Figure 2 was difficult to follow. Panel(a) now labels the streams and codebooks, separates realized interaction from masked future inputs, and distinguishes the training-only paths. We also corrected an illustration-only masking error: assistant inputs are masked from , user codebook 0 (cb0) from , and user cb1-7 from , consistent with Appendix C.2-C.3, Eq.8, and Figure 4. The implementation and reported results are unchanged.
Panel(b) connects stored user predictions to observed tokens through schematic probability distributions, with the observed tokens highlighted. These probabilities contribute to the revision score together with the assistant-side term in Eq.4. In the example, observing gives , so the retained draft continues without another backbone call. Observing gives , triggering re-masking of the unplayed future and prediction at the next frame. The figure also distinguishes the retained prefix, played audio, and unplayed future.
Thank you for the careful reading and detailed feedback. Your comments prompted additional experiments and helped us clarify the method and its practical benefits. We address related weaknesses and questions together below. Manuscript changes are highlighted in red.
W7 & Q7. Evaluation with GPT-4o as the interruption judge.
Thank you for this suggestion. We rescored the original Table 1 outputs and transcripts using
gpt-4o-2024-08-06, retaining the official FDB v1.0 evaluation prompt and scoring procedure.Both DiffuPlex policies achieve GPT-4o scores similar to the AR backbone. These results support our finding that reducing backbone calls largely preserves interruption-response appropriateness.
We have updated all FDB v1.0 interruption-judge scores in the manuscript, including the main comparisons, ablations, and corresponding paired statistics. All corresponding scores in the remaining responses, including additional experiments, also use
gpt-4o-2024-08-06.W3 & Q4. What practical benefits does acceleration provide when the backbone already meets the 80ms deadline?
Your question motivated us to examine how computational savings affect real-time operation. We conducted additional experiments with input frames arriving every 80ms, measuring resource consumption and evaluating multiple conversations on the same GPU.
Single-session timing and resource consumption.
As you point out, RL-Seamless already meets the 80ms deadline for a single session on the A6000. DiffuPlex predicts multiple future frames at each wake, making individual wakes more expensive but less frequent. This explains the higher wake-frame p99 in Table 2: 46.85ms for RL-Seamless versus 61.34ms for DiffuPlex-LISTEN and 61.41ms for DiffuPlex-SPEAK. All measured wakes remain within 80ms, while total computation over the conversation decreases.
Under real-time pacing on the 60-clip runtime subset, GPU utilization decreases from 58.0% for RL-Seamless to 38.0% for DiffuPlex-LISTEN and 34.4% for DiffuPlex-SPEAK. GPU energy consumption decreases by 16.1% and 19.9%, with 95% confidence intervals of [12.6,19.4]% and [17.4,22.5]%, respectively. These savings provide a practical benefit even when a single session already meets its deadline.
Multiple conversations on the same GPU.
In deployment, multiple conversations may share a GPU. We evaluated all 727 FDB v1.0 clips with 3 concurrent sessions on an RTX PRO 6000 Blackwell Max-Q (96GB) and 2 on an RTX A6000. Each session uses its own model instance, with one run per setting.
If a frame’s output is not ready within its 80ms interval, the resulting audio is buffered for later playback. All systems use the same playback policy. We report the official Smooth Turn Taking metrics (TOR and latency) and User Interruption metrics (TOR, Judge, and latency), measured from the same playback outputs.
Latency measures the interval from the end of the user’s utterance to the assistant’s first response word in the corresponding task. Judge evaluates interruption-response appropriateness using GPT-4o.
On the PRO 6000 Max-Q, increasing from 1 to 3 sessions raises turn-taking latency from 0.101s to 0.894s for RL-Seamless, from 0.144s to 0.191s for DiffuPlex-LISTEN, and from 0.156s to 0.203s for DiffuPlex-SPEAK. The increase is therefore 793ms for RL-Seamless and 47ms for either DiffuPlex policy. Thus, the savings help maintain more stable response times under concurrent load, with interruption-appropriateness scores similar to RL-Seamless at the same load.
CPU/NPU deployment.
We also attempted CPU inference on a dual-socket Intel Xeon Gold 6348 server with 56 physical cores. In the tested configuration, even a single AR decoding step exceeded 80ms, preventing frame-by-frame inference at the original interaction rate. Enabling efficient full-duplex interaction on CPUs and NPUs remains a valuable research direction.
Section 5.2 now reports the resource savings and concurrent-session protocol, with complete FDB v1.0 automatic metrics in Table 3. We appreciate your guidance in helping us substantiate these practical benefits beyond offline speedups.