- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper extends DFlash-style block diffusion speculative decoding by reusing target-verifier hidden states from rejected speculative positions. DFlow projects these states using DFlash’s feature projection, aligns them with the drafter’s masked-token predictions, and injects them into the next drafting round through modulation conditioned on the bonus token and the first rejected token. Multi-Round Self-Conditioned Training unfolds an initial mask prediction round followed by two reject conditioned rounds. Experiments on Qwen3-1.7B, 4B, and 8B report approximately 10-13% higher average acceptance length and 9-12% higher relative decoding speed than DFlash under greedy decoding, with improvements also reported at temperature 1 and under concurrent SGLang serving. An integration with Domino further improves acceptance length.
Critical strengths
-
DFlow offers a simple method to improve drafting by reusing verifier representations that would otherwise be discarded after rejection. The method aligns these representations with subsequent draft positions and conditions their reuse on the correction at the rejection boundary, without requiring additional target model forward passes at inference time. Table 3 supports the benefit of relay beyond multi-round mask-only training, and Figure 4 reports only 0.51 ms of additional latency in the profiled Qwen3 8B setting.
-
Experimental results show consistent improvements in actual inference efficiency. Table 1 shows improvements over DFlash in acceptance length and end-to-end decoding speed across six benchmarks, three model sizes, and two temperatures, while Table 2 shows higher serving throughput across the evaluated concurrency levels. Combining DFlow with Domino further increases acceptance length from 4.02 to 4.50, suggesting that verifier reuse can complement stronger drafting mechanisms.
Critical weaknesses
-
The additional training cost and performance under matched training budgets remain unclear. The DFlash + Self-Conditioning baseline in Table 3 controls for multi-round training, and its comparison with DFlow supports the benefit of verifier relay. However, this does not establish a comparison under matched computational budgets. DFlow constructs relay features through online verification by the frozen target model, but the associated verification cost and total training GPU hours are not reported. It remains unclear whether standard DFlash trained for more steps under a comparable compute budget could narrow the gap.
-
Stochastic verification and runtime measurement are insufficiently specified. Section 3 describes exact token matching without specifying the sampling and acceptance/replacement procedure used at temperature 1 or reporting correctness checks. Conditioning on relayed states does not necessarily introduce bias, but the requirements for preserving the target distribution and managing the cache should be clearly specified. The evaluation protocol for Table 1 leaves some key settings unspecified, including the inference framework, batch/concurrency, and whether timing includes prefill or only decoding, and does not report the number of repeated measurements or their variability. These gaps limit independent assessment of the claimed lossless acceleration.
Questions for authors
-
How is temperature=1 verification implemented, including target sampling, acceptance, bonus-token handling, and cache updates? Is the same verification scheme applied to EAGLE-3, DFlash, and DFlow at T=1?
-
For the three training settings in Table 3, how do training GPU hours, target model verification costs, and the number of supervised draft blocks compare?
-
To assess the training cost tradeoff, could the authors compare DFlow against a DFlash model trained with a computational budget comparable to DFlow’s?
-
What implementation and timing settings were used for Table 1, including the inference framework, batch/concurrency, and whether timing includes prefill or only decoding? How many repeated measurements were taken and how variable are the results?
-
In Figure 4, verify latency is reported as 37.55 ms for all three methods, and draft latency as 6.09 ms for both DFlash and DFlow. Were these component latencies measured separately for each method, or were single measurements reused across methods? Does the 0.51 ms relay component include all additional feature projection and modulation costs?
-
Does the increase in acceptance length from 4.02 to 4.50 with Domino+DFlow in Table 4 also yield an end-to-end decoding speedup over Domino alone?
5 comments
Gentle reminder regarding our rebuttal
Dear Reviewer KZ1N,
Thank you once again for your valuable comments on our submission. As the discussion phase is approaching its end, we would like to kindly confirm whether we have sufficiently addressed all of your concerns (or at least part of them). Should there be any remaining questions or areas requiring further clarification, please do not hesitate to let us know. If you are satisfied with our responses, we would greatly appreciate your consideration in adjusting the evaluation scores accordingly.
We sincerely look forward to your feedback.
Best regards,
The Authors
As the rebuttal period draws to a close, I encourage reviewers to engage with the authors’ responses and revised manuscript, clearly identifying which concerns have been addressed and which remain unresolved.
Given the limited time remaining, please prioritize evaluating the evidence and clarifications already available in the current revision. If you nevertheless consider additional experiments essential to support the paper’s claims, please specify what is needed and why. Even if the authors cannot complete these experiments within the remaining time, a discussion of the underlying concerns and the available evidence would help inform our final assessment.
Thank you all for your time and constructive engagement.
Response to Reviewer KZ1N
Response to Reviewer KZ1N
1. Training Cost and Performance under Comparable Computational Budgets (Q2, Q3, W1)
All three settings in Table 3 use the same total number of supervised draft blocks. DFlash + Self-Conditioning and DFlow also share the same training procedure across multiple drafting rounds, with verifier replay disabled and enabled, respectively. This comparison controls the amount of supervision and isolates the contribution of verifier replay within the same training framework. The ratio of GPU hours consumed per training step is 1.00 : 1.75 : 3.50 for DFlash, DFlash + Self-Conditioning, and DFlow, respectively.
To compare performance under the same computational budget, we extended DFlash training to match DFlow’s total GPU hours. DFlash achieves an average acceptance length of 3.68, comparable to the DFlash + Self-Conditioning baseline in Table 3 but still below DFlow. Our extended training experiments further show that DFlash’s performance largely saturates, with additional training yielding no sustained improvement. These results indicate that DFlow’s gains under the evaluated setup cannot be attributed solely to additional training computation.
Although DFlow requires additional training computation, it delivers greater decoding speedups than DFlash. For a fixed target model, the extra training cost is incurred upfront, whereas the computational savings accumulate across subsequent inference requests. In deployments with sufficiently large inference volumes, these savings can outweigh the additional training cost, yielding a net reduction in overall computational cost.
2. Sampling and Verification at Temperature 1 (Q1, W2)
At temperature (T = 1), verification tokens are sampled from the target distribution, while draft tokens are selected by argmax. Draft tokens are accepted consecutively until the first mismatch, where the target sample is emitted as the bonus token. Relay conditioning only affects draft proposals.The emitted tokens remain target samples conditioned on the actual emitted prefix, preserving the target distribution. After verification, the target KV cache is truncated to the accepted prefix, while hidden states from rejected positions are retained only to guide the next drafting round.EAGLE-3, DFlash, and DFlow use the same inference framework, with identical target sampling, acceptance rules, bonus-token handling, and cache updates. The only difference is how draft tokens are generated.
3. Evaluation Settings and Overhead Measurement (Q4, Q5)
The results in Table 1 were obtained using the Transformers backend with a batch size of 1. Following DFlash, timing covers only the decoding stage and excludes prompt prefill. The serving experiments in Table 2 use SGLang with concurrency levels of 2, 4, 8, 16, and 32, and likewise measure decoding performance only. Timing measurements were repeated three times. The observed variation in decoding speedup is approximately 0.03×, which is small relative to the gains achieved by DFlow over DFlash.
Regarding Figure 4, target verification is primarily memory bound, leading to similar verification latencies across all methods. DFlash and DFlow also share the same draft backbone, with relay computation measured separately, so their draft execution latencies are expected to be similar. Our empirical measurements are consistent with this analysis. For simplicity, Figure 4 presents these shared components using common representative values. The reported relay latency of 0.51 ms accounts for all additional runtime costs introduced by DFlow relative to DFlash, including the additional feature projection and modulation costs.
4. End-to-End Speedup with Domino (Q6)
We additionally measured the decoding speed of Domino and Domino + DFlow on a PRO 5000 GPU. Both methods were evaluated under identical runtime settings, with results averaged across six benchmarks. DFlow provides an end-to-end decoding speed improvement of approximately 9.9% over Domino, demonstrating that the acceptance length improvement translates into practical acceleration.