- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes DEdit, a draft generation method based on a diffusion mechanism, which iteratively edits and refines drafts, and utilizes bidirectional context to correct previous prediction errors and extend the acceptable prefix. During training, a ProposalMix strategy is employed to mix model predictions with real tokens, enhancing error correction capabilities. Experiments show that DEdit achieves the highest macro-average token acceptance rate and inference acceleration ratio across seven benchmarks on Qwen3-4B and Qwen3-8B.
Critical strengths
-
The comparative experiments are thorough, comparing with DFlash, Domino, and DSpark. Among them, DFlash shares the same architecture as DEdit, with clear control of variables, effectively validating the benefits of the editing mechanism itself.
-
The architectural design is simple. DEdit uses the same drafter to complete both the initial proposal and subsequent editing, without requiring an additional editing model, and the editing capability acquired during training can be repeatedly applied to multiple rounds of editing such as P2, P3, during inference.
Critical weaknesses
-
Under Qwen3-4B in greedy mode, the accepted length is 14.8% longer than Domino, yet the final speedup is only 7.9%. When does the benefit of an additional edit outweigh the cost of an extra drafter forward?
-
This paper validates this under a specific environment with batch=1. How would the benefits change with larger batch sizes or concurrent requests?
-
The main experiments use two editing passes(K = 3). How was this number of edits determined, and how many rounds of edits should be employed in practical use?
Questions for authors
- Can the number of edits K be adjusted dynamically, determining when to stop editing based on the quality of the current draft, thereby avoiding unnecessary drafter forward overhead?
4 comments
W3: Choice of K
TLDR — We chose K=3 from the measured trade-off: the speedup stops improving after K=3, and we prefer fewer passes. The round absorbs the first two edits, so they raise τ at almost no added round time. A fourth pass raises τ by only 2.6% and brings no further speedup. In practice, add a pass while it raises τ more than round time; K can be changed at inference without retraining.
K counts drafter passes: one proposal pass plus K−1 edits. The table splits each round into its drafter part and its target part (W=32, rerun on a 30% prompt subset):
| K (passes) | τ | Drafter (ms) | Target (ms) | Round (ms) |
|---|---|---|---|---|
| 1 | 6.03 | 3.8 | 21.4 | 25.5 |
| 2 | 7.31 | 7.4 | 17.8 | 25.5 |
| 3 (ours) | 7.79 | 11.0 | 14.7 | 25.9 |
| 4 | 7.99 | 14.6 | 14.5 | 29.4 |
-
How we chose K. Each pass adds about 3.6 ms of drafter time. Up to K=3, the target part shrinks by the same amount, because the CPU enqueues the target operations while the drafter runs. At K=3 the target part is close to its floor (14.7 ms, vs. 14.5 ms at K=4). A fourth pass therefore adds 3.5 ms to the round for a 2.6% gain in τ, so the speedup stops rising after K=3 (5.83× vs. 5.26× for K=4). We will update Appendix C.1 with these measurements.
-
K in practice. A pass costs more as the batch grows (W2), and how much of it the round absorbs depends on the schedule. Passes that fit in the time overlapped with verification add no round time and should be used. Beyond that, the rule from W1 applies: add a pass while its relative gain in τ exceeds the relative round time it adds. K can be changed at inference without retraining.
-
Making K=3 work is part of our contribution. Training includes only the proposal pass and one edit, yet the second and third edits still raise τ (7.31 → 7.79 → 7.99). This is not a given: an edit can rewrite every position, including correct ones, and DFlash and Domino lose τ when run again on their own proposals (our response to Reviewer HMLz, W1). ProposalMix teaches the editor to keep correct tokens, so DEdit can iterate beyond what it was trained on, and K becomes a choice at inference.
Q: Dynamic K
TLDR — Yes, dynamic K can work, and we show a first result: a learned gate skips a third of the edits and keeps 98.5% of τ. Without scheduling designed for it, fixed K=3 still has higher τ and is faster. We see dynamic K with scheduling design as promising future work, where a rule's accuracy has to be traded against its decision time (up to 11% of a drafter pass).
| Rule (W=32) | Passes / round | τ | Decision cost, ms (% of a pass) |
|---|---|---|---|
| Fixed K=2 | 2 | 7.31 | — |
| Fixed K=3 | 3 | 7.79 | — |
| Confidence threshold (heuristic) | 2.30 | 7.39 | 0.13 (4%) |
| Learned gate | 2.37 | 7.67 | 0.41 (11%) |
-
A learned gate works better than a heuristic rule. We tried a heuristic rule (following standalone diffusion LMs such as DMax (Chen et al., 2026), it stops when the drafter is confident) and a learned gate, a small classifier on drafter features that decides whether to run the second edit. The learned gate works better: it skips a third of the edits and keeps 98.5% of τ, well above fixed K=2 at a similar number of passes.
-
Turning skipped edits into speed needs scheduling. How much time a skipped edit saves depends on how drafting and verification are scheduled. Our implementation has no scheduling designed for dynamic K, so fixed K=3 has higher τ and is also faster than every rule we tried. The decision has its own cost as well, 11% of a drafter pass for the learned gate. With a more accurate gate, and with drafting, the decision, and verification scheduled together, dynamic K can turn skipped edits into speed. Our response to Reviewer XauH (W1) gives the full comparison.
We will (1) add this analysis of the τ gain and the stopping-rule results to the appendix, (2) update Appendix C.1 with the per-pass timing above, (3) add the batch-size results to the appendix, and (4) discuss batch scheduling and dynamic K in Future Work, citing MineDraft and JetSpec++.
References
Tang et al., 2026. MineDraft: A Framework for Batch Parallel Speculative Decoding. ICML 2026.
Feng et al., 2026. JetSpec++: Foresight Parallel Drafting with Self Refinements. arXiv:2610.19792.
Cheng et al., 2026. DSpark: Confidence-Scheduled Speculative Decoding with Semi-Autoregressive Generation. arXiv:2607.05147.
Chen et al., 2026. DMax: Aggressive Parallel Decoding for dLLMs. arXiv:2604.08302.
Thanks for your timely and detailed response. It partially solves my concerns. I keep my score. Well done and good luck!
Thank you for your support and helpful questions. The batch-size and dynamic-K results they led to make DEdit's position among recent concurrent work clearer. We will add them to the revision.
We thank the reviewer for recognizing our thorough comparisons, the clear control of variables that "effectively validat[es] the benefits of the editing mechanism itself", and the simple architecture. Below we address the weaknesses and the question.
W1: τ is 14.8% higher than Domino's, but speedup only 7.9% higher
TLDR — DEdit raises τ most where τ is already high: +16% on the high-τ half of the responses and +8% on the low-τ half. The low-τ responses are the bottleneck: they need more rounds per token and take 64% of the decoding time, so the overall speedup follows them. The two extra drafter passes also make each round slightly longer.
We reran both drafters on the same prompts (seven benchmarks, 30% prompt subset) and split the responses into two halves by Domino's τ:
A two-response example. The time to generate a token is proportional to 1/τ, the number of rounds per token. Take two equally long responses with τ = 4 and τ = 10. The first needs 2.5× as many rounds and takes 71% of the total time. If τ rises by 20% on the second response only, the mean τ rises by 14%, but the total time falls by only 5%. Speedup therefore follows the low-τ responses, while the mean τ counts all responses equally.
When an additional edit outweighs its cost. An extra edit pays off when its relative gain in τ exceeds the relative round time it adds. In our setting, the second edit adds 6.6% to τ and 1.6% to round time (W3). The round time that a pass adds depends on how the system schedules drafting (W2).
W2: Larger batches and concurrent requests
TLDR — Batch size 1 is the standard setting for main results in the literature. Following your suggestion, we also measured larger static batches. τ is the same at every batch size. DEdit is the fastest at batch sizes 1 and 4. At batch sizes 16 and 32, GPU compute becomes the limit, and DEdit is slower than the baselines.
Our main results use batch size 1, as do the main evaluations of EAGLE-3, DMax, and Fast-dLLM. For this response we ran static batches that replicate each prompt on one GPU (seven benchmarks, 72 responses, same execution setting as the paper). Speedup is over AR at the same batch size.
τ holds at every batch size. τ stays at 7.1–7.2 from batch size 1 to 32. At batch sizes 1 and 4, K=3 is 21–23% faster than K=1 and 8–11% faster than Domino.
At batch sizes 16 and 32, GPU compute is the limit and DEdit is slower. The round then no longer absorbs the drafter passes (W3): the two edits add 10–18 ms per round, and K=3 is within 3% of K=1. DEdit at W=32 is also slower than Domino at W=16, because verifying a 32-token block costs more than verifying a 16-token block. Verifying only a confidence-selected prefix of the block, as DSpark does (Cheng et al., 2026), may recover this cost at wide windows.
Serving needs scheduling. These are static batches of identical prompts. A serving engine adds varied prompts and a scheduler, which can overlap drafting with verification (MineDraft, Tang et al., 2026; JetSpec++, Feng et al., 2026) and decide which requests get extra passes. DEdit contributes the drafter side of this problem: a drafter that iterates over a wide window and reaches a higher acceptance length than DFlash and Domino, with K chosen at inference without retraining. We see this co-design of editing and scheduling as important future work.