acceptodds
Under review as a conference paper at ICLR 2027

D-Loop: Looped Diffusion Drafting for Speculative Decoding

21 views

Abstract

Block diffusion accelerates speculative decoding by drafting multiple tokens in one forward pass. However, each position predicts a marginal distribution without observing earlier proposed tokens, limiting draft quality and acceptance length. We identify a concrete failure, the repetition trap, in which neighboring positions produce redundant copies of the same token. We explain this tendency theoretically and empirically examine its association with shorter accepted drafts. Recent methods refine marginal predictions with an additional causal head or a separately trained drafter, increasing parameter storage and introducing separate training objectives. We instead propose D-Loop, which introduces intra-block causal conditioning within the original diffusion drafter without additional model components. Inspired by semi-autoregressive generation and parameter sharing, D-Loop reuses the same backbone across looped passes. The first pass proposes a block, and the second conditions on a selected prefix to regenerate the suffix in parallel. A complementary prefix–suffix objective trains the shared drafter for both anchor-only prefix prediction and prefix-conditioned suffix prediction. Across eight math, code, and chat benchmarks, D-Loop can beat DFlash and DSpark on Qwen3-4B and Qwen3-8B with obvious gains.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Speculative Decoding, Looped Transformer, Diffusion Language Model

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_n7DyOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper improves diffusion-based drafting for speculative decoding of autoregressive language models. dloops identify a “repetition trap”: when draft positions predict tokens without observing earlier proposed tokens, neighboring positions can produce redundant repetitions, leading to early rejection. It addresses this through two passes of the same drafter. The first generates an initial token block. The second retains a selected prefix and regenerates the suffix conditioned on that prefix. A complementary prefix–suffix training objective prepares the shared model for both tasks. At inference, multiple candidate prefixes can be used to construct branches for joint target verification. Experiments on Qwen3-4B and Qwen3-8B ashows improvements in both acceptance length and decoding speed.

Critical strengths

The design that reuses existing parameters is simple and straightforward.

The repetition trap seems interesting.

Experiments show gains on speedups on qwen-4b and 8b models over dspark and dflash.

Critical weaknesses

Is it possible for the method to extend to even larger model like 32b? Experiments use two target sizes from the same Qwen3 family. It remains unclear whether the gains persist for larger targets.

Could you discuss and compare your method with dflash2 if possible?

Could you please discuss the situation when batch size > 1, and potentially in large batch settings, for the speedup gains. Will there be any obstacles/reduction in speeding up in such scenarios?

Could you discuss the resource utilization for using such drafting mechanisms during inference?

Questions for authors

see weakness

Reviewer_ZaViOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes D-Loop, a speculative decoding method for block diffusion drafters. The authors identify a “repetition trap”: because each position is predicted as an independent marginal, a draft block can contain adjacent duplicate tokens. D-Loop runs the same drafter twice. The first pass proposes a block; a confidence score selects top-K prefixes; the second pass regenerates the suffix conditioned on each retained prefix, and all branches are verified jointly. The shared drafter is trained with a complementary prefix–suffix objective. On Qwen3-4B and Qwen3-8B across eight math, code and chat benchmarks, D-Loop improves acceptance length and speedup over DFlash and DSpark.

Critical strengths
  1. The idea is simple and easy to integrate: it adds intra-block conditioning by reusing the existing drafter, with no extra parameters.
  2. The repetition trap is a clear, intuitive motivation, supported by draft-level statistics and a simple theoretical explanation.
  3. Acceptance length improves consistently across two model scales, three task domains, and both greedy and sampling decoding.
  4. The appendix gives a careful argument that verification is lossless, and the limitations section openly reports the case where the method does not help (greedy Alpaca).
Critical weaknesses
  1. Deployment overhead is understated. The second pass more than doubles drafting time (4.21 → 9.43 ms, Table 6), so a 15% gain in acceptance length over DSpark on Qwen3-4B (greedy) yields only a 2.2% speedup gain. With K = 4, D-Loop also verifies ~4× more tokens than chain-based baselines. This is nearly free only at batch size 1, the only setting evaluated; throughput at larger batch sizes should be reported.

  2. The advantage over DSpark is not explained. The theory compares D-Loop only to one-pass marginal drafting, yet DSpark's Markov head already conditions each position on the preceding token, which is the dependence behind the repetition trap. The gain may instead come from multi-branch verification or doubled drafting compute. Comparisons with K = 1 and with a budget-matched DSpark are needed.

  3. The comparison with D2SD is limited. D-Loop adopts D2SD’s prefix-scoring rule and prefix-conditioned suffix regeneration, so its contribution over D2SD reduces to weight sharing and the training objective. Its main practical advantage, memory savings, is likely modest. Both methods run two 5-layer drafter passes per cycle, so D-Loop only saves the weights of one small drafter, which is minor relative to the target model.

  4. Only block size 16 is evaluated. Long blocks favor D-Loop, since the second pass has a fixed cost but more suffix to repair. Practical deployments use shorter blocks (DSpark's production deployment caps the block size at 5), so results for smaller B are needed.

  5. Speedup measurements are inconsistent. In the same setting (Qwen3-8B, GSM8K, T = 1), D-Loop's speedup is 5.76× in Table 4 but 5.24× in Table 6. Speedup/τ is also nearly constant across datasets for the self-run methods, which suggests the speedups were computed from fixed latencies rather than measured. The hardware is not specified.

Questions for authors
  1. What is the throughput at larger batch sizes (e.g., 8, 32, 64)? How many tokens are verified per cycle with K = 4, are shared prefixes deduplicated, and what is the ratio of accepted to verified tokens?
  2. How does D-Loop with K = 1 compare to DSpark? How does DSpark perform with a matched verification budget (e.g., multiple branches or a draft tree)? What are DSpark's repetition rates under the statistics in Tables 1–2?
  3. Can you report acceptance length and speedup for smaller block sizes, at least B = 8?
  4. Were speedups measured end-to-end on each benchmark, or computed from Eq. 9 with fixed latencies? Why does D-Loop's speedup differ between Table 4 (5.76×) and Table 6 (5.24×) in the same setting? Which hardware and inference framework were used?
Reviewer_CuznOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
3(+1)
Read review
Paper summary

This paper identifies the repetition trap in block diffusion drafting, where neighboring positions generate redundant tokens due to the lack of intra-block causal conditioning. It proposes D-Loop, which reuses the same diffusion drafter for initial block generation and prefix-conditioned suffix regeneration, supported by a complementary prefix–suffix training objective. Experiments show improved acceptance length and decoding speed over DFlash and DSpark. Compared with published D²SD results, D-Loop achieves modest speedup gains without inducing additional model components, suggesting improved parameter efficiency.

Critical strengths
  1. Valuable problem identification: The paper identifies the repetition trap in block diffusion drafting and provides empirical evidence of its prevalence and association with shorter acceptance lengths.
  2. Parameter-efficient design: D-Loop reuses the same drafter for initial proposal and prefix-conditioned suffix regeneration, avoiding a separate drafter while enabling improved draft quality.
  3. Promising empirical results: Experiments across two model scales and eight benchmarks show improvements in acceptance length and decoding speed over DFlash and DSpark. Compared with published D²SD results, D-Loop also achieves modest speedup gains with fewer drafter layers.
Critical weaknesses
  1. Repetition reduction is not clear. The paper frames reducing repetition as a core motivation for D-Loop, but not reported the repetition rates under the evaluation settings.
  2. Where the gains come from is unclear. Table 5 compares several training objectives, but the individual contributions of looped inference, prefix selection, and the complementary prefix–suffix objective are never disentangled.
  3. The comparison against D²SD is not fair. The speedup over D²SD is already modest, and it's measured with fewer drafter layers. On top of that, the two are not evaluated under a unified setup,—it's hard to be convinced that the improvement is solid.
Questions for authors

NA

Comments on the paper

1 comment
AuthorsOpenReview ↗

General response

Dear AC,

We sincerely appreciate your time and effort in coordinating our submission. We also thank the reviewers for their constructive feedback and for recognizing the following strengths:

  • Well-motivated problem identification:\color{red}{\text{Well-motivated problem identification:}}

    “The paper identifies the repetition trap in block diffusion drafting and provides empirical evidence of its prevalence and association with shorter acceptance lengths.” (@Cuzn); “The repetition trap is a clear, intuitive motivation, supported by draft-level statistics and a simple theoretical explanation.” (@ZaVi); “The repetition trap seems interesting.” (@n7Dy).

  • Simple, parameter-efficient architecture and loss designs:\color{red}{\text{Simple, parameter-efficient architecture and loss designs:}}

    “D-Loop reuses the same drafter for initial proposal and prefix-conditioned suffix regeneration, avoiding a separate drafter while enabling improved draft quality.” (@Cuzn); “The idea is simple and easy to integrate: it adds intra-block conditioning by reusing the existing drafter, with no extra parameters.” (@ZaVi); “The design that reuses existing parameters is simple and straightforward.” (@n7Dy).

  • Consistent empirical improvements:\color{red}{\text{Consistent empirical improvements:}}

    “Experiments across two model scales and eight benchmarks show improvements in acceptance length and decoding speed over DFlash and DSpark.” (@Cuzn); “Acceptance length improves consistently across two model scales, three task domains, and both greedy and sampling decoding.” (@ZaVi).

During the discussion, we provided additional experiments and point-by-point responses. The main responses and their locations in the revised paper are summarized below.

  • Repetition reduction and component analysis (Cuzn, ZaVi)\color{blue}{\text{Repetition reduction and component analysis (Cuzn, ZaVi)}}

(1) Section 3.1, Table 2 reports repetition rates for DFlash, D-Loop, and DSpark. D-Loop reduces first-pass repetition by 73.7%-83.0% across three benchmarks.

(2) Section 4.2, Table 7 adds comparisons of inference schedules, prefix selection, and training objectives. The full looped schedule gives the largest acceptance gain.

  • Stronger baseline comparisons (Cuzn, ZaVi, n7Dy)\color{blue}{\text{Stronger baseline comparisons (Cuzn, ZaVi, n7Dy)}}

(1) Section 4.2, Table 5 clarifies that both D-Loop and D²SD execute two five-layer passes, but D-Loop shares one parameter set.

(2) Appendix B.3, Table 11 adds DFlash 2 comparisons: D-Loop improves acceptance on all three datasets, while DFlash 2 remains faster on Alpaca.

(3) Our responses also include D-Loop K=1 and exploratory DSpark branching comparisons. Against reproduced D²SD, mean acceptance improves by 9.7% at T=0 and 11.2% at T=1; reproduced speedups are explicitly labeled as estimates.

  • Smaller blocks and a larger target (ZaVi, n7Dy)\color{blue}{\text{Smaller blocks and a larger target (ZaVi, n7Dy)}}

(1) Appendix B.1, Table 9 adds B=8 results, with 21.9% higher mean acceptance at T=1.

(2) Appendix B.2, Table 10 extends evaluation to Qwen3-Coder-30B-A3B: mean acceptance improves by 25.7% at T=0 and 46.1% at T=1 over single-pass inference with the same checkpoint. Supplementary speedups are labeled as estimates.

  • Concurrency and resource costs (ZaVi, n7Dy)\color{blue}{\text{Concurrency and resource costs (ZaVi, n7Dy)}}

(1) Section 4.2, Table 8 evaluates 1-32 concurrent requests per GPU, showing 1.19-1.42x higher throughput than the single-pass control.

(2) We also report memory and latency costs, acknowledging that high concurrency reduces absolute throughput and that weight sharing does not remove runtime overhead.

  • Timing and implementation clarifications (ZaVi)\color{blue}{\text{Timing and implementation clarifications (ZaVi)}}

(1) Section 4 specifies the H800 hardware, inference framework, and main benchmark timing protocol.

(2) Table 6 distinguishes instrumented profiling from benchmark timing.

(3) Appendix B distinguishes supplementary speedup estimates from measurements using per-dataset AR baselines.

Our work identifies and analyzes the repetition trap and introduces shared-parameter looped refinement with complementary prefix-suffix training. The revised paper strengthens the empirical support while clarifying the conditions under which improved acceptance offsets the additional inference cost.

Thank you again for your time and consideration.

Best regards,
The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.