- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
DLoop lets one target-model verification cover multiple drafting stages. A log-probability gate decides after each stage whether to keep drafting, and later stages condition on the drafter’s own hidden states for unverified tokens. A one-epoch “loop-aware” fine-tune teaches released drafters to use these states. Across EAGLE-3, DFlash, Domino, DSpark, and MTP modules, the paper reports 5–41% speedup gains under greedy decoding while remaining lossless.
Critical strengths
- The method is simple, lossless, and introduces no additional parameters. Unlike existing adaptive draft length methods, it also benefits parallel drafters.
- The evaluation covers a wide range of drafters and target models. The analysis in Fig. 3, Table 4, and Table A4 clearly identifies the source of the speedup.
- The ablation in Table 2 shows that loop-aware training and the drafting loop are complementary. Training alone has little effect and the loop alone mainly benefits autoregressive drafters, while their combination improves all settings.
Critical weaknesses
- The proposed method only works under low concurrencies and is not practical for large-batch serving. In real-world serving, the draft block size is usually 4 or 8 because of the non-negligible verification cost. In this setting, doubling the verification length would not benefit serving throughput.
- No comparison with drafters trained for a larger block size. DLoop verifies up to S·k tokens per verification. The obvious alternative is to fine-tune the same drafter with block size 2k or 3k, which would be compute-comparable to the one-epoch loop-aware training. Especially, for parallel drafters, DLoop’s drafting cost grows nearly linearly with S (Table A4: DFlash 3.66 → 13.96 ms for S=1–4), whereas a 2k block takes a single forward pass. Without this baseline, it is unclear whether the gains come from looping or simply from drafting more tokens with a drafter trained for that length.
- The motivation and headline results are greedy-only. Fig. 1 is measured at T=0. At T=1 (Table A10), gains shrink (DSpark on Qwen3-4B: +26.3% → +16.3%), and Domino on Qwen3-8B gets slightly slower (3.06× → 3.04×). The abstract should state the decoding setting, and the paper should report full acceptance at T=1.
Questions for authors
- How does DLoop compare with drafters fine-tuned for 2k or 3k block sizes, or with a larger draft depth for EAGLE-3 and MTP?
- How does a fixed S=2 or S=3 perform without the gate?
- What is the full acceptance ratio at T=1?
5 comments
Response to Reviewer Lgqu (Part2)
W2 & Q1: Comparison with drafters trained for larger blocks or deeper drafts
Thank you for suggesting this baseline. The new results indicate that the gain of DLoop does not come simply from drafting more tokens, since drafters fine-tuned for and blocks stay below + DLoop in speedup, and DLoop further improves both of them.
Table R3. Acceptance length and speedup ( / spd×) of DSpark trained with block sizes , , and , with and without DLoop, using Qwen3-8B at .
| Setup | Math | Code | Chat | Avg |
|---|---|---|---|---|
| 6.32 / 4.46× | 5.81 / 3.96× | 4.13 / 2.61× | 5.58 / 3.81× | |
| + DLoop | 11.80 / 6.20× | 8.84 / 4.83× | 5.39 / 2.74× | 9.08 / 4.82× |
| 8.14 / 5.49× | 7.12 / 4.73× | 4.51 / 2.59× | 6.85 / 4.48× | |
| + DLoop | 11.58 / 6.57× | 8.42 / 5.02× | 5.27 / 2.62× | 8.82 / 5.00× |
| 9.28 / 5.76× | 7.52 / 4.74× | 4.82 / 2.54× | 7.51 / 4.57× | |
| + DLoop | 11.80 / 6.78× | 8.72 / 5.15× | 5.06 / 2.57× | 8.96 / 5.12× |
Training for a larger block does not replace the loop. For Table R3, we fine-tune the -block DSpark for one more epoch with a or block, following the training recipe released with DSpark, as in loop-aware training, so that the training compute matches that of DLoop. Training for a larger block improves both the acceptance length and the speedup over the -block drafter on average. We agree that a or block is drafted in a single forward pass, whereas each DLoop stage adds a drafter pass (Table A6). However, neither the nor the drafter reaches the speedup of + DLoop. A larger block adds positions that are predicted farther from the verified context, and these positions are accepted less often, so the acceptance length grows much less than the number of drafted tokens.
Applying DLoop on top of the and drafters further improves the speedup, which indicates that the gain of DLoop goes beyond drafting more tokens with a drafter trained for that length. The gate adds a drafting stage only after a confident block, so DLoop verifies more tokens per target forward pass without spending drafting stages on blocks that are later discarded.
Table R4. Acceptance length and speedup ( / spd×) of EAGLE-3 with Qwen3-8B and MTP with Qwen3.5-9B for different draft depths at . "+ Loop only" enables the confidence-gated drafting loop, and "+ DLoop" also applies loop-aware training. Bold marks the best speedup in each column for each draft model.
| Method | Math | Code | Chat | Avg. |
|---|---|---|---|---|
| EAGLE-3 (Qwen3-8B) | ||||
| Depth 7 (default) | 7.59 / 3.78× | 6.73 / 3.30× | 4.98 / 2.26× | 6.61 / 3.22× |
| Depth 10 | 8.78 / 3.88× | 7.32 / 3.20× | 5.22 / 1.99× | 7.34 / 3.15× |
| Depth 14 | 9.23 / 3.54× | 7.38 / 2.83× | 5.30 / 1.75× | 7.56 / 2.82× |
| Depth 7 + Loop only | 8.97 / 4.10× | 7.16 / 3.37× | 5.14 / 2.33× | 7.34 / 3.38× |
| Depth 14 + Loop only | 10.88 / 3.82× | 8.05 / 3.07× | 5.59 / 1.94× | 8.50 / 3.07× |
| Depth 7 + DLoop | 10.45 / 4.31× | 7.83 / 3.46× | 5.46 / 2.38× | 8.22 / 3.51× |
| MTP (Qwen3.5-9B) | ||||
| Depth 3 (default) | 3.54 / 1.74× | 3.39 / 1.58× | 2.88 / 1.21× | 3.32 / 1.55× |
| Depth 4 | 4.15 / 1.90× | 3.93 / 1.72× | 3.18 / 1.27× | 3.82 / 1.68× |
| Depth 5 | 4.64 / 1.97× | 4.35 / 1.79× | 3.41 / 1.25× | 4.23 / 1.72× |
| Depth 6 | 5.05 / 2.02× | 4.69 / 1.80× | 3.56 / 1.22× | 4.54 / 1.74× |
| Depth 3 + Loop only | 5.66 / 2.10× | 5.17 / 1.86× | 3.57 / 1.27× | 4.96 / 1.80× |
| Depth 6 + Loop only | 5.90 / 2.07× | 5.50 / 1.87× | 3.83 / 1.24× | 5.23 / 1.79× |
| Depth 3 + DLoop | 8.26 / 2.66× | 6.92 / 2.26× | 4.02 / 1.33× | 6.70 / 2.18× |
A larger draft depth shows the same trend. Table R4 increases the draft depth of EAGLE-3 and MTP and applies the loop at both the default and a larger depth. A larger depth raises the acceptance length, but the speedup does not rise accordingly, since the additional drafting steps are spent in every drafting stage, including those whose later draft tokens are rejected. The gate instead adds a drafting stage only after a confident block, and DLoop at the default depth gives the highest speedup for both draft models. Even on top of a larger depth and without loop-aware training, the loop still raises the speedup, which indicates that the gain comes from looping rather than from drafting more tokens.
We have added Tables R3 and R4 and this discussion to Appendix A.10.1.
Response to Reviewer Lgqu (Part3)
Q2: Fixed number of drafting stages versus the gate
Thank you for this question. A fixed raises the acceptance length but lowers the speedup in most settings, whereas the gate gives the highest speedup for every draft model in Table R5.
Table R5. Fixed versus the gate. All rows use the same loop-aware trained drafter and differ only in how is set. Average / speedup over the eight benchmarks on Qwen3-8B at .
| Method | EAGLE-3 | DFlash | Domino | DSpark |
|---|---|---|---|---|
| (no loop) | 6.60 / 3.15× | 6.48 / 4.33× | 7.23 / 4.52× | 5.50 / 3.77× |
| (fixed) | 8.89 / 3.21× | 7.30 / 4.14× | 8.25 / 4.07× | 7.45 / 4.25× |
| (fixed) | 9.72 / 2.77× | 7.53 / 3.75× | 8.55 / 3.54× | 8.45 / 4.20× |
| DLoop (gate) | 8.22 / 3.51× | 7.40 / 4.71× | 8.29 / 4.76× | 9.08 / 4.82× |
With a fixed , every loop performs the additional drafting stages even after an uncertain block, and a rejection in an earlier block discards all later blocks. The acceptance length therefore increases with , but the cost of the additional drafting stages outweighs this gain, and the speedup falls below that of already at for DFlash and Domino and at for EAGLE-3. DSpark gains from a fixed , but its gain stays well below that of the gate. The gate instead adds a drafting stage only after a confident block, which avoids the drafting stages that a fixed spends on blocks that are later discarded.
We have added Table R5 and this discussion to Appendix A.10.2.
W3 & Q3: Results at T=1 and full acceptance ratio
Thank you for pointing this out. The full acceptance ratio at is lower than at for every draft model in Table R6, both with and without tree verification, consistent with the lower acceptance length of the draft models themselves at in Table A17.
Table R6. Full acceptance ratio (%) at / , averaged over the benchmarks in each task category and over all eight benchmarks (Avg.). Tree verification of EAGLE-3 is part of its original recipe.
| Draft model | Math | Code | Chat | Avg. |
|---|---|---|---|---|
| Without tree verification | ||||
| EAGLE-3 (Qwen3-4B) | 39.4 / 26.7 | 28.2 / 21.0 | 8.1 / 5.8 | 27.4 / 19.3 |
| DFlash (Qwen3-4B) | 13.6 / 7.9 | 13.4 / 7.3 | 1.2 / 1.1 | 10.4 / 6.0 |
| Domino (Qwen3-4B) | 19.6 / 8.9 | 12.6 / 4.8 | 1.8 / 0.8 | 12.5 / 5.3 |
| DSpark (Qwen3-4B) | 55.8 / 45.9 | 44.3 / 36.2 | 15.6 / 12.9 | 41.4 / 34.0 |
| EAGLE-3 (Qwen3-8B) | 32.4 / 22.8 | 20.4 / 13.9 | 5.7 / 3.9 | 21.2 / 14.8 |
| DFlash (Qwen3-8B) | 14.6 / 8.3 | 12.4 / 7.8 | 1.1 / 0.8 | 10.4 / 6.3 |
| Domino (Qwen3-8B) | 21.6 / 9.2 | 12.3 / 5.1 | 2.3 / 0.9 | 13.3 / 5.6 |
| DSpark (Qwen3-8B) | 57.5 / 46.1 | 46.6 / 39.1 | 16.0 / 13.0 | 43.0 / 35.2 |
| With tree verification | ||||
| EAGLE-3 (Qwen3-4B) | 68.0 / 51.9 | 52.6 / 44.8 | 18.3 / 14.4 | 49.8 / 39.8 |
| DFlash + DDTree (Qwen3-4B) | 23.2 / 15.7 | 22.1 / 16.4 | 2.8 / 2.3 | 17.7 / 12.6 |
| Domino + DARTree (Qwen3-4B) | 31.8 / 19.7 | 20.9 / 12.8 | 3.1 / 2.3 | 20.5 / 12.8 |
| DSpark + DARTree (Qwen3-4B) | 67.3 / 52.7 | 62.5 / 51.1 | 25.1 / 19.6 | 54.9 / 43.8 |
| EAGLE-3 (Qwen3-8B) | 61.2 / 46.1 | 44.8 / 34.5 | 13.4 / 10.3 | 43.1 / 32.8 |
| DFlash + DDTree (Qwen3-8B) | 23.5 / 15.4 | 22.5 / 13.9 | 2.5 / 1.8 | 17.8 / 11.5 |
| Domino + DARTree (Qwen3-8B) | 35.2 / 20.0 | 21.9 / 13.6 | 3.3 / 2.3 | 22.3 / 13.2 |
| DSpark + DARTree (Qwen3-8B) | 71.6 / 56.9 | 64.9 / 54.5 | 28.3 / 21.7 | 58.3 / 47.2 |
A lower full acceptance ratio leaves fewer blocks after which an additional drafting stage can be accepted, which is consistent with the smaller gain at . Without tree verification, Domino has the lowest full acceptance ratio at for both target models. However, tree verification raises the full acceptance ratio of every draft model in Table R6. With tree verification, DLoop raises the speedup at for every draft model in Table A17, including Domino on Qwen3-8B from to .
We have stated the decoding setting in the abstract and added Table R6 to Appendix A.3.
Response to Reviewer Lgqu (Part1)
Table, figure, section, and appendix numbers of the paper in our responses refer to the revised manuscript.
W1: Practicality in large-batch serving
Thank you for raising this practical concern. Limiting the loop depth of each request recovers the gain of DLoop at high concurrency, and DLoop then improves throughput over the draft model at every concurrency up to c=32, including DSpark with k=7 and MTP with k=3. As discussed in Section 4.2, DLoop chooses the number of drafting stages independently for each request, which leads to different loop depths within a batch. We agree that these different depths, together with the additional speculative work on a saturated target model, reduce the gain of the loop in Table 6 at high concurrency. However, simply capping the loop depth of each request at a maximum Smax reduces both effects.
Our new experiment 1. Bounded loop depth. We cap the loop depth at Smax=3 and measure DFlash and DSpark in SGLang under the same setting.
Table R1. Results in SGLang with Qwen3-8B (throughput in tokens(tok)/s and speedup over autoregressive decoding). Bold marks results above the draft model.
With Smax=3, DLoop exceeds the draft model at every concurrency for both draft models and both benchmarks. At high load, a smaller Smax keeps the loop depths within a batch close and limits the speculative work on the saturated target model.
Our new experiment 2. To cover the small block sizes that the reviewer mentions, we also measure DLoop in SGLang with the MTP module of Qwen3.5-9B, which drafts k=3 tokens per drafting stage. With Smax=3, one verification covers at most 9 draft tokens.
Table R2. Results in SGLang with the MTP module of Qwen3.5-9B (k=3). The notation follows Table R1.
With Smax=3, DLoop exceeds MTP at every concurrency on both benchmarks. At c=32, it raises the throughput from 5,920 to 6,227 tok/s on MATH-500 and from 4,383 to 4,582 tok/s on HumanEval. These results suggest that adjusting the loop depth of each request with the load keeps DLoop beneficial in large-batch serving with small draft blocks, in line with the shorter optimal speculation length at high load (Su et al., 2023; Liu et al., 2024b; Li et al., 2025; Cheng et al., 2026).
We have added this discussion and the new results to Section 4.2 of the main paper (Table 6).