- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This work proposes dSpec, a speculative decoding method for block-diffusion language models such as LLaDA2.0-mini and SDAR-8B. It uses the target model’s own predictions on a fully masked block as the draft, and can optionally refine these predictions with a small one-layer head. It then runs one target forward pass that does two things concurrently. It verifies the draft from left to right, and it prepares the cache and the next block’s draft in case the whole block is accepted. When requests in a batch are rejected at different positions, their states are rebuilt together in one shared pass. Experiments in SGLang show clear throughput gains over confidence-threshold decoding across batch sizes, with similar average quality.
Critical strengths
The method is implemented in SGLang and test it at batch sizes from 1 to 128, which is closer to real serving settings, making the evaluation part comprehensive. In addition, the manuscript reports its results honestly. It gives a simple rule for when the learned head is worth its cost, and it admits that the head does not help on SDAR. The ablations also show clearly how much each part of the system contributes.
Critical weaknesses
(1) In terms of Appendix H, when the model checks a draft token, it can indirectly see that same token through the shared attention layout. As a result, the output is not guaranteed to match the target model. Lossless output is the main advantage of speculative decoding, so this is a serious limitation.
(2) TiDAR (arXiv:2511.08923) also verifies the current draft and drafts the next block from mask tokens in one forward pass with a structured attention mask. TiDAR also prepares drafts for every possible acceptance length and uses exact causal verification. The paper should cite TiDAR and explain which parts of folded verification are new.
(3) I find that the baseline LC1 decodes in confidence order, while dSpec verifies in left-to-right order. So dSpec does not reproduce the baseline’s output, even approximately. The paper reports that the average quality is similar, but some tasks drop clearly.
Questions for authors
I wanna know how much compute does training the Learned Draft head take? Given that the head improves throughput by only about 5% on LLaDA2 and slows down decoding on SDAR, this training cost should be justified.
3 comments
Follow-up Before the End of the Discussion Period
Dear Reviewer UsHD,
As the discussion period closeing, we would like to follow up on our response. The main change in the revision was prompted by your W1: dSpec now performs exact left-to-right verification by default. It commits the same tokens as greedy L2R decoding of the target, apart from BF16 near-ties, and remains 2.92x / 4.58x faster than LC1 at batch one without a trained head. The IFEval drop you noted came from the original shared layout and does not appear with the lossless layout. We have also added a comparison with TiDAR.
We hope this resolves your concern regarding losslessness. If so, we would be grateful if you could take it into account in your final rating.
Hi authors of submission 60266,
Thank you for the response and the additional experiments! The lossless layout addresses my concern about exactness, and the TiDAR discussion is helpful. However, given TiDAR, the novelty is limited. The Learned Draft head also adds little benefit for its training cost. In addition, the main results still use the approximate shared layout, and some per-task quality drops remain.
So I will keep my original score, and I encourage the authors to further improve this work and resubmit a stronger version in the future!
Thanks,
Reviewer UsHD
Thank you for the careful review, and for noting the SGLang evaluation at various batch sizes and the clear ablations. W1 and W3 both concern what dSpec's output should match. In the revision we implement an exact version of dSpec's left-to-right(L2R) verification.
W1. The shared layout does not guarantee the target's output
We agree. The shared layout approximates L2R verification over independent partial blocks (Appendix H). We implemented that exact form and now recommend it as the default. Each verified position gets its own copy of the block, and the copies do not attend to one another, so a candidate cannot influence its own verification. For any draft, the committed tokens are then those of the L2R decoding of the same target, one token at a time on the model's own block grid.
Both columns below use Self-Draft, with no training, so they differ only in the verification layout.
The lossless layout disagrees with the L2R decoding only where the two highest logits are within 0.75 (median gap 10-12), that is, at BF16 near-ties. Because its passes are wider, its gain shrinks at large batch, but it stays faster than LC1 at every batch size we tested. We keep the shared layout as an option for large batches and describe it as approximate.
W2. TiDAR
We now cite TiDAR in related work (Section 5). It is an important prior design, since it also verifies the current draft and pre-drafts the next block in one pass. The differences:
What is new in dSpec is how this pass is built for existing bidirectional targets, where prefixes cannot share representations and KV cannot be truncated: exact verification uses one block copy per verified position, and the next block's state is prepared in the same pass.
W3. LC1 commits in confidence order; dSpec verifies left to right
You are right that dSpec does not reproduce LC1. The revision now states the claim as follows: dSpec is lossless with respect to the L2R decoding defined above, and LC1 serves only as the speed baseline. We measured what this choice costs:
The greedy L2R decoding differs from LC1 mainly in the order of commitment, and on neither target is its quality difference from LC1 statistically significant. Following LC1's order exactly is possible but slower (0.461 vs 0.368 target passes per token), because the draft must then predict the unmasking order as well as the tokens (Appendix K.4). L2R order removes this constraint at comparable quality, and verification starts where the draft is most accurate (third row).
On the task-level drops you noticed: for LLaDA2 IFEval, Table 2 had 82.07 for LC1 and 78.62 for dSpec with the shared layout. In the new runs, the greedy L2R decoding gives 81.33 and the lossless layout 81.70, against 81.89 for LC1. The drop therefore does not come from the L2R order, but from the approximated shared layout, in which a candidate can influence its own verification and the next block starts right after each correction.
Q. Training cost of Learned Draft
About 1.3k (SDAR) and 0.75k (LLaDA2) H20 GPU-hours per head, for a gain of 1.03-1.04x on LLaDA2 and a slowdown on SDAR (0.92-0.97x). We agree that the cost is not justified, and all lossless results use Self-Draft, which needs no training. We keep the head as an optional component because it illustrates the cost criterion in Section 4.3: a longer accepted prefix helps only if it outweighs the added step time.
If the lossless layout address W1 and W3, we would appreciate it if you could reconsider your rating.