- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes S2D2, a training-free self-speculative decoding framework for block-diffusion language models. It exploits the observation that a block-diffusion model can operate autoregressively with block size one, allowing the same pretrained model to serve as both diffusion drafter and autoregressive verifier. S2D2 combines speculative rejection sampling, parallel verification masks, and lightweight routing policies to improve decoding efficiency while mitigating quality degradation under aggressive parallel decoding. Experiments across SDAR, Fast-dLLM v2, and LLaDA2.1 demonstrate improved accuracy–latency tradeoffs in several settings.
Critical strengths
- Practical reuse of pretrained block-diffusion models. S2D2 provides a concrete training-free implementation of diffusion drafting and autoregressive verification using shared model weights.
- Meaningful accuracy–efficiency improvements in challenging decoding regimes. The paper benefits multiple regimes of block diffusion models, from SDAR to Fast-dLLM v2 and EDLM and LLaDA2.1 mini.
- Useful supplementary analyses and explicit failure characterization. The paper investigates routing policies, alternative correction strategies, batched throughput, and sampling robustness.
Critical weaknesses
- The accuracy-speed trade-off is not consistently convincing across tasks. In Table 1, Diffusion (dynamic) serves as a practical baseline with relatively stable speedups and accuracy. Although S2D2 improves average results, some tasks suffer noticeable accuracy drops, e.g., HumanEval on SDAR-4B (73.8% → 68.3%) and SDAR-8B (78.0% → 74.4%) under config-B.
- The authors use the same dLLM checkpoints restricted to block_size=1 as the verifier. However, the AR mode is not necessarily a more reliable verifier than diffusion decoding, as also acknowleged in Table 8. The conditions under which AR verification is beneficial remain unclear.
- Block diffusion models could become compute-bound at large batch sizes. In this case, self-speculative decoding, where the drafter and verifier use the same full-sized model, could introduce substantial overhead and fail to provide speedup.
Questions for authors
- Beyond the training-free setting, what’s the key algorithmic difference between S2D2 and Nemotron-Labs-Diffusion [1], particularly in their AR verification and token acceptance mechanisms?
- What’s the actual invocation rate of verification for each model and configuration? 3.Could the authors explain why different routing policies are applied to different model families and setups? There’s no uniform routing policy, and the paper lacks an explanation of which policy particularly suits each model.
- Could the authors compare with other correction/speculative-decoding baselines, including AR-confidence correction, AR-diffusion agreement, and rejection without residual resampling, under the same verification budget? Although Table 8 includes a limited reject-only experiment, a broader comparison would help clarify the contribution of rejection sampling.
[1] Y. Fu et al., “Nemotron-Labs-Diffusion: A Tri-Mode Language Model Unifying Autoregressive, Diffusion, and Self-Speculation Decoding”.
5 comments
Q2: Invocation rate of verification. Results for the paper configurations, given as GSM8K / MBPP:
| Model · config | Verify rate¹ | Accepted tokens per verification | Per-token accept rate | Verifier share of forwards |
|---|---|---|---|---|
| SDAR-1.7B config-A (B4, min-span 2) | 0.88 / 0.90 | 2.9 / 3.3 | 0.88 / 0.92 | 0.33 / 0.32 |
| SDAR-1.7B (B16, always-on)² | 1.00 / 1.00 | 3.1 / 4.3 | 0.80 / 0.86 | 0.44 / 0.41 |
| SDAR-4B config-A (B16, always-on) | 1.00 / 1.00 | 4.0 / 4.6 | 0.84 / 0.87 | 0.43 / 0.41 |
| SDAR-8B config-A (B4, score) | 0.93 / 0.91 | 3.2 / 3.3 | 0.91 / 0.93 | 0.33 / 0.32 |
| SDAR-8B config-B (B16, always-on) | 1.00 / 1.00 | 4.2 / 4.4 | 0.85 / 0.86 | 0.43 / 0.41 |
| Fast-dLLM v2 config-A (SB4, hysteresis) | 0.96 / 0.99 | 3.4 / 3.4 | 0.96 / 0.96 | 0.46 / 0.47 |
| Fast-dLLM v2 config-B (SB16, hysteresis) | 0.99 / 1.00 | 6.8 / 6.4 | 0.92 / 0.91 | 0.44 / 0.44 |
| Fast-dLLM v2 config-C (SB32, min-span 8) | 0.71 / 0.66 | 8.8 / 8.1 | 0.91 / 0.90 | 0.36 / 0.35 |
| LLaDA2.1 quality | 0.71 / 0.66 | 9.5 / 11.5 | 0.92 / 0.94 | 0.36 / 0.34 |
| LLaDA2.1 conservative | 1.00 / 1.00 | 8.9 / 10.1 | 0.93 / 0.94 | 0.43 / 0.42 |
¹ Verifications per denoising step. ² B = 16 in place of the paper's B = 32.
- How often verification runs. It runs at 88–100% of steps in most configurations, and at 66–71% for Fast-dLLM v2 config-C and LLaDA2.1 quality.
- Where the speedup comes from: long accepted spans, not skipped verification. Each verification accepts 3–11 tokens, which more than amortizes the verifier's 32–47% share of forwards.
- Correction to the paper. Under its routing parameters, the LLaDA2.1 "conservative" setting is always-on by construction (rate 1.00); it is the "quality" setting that skips verification. We will correct the description.
Q3: Why different routing policies per family? Please see C3. The choice mainly reflects how expensive verification is relative to the work it saves:
- SDAR at B = 16. Verification almost always pays off, so it runs always-on.
- Fast-dLLM v2 at SB = 32. Verifying short spans is costly, so we use min-span routing.
- LLaDA2.1. Its token editing already corrects many errors, so the cost penalty is higher.
Tables 13–14 show S2D2 is robust to this choice at large blocks, and always-on is a strong default.
Q4: Other correction rules under the same verification budget. Thank you for this insightful suggestion. The main goal of S2D2 is to show that the self-speculative framework works: a block-diffusion model can draft for, and be verified by, its own AR mode, with routing deciding when verification pays off. We therefore instantiated it with the standard speculative rejection-sampling rule and focused our ablations on when to verify (Tables 13–17). Other verification and correction rules from the speculative-decoding literature plug into the same framework, reusing the same verifier forward and routing. As with the D²SD extension (Y4TA), we ran this study for the rebuttal.
Setup. All rules share the drafting, the verifier forward and routing. Routing is always-on, so every step is verified exactly once; only the accept/commit rule differs:
- S2D2: accept with min(1, q/p); at the first rejection, resample from the residual (q − p)+.
- Resample from q: same acceptance, but resample from q instead of the residual.
- Exact SD with a deterministic draft: treat the draft as a point mass, i.e., accept with probability q(x̂); at rejection, sample from q with x̂ excluded.
- Agreement → argmax q: accept while argmax q = x̂; at the first disagreement, commit argmax q (standard greedy verification).
Results on 500 GSM8K examples and full MBPP (500), 3 seeds, on the Table 1–3 models. Entries are accuracy GSM8K / MBPP (change in forward passes vs. S2D2):
| Commit rule | SDAR-8B (B=16) | Fast-dLLM v2 (SB=32) | LLaDA2.1-Mini (quality) |
|---|---|---|---|
| S2D2 (residual resampling) | 86.0±1.4 / 61.1±1.3 | 84.1±1.5 / 45.8±1.1 | 89.4±0.8 / 64.5±0.4 |
| Resample from q | 86.3±1.1 / 61.7±1.4 (−1%) | 83.8±1.2 / 45.7±0.1 (−1%) | 88.5±0.5 / 64.5±0.1 (−3%) |
| Exact SD, deterministic draft | 84.8±1.0 / 51.5±1.0 (+21%) | 83.7±0.8 / 39.7±0.4 (+27%) | 86.1±1.5 / 64.1±1.3 (+63%) |
| Agreement → argmax q | 85.6±0.0 / 65.2† (0%) | 84.8† / 52.8† (−1%) | 91.0† / 65.0† (−5%) |
† Deterministic (greedy decoding), so a single run is exact.
- Ratio-based acceptance matters. Treating the draft as deterministic roughly halves the accepted length (e.g., 8.1 → 4.5 tokens per verification on LLaDA2.1 GSM8K). It costs 11–66% more forward passes and lowers accuracy (up to −9.6 on SDAR MBPP). Accepting with min(1, q/p), using the drafter's own probability, is what makes self-verification cheap.
- Resampling from q is equivalent to the residual in our settings: within seed noise on all six model–task pairs, at the same cost.
- Correcting immediately matters for speed. In a smaller preliminary study (200 + 200 examples), rules that hand rejected positions back to diffusion (re-masking, AR-confidence gating) reached similar accuracy but needed 14–92% more forward passes.
- Committing the AR token at the first rejection is a promising extension. Agreement → argmax q is the standard greedy-verification variant of speculative decoding (and the exact-match rule of Nemotron-Labs-Diffusion). It costs the same as S2D2, ties on GSM8K, and improves MBPP by +4.1 (SDAR-8B) and +7.0 (Fast-dLLM v2). Both of these settings use greedy decoding, where committing the AR model's greedy token is the correction consistent with the greedy target.
- Complementary to S2D2. These rules plug into S2D2's verification and routing without changing them. Better correction rules from the speculative-decoding literature can therefore strengthen S2D2's results further, without changing its claims or findings.
We will include this study in the revision. We thank the reviewer for this very helpful suggestion.
References
- Nemotron-Labs-Diffusion: Fu et al. Nemotron-Labs-Diffusion: A Tri-Mode Language Model Unifying Autoregressive, Diffusion, and Self-Speculation Decoding. arXiv:2607.05722, 2026.
- SSD: Gao et al. Self Speculative Decoding for Diffusion Large Language Models. arXiv:2510.04147, 2025.
- D2F: Wang et al. Diffusion LLMs Can Do Faster-Than-AR Inference via Discrete Diffusion Forcing. arXiv:2508.09192, 2025.
We thank the reviewer for the detailed review and excellent questions.
W1: The trade-off is not consistent across tasks. Please see C3 and our response to nh9s W3.
W2: The AR mode may not be a reliable verifier. Please see our response to RK1a:
W3: Compute-bound regime at large batch sizes. This concern shaped S2D2's design.
Q1: Difference from Nemotron-Labs-Diffusion. Both draft with the model's diffusion mode and verify with its AR mode, left to right. The differences: