acceptodds
Under review as a conference paper at ICLR 2027

S2D2: Fast Decoding for Diffusion LLMs via Training-Free Self-Speculation

Abstract

Block-diffusion language models offer a promising path toward faster-than-autoregressive generation by combining block-wise autoregressive decoding with within-block parallel denoising. However, in the few-step regime needed for practical acceleration, standard confidence-thresholded decoding is often brittle: aggressive thresholds hurt quality, while conservative thresholds require unnecessary denoising steps. Existing approaches that address this issue either require additional training or incur extra test-time compute. We present S2D2, a training-free self-speculative decoding framework for block-diffusion language models. Our key observation is that a block-diffusion model becomes autoregressive when the block size is reduced to one, allowing the same pretrained model to act as both drafter and verifier. S2D2 inserts a speculative verification step into standard block-diffusion decoding and uses lightweight routing policies to decide when verification is worth its cost. This yields a hybrid decoding trajectory in which the autoregressive mode acts as a local sequence-level critic, without claiming global AR distribution preservation. Across three mainstream block-diffusion families, S2D2 consistently improves the accuracy–speed tradeoff over strong confidence-thresholding baselines. On SDAR, we observe up to speedup over autoregressive decoding, and approximately over a tuned dynamic decoding baseline while improving accuracy by points. On LLaDA2.1-Mini, S2D2 remains complementary to built-in self-correction, including a conservative setting where it is faster than the static baseline with comparable accuracy.

Primary area:
foundation or frontier models, including LLMs
Keywords:
diffusion language model, efficient inference, speculative decoding

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 6 reviews, above 51% of the others.
Reviewer_yKG4OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes S2D2, a training-free self-speculative decoding framework for block-diffusion language models. It exploits the observation that a block-diffusion model can operate autoregressively with block size one, allowing the same pretrained model to serve as both diffusion drafter and autoregressive verifier. S2D2 combines speculative rejection sampling, parallel verification masks, and lightweight routing policies to improve decoding efficiency while mitigating quality degradation under aggressive parallel decoding. Experiments across SDAR, Fast-dLLM v2, and LLaDA2.1 demonstrate improved accuracy–latency tradeoffs in several settings.

Critical strengths
  1. Practical reuse of pretrained block-diffusion models. S2D2 provides a concrete training-free implementation of diffusion drafting and autoregressive verification using shared model weights.
  2. Meaningful accuracy–efficiency improvements in challenging decoding regimes. The paper benefits multiple regimes of block diffusion models, from SDAR to Fast-dLLM v2 and EDLM and LLaDA2.1 mini.
  3. Useful supplementary analyses and explicit failure characterization. The paper investigates routing policies, alternative correction strategies, batched throughput, and sampling robustness.
Critical weaknesses
  1. The accuracy-speed trade-off is not consistently convincing across tasks. In Table 1, Diffusion (dynamic) serves as a practical baseline with relatively stable speedups and accuracy. Although S2D2 improves average results, some tasks suffer noticeable accuracy drops, e.g., HumanEval on SDAR-4B (73.8% → 68.3%) and SDAR-8B (78.0% → 74.4%) under config-B.
  2. The authors use the same dLLM checkpoints restricted to block_size=1 as the verifier. However, the AR mode is not necessarily a more reliable verifier than diffusion decoding, as also acknowleged in Table 8. The conditions under which AR verification is beneficial remain unclear.
  3. Block diffusion models could become compute-bound at large batch sizes. In this case, self-speculative decoding, where the drafter and verifier use the same full-sized model, could introduce substantial overhead and fail to provide speedup.
Questions for authors
  1. Beyond the training-free setting, what’s the key algorithmic difference between S2D2 and Nemotron-Labs-Diffusion [1], particularly in their AR verification and token acceptance mechanisms?
  2. What’s the actual invocation rate of verification for each model and configuration? 3.Could the authors explain why different routing policies are applied to different model families and setups? There’s no uniform routing policy, and the paper lacks an explanation of which policy particularly suits each model.
  3. Could the authors compare with other correction/speculative-decoding baselines, including AR-confidence correction, AR-diffusion agreement, and rejection without residual resampling, under the same verification budget? Although Table 8 includes a limited reject-only experiment, a broader comparison would help clarify the contribution of rejection sampling.

[1] Y. Fu et al., “Nemotron-Labs-Diffusion: A Tri-Mode Language Model Unifying Autoregressive, Diffusion, and Self-Speculation Decoding”.

5 comments

We thank the reviewer for the detailed review and excellent questions.

W1: The trade-off is not consistent across tasks. Please see C3 and our response to nh9s W3.

  • Config-A and config-B are intentionally quality-oriented and speed-oriented.
  • On SDAR-8B, config-B is better on both axes.
  • At B = 16 the gains are far above seed noise (+8.5 / +13.3; C4).
  • We acknowledge the HumanEval drops and will discuss them.

W2: The AR mode may not be a reliable verifier. Please see our response to RK1a:

  • S2D2 uses the AR mode only locally, routed per step.
  • AR is the most accurate decoder for most models.
  • Per-token acceptance rates are 0.80–0.96.
  • Table 8 characterizes the failure case (D2F).

W3: Compute-bound regime at large batch sizes. This concern shaped S2D2's design.

  • No batched hypothetical states. Unlike SSD-style methods, S2D2 does not batch hypothetical states and uses a single extra forward.
  • Right-shifted models (Fast-dLLM v2). Verification is a causal forward over the same span, with no extra width.
  • Batched throughput (Tables 5–6, up to bs = 64). S2D2 improves Fast-dLLM v2 throughput by 1.13× (GSM8K) and 1.49× (MBPP) at bs = 64.
  • SDAR. The 2L verifier's extra width makes throughput on par with BD3 at bs = 64, while MBPP accuracy stays far higher (0.70 vs. 0.40). We will state this limitation explicitly.

Q1: Difference from Nemotron-Labs-Diffusion. Both draft with the model's diffusion mode and verify with its AR mode, left to right. The differences:

  • Training. Nemotron-Labs-Diffusion trains a tri-mode model. S2D2 is training-free on existing checkpoints, which requires verifier constructions for position-aligned (2L mask) and right-shifted models.
  • Acceptance. Nemotron-Labs-Diffusion accepts the draft prefix that exactly matches the AR predictions. S2D2 uses probabilistic acceptance, min(1, q/p), with residual resampling.
  • When to verify. Nemotron-Labs-Diffusion verifies every cycle, and its decoding mode is chosen per deployment. S2D2's routing policy decides per step whether to verify or to commit by confidence-based diffusion decoding.

Q2: Invocation rate of verification. Results for the paper configurations, given as GSM8K / MBPP:

Model · configVerify rate¹Accepted tokens per verificationPer-token accept rateVerifier share of forwards
SDAR-1.7B config-A (B4, min-span 2)0.88 / 0.902.9 / 3.30.88 / 0.920.33 / 0.32
SDAR-1.7B (B16, always-on)²1.00 / 1.003.1 / 4.30.80 / 0.860.44 / 0.41
SDAR-4B config-A (B16, always-on)1.00 / 1.004.0 / 4.60.84 / 0.870.43 / 0.41
SDAR-8B config-A (B4, score)0.93 / 0.913.2 / 3.30.91 / 0.930.33 / 0.32
SDAR-8B config-B (B16, always-on)1.00 / 1.004.2 / 4.40.85 / 0.860.43 / 0.41
Fast-dLLM v2 config-A (SB4, hysteresis)0.96 / 0.993.4 / 3.40.96 / 0.960.46 / 0.47
Fast-dLLM v2 config-B (SB16, hysteresis)0.99 / 1.006.8 / 6.40.92 / 0.910.44 / 0.44
Fast-dLLM v2 config-C (SB32, min-span 8)0.71 / 0.668.8 / 8.10.91 / 0.900.36 / 0.35
LLaDA2.1 quality0.71 / 0.669.5 / 11.50.92 / 0.940.36 / 0.34
LLaDA2.1 conservative1.00 / 1.008.9 / 10.10.93 / 0.940.43 / 0.42

¹ Verifications per denoising step. ² B = 16 in place of the paper's B = 32.

  • How often verification runs. It runs at 88–100% of steps in most configurations, and at 66–71% for Fast-dLLM v2 config-C and LLaDA2.1 quality.
  • Where the speedup comes from: long accepted spans, not skipped verification. Each verification accepts 3–11 tokens, which more than amortizes the verifier's 32–47% share of forwards.
  • Correction to the paper. Under its routing parameters, the LLaDA2.1 "conservative" setting is always-on by construction (rate 1.00); it is the "quality" setting that skips verification. We will correct the description.

Q3: Why different routing policies per family? Please see C3. The choice mainly reflects how expensive verification is relative to the work it saves:

  • SDAR at B = 16. Verification almost always pays off, so it runs always-on.
  • Fast-dLLM v2 at SB = 32. Verifying short spans is costly, so we use min-span routing.
  • LLaDA2.1. Its token editing already corrects many errors, so the cost penalty is higher.

Tables 13–14 show S2D2 is robust to this choice at large blocks, and always-on is a strong default.

Q4: Other correction rules under the same verification budget. Thank you for this insightful suggestion. The main goal of S2D2 is to show that the self-speculative framework works: a block-diffusion model can draft for, and be verified by, its own AR mode, with routing deciding when verification pays off. We therefore instantiated it with the standard speculative rejection-sampling rule and focused our ablations on when to verify (Tables 13–17). Other verification and correction rules from the speculative-decoding literature plug into the same framework, reusing the same verifier forward and routing. As with the D²SD extension (Y4TA), we ran this study for the rebuttal.

Setup. All rules share the drafting, the verifier forward and routing. Routing is always-on, so every step is verified exactly once; only the accept/commit rule differs:

  • S2D2: accept with min(1, q/p); at the first rejection, resample from the residual (q − p)+.
  • Resample from q: same acceptance, but resample from q instead of the residual.
  • Exact SD with a deterministic draft: treat the draft as a point mass, i.e., accept with probability q(x̂); at rejection, sample from q with x̂ excluded.
  • Agreement → argmax q: accept while argmax q = x̂; at the first disagreement, commit argmax q (standard greedy verification).

Results on 500 GSM8K examples and full MBPP (500), 3 seeds, on the Table 1–3 models. Entries are accuracy GSM8K / MBPP (change in forward passes vs. S2D2):

Commit ruleSDAR-8B (B=16)Fast-dLLM v2 (SB=32)LLaDA2.1-Mini (quality)
S2D2 (residual resampling)86.0±1.4 / 61.1±1.384.1±1.5 / 45.8±1.189.4±0.8 / 64.5±0.4
Resample from q86.3±1.1 / 61.7±1.4 (−1%)83.8±1.2 / 45.7±0.1 (−1%)88.5±0.5 / 64.5±0.1 (−3%)
Exact SD, deterministic draft84.8±1.0 / 51.5±1.0 (+21%)83.7±0.8 / 39.7±0.4 (+27%)86.1±1.5 / 64.1±1.3 (+63%)
Agreement → argmax q85.6±0.0 / 65.2† (0%)84.8† / 52.8† (−1%)91.0† / 65.0† (−5%)

† Deterministic (greedy decoding), so a single run is exact.

  • Ratio-based acceptance matters. Treating the draft as deterministic roughly halves the accepted length (e.g., 8.1 → 4.5 tokens per verification on LLaDA2.1 GSM8K). It costs 11–66% more forward passes and lowers accuracy (up to −9.6 on SDAR MBPP). Accepting with min(1, q/p), using the drafter's own probability, is what makes self-verification cheap.
  • Resampling from q is equivalent to the residual in our settings: within seed noise on all six model–task pairs, at the same cost.
  • Correcting immediately matters for speed. In a smaller preliminary study (200 + 200 examples), rules that hand rejected positions back to diffusion (re-masking, AR-confidence gating) reached similar accuracy but needed 14–92% more forward passes.
  • Committing the AR token at the first rejection is a promising extension. Agreement → argmax q is the standard greedy-verification variant of speculative decoding (and the exact-match rule of Nemotron-Labs-Diffusion). It costs the same as S2D2, ties on GSM8K, and improves MBPP by +4.1 (SDAR-8B) and +7.0 (Fast-dLLM v2). Both of these settings use greedy decoding, where committing the AR model's greedy token is the correction consistent with the greedy target.
  • Complementary to S2D2. These rules plug into S2D2's verification and routing without changing them. Better correction rules from the speculative-decoding literature can therefore strengthen S2D2's results further, without changing its claims or findings.

We will include this study in the revision. We thank the reviewer for this very helpful suggestion.

References

  • Nemotron-Labs-Diffusion: Fu et al. Nemotron-Labs-Diffusion: A Tri-Mode Language Model Unifying Autoregressive, Diffusion, and Self-Speculation Decoding. arXiv:2607.05722, 2026.
  • SSD: Gao et al. Self Speculative Decoding for Diffusion Large Language Models. arXiv:2510.04147, 2025.
  • D2F: Wang et al. Diffusion LLMs Can Do Faster-Than-AR Inference via Discrete Diffusion Forcing. arXiv:2508.09192, 2025.
Reviewer_nh9sOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes S2D2, a training-free self-speculative decoding method for block-diffusion LMs. Standard block-diffusion decoding acts as the drafter, and the same model with block size 1 (i.e., autoregressive mode) acts as the verifier. The first contiguous masked span is verified in one extra forward pass. Drafted tokens are accepted with probability min(1, q/p), and the first rejected token is resampled from the residual. Routing policies decide when verification is worth the extra pass. Experiments cover SDAR-1.7B/4B/8B, Fast-dLLM v2 and LLaDA2.1-Mini on GSM8K, MBPP, HumanEval and IFEval. The baselines are static/dynamic confidence decoding and LLaDA2.1's token editing. The paper reports up to 4.13× speedup over AR on SDAR-1.7B (1.60× over dynamic decoding, +4.5 average points). On LLaDA2.1-Mini it reports 79.3 vs 78.7 average accuracy at 2.15× vs 1.75× over AR.

Critical strengths
  • Training-free plug-in for both position-aligned and right-shifted block-diffusion models. On every model some operating point beats the default decoder in speed at comparable or better accuracy.

  • Verification stabilizes regimes where confidence thresholding degrades. Large blocks are where block diffusion should pay off.

Critical weaknesses
  1. No comparison with existing speculative decoding for diffusion LMs; showing limited novelty. Main-text baselines are only the model's own decoders, but SSD and BlockSpec, discussed in Section 2, are not evaluated. How S2D2 compares with existing dLLM speculative decoders is unknown.
  2. The source of the gains is never decomposed. Five of the six SDAR operating points (Section 5.1) also enable AR caching, while the baselines use standard BD3. No run isolates it (BD3 + AR caching without verification, or S2D2 without it). For most SDAR points the gain cannot be distinguished from the mask change.
  3. Accuracy gains rest on selected operating points. The +4.5 points on SDAR-1.7B come from MBPP, where dynamic decoding collapses to 21.6 (AR: 48.4), and the other three tasks net -4.7 (Table 1). HumanEval drops 3.0-5.5 points vs dynamic in four of six SDAR configurations.
Questions for authors
  1. How does S2D2 compare with SSD and BlockSpec on the same model and hardware at matched latency?
  2. On SDAR, what are the results for BD3 with AR caching but no verification, and for S2D2 without AR caching?
  3. How many denoising steps does static decoding use in Table 1, and how is the AR baseline timed?
1 comment

We thank the reviewer for the detailed review. The two suggested ablations were very helpful, and we have run both.

W1/Q1: Comparison with SSD and BlockSpec at matched latency. Please see C1. BlockSpec has no public code.

  • B = 16. S2D2 is both faster and more accurate than SSD: GSM8K 86.6 at 4.51× vs. 79.2 at 3.33×; MBPP 62.6 at 3.19× vs. 49.0 at 2.87×.
  • B = 4. The two are at similar latency and accuracy.
  • Frontier. S2D2's accuracy–latency frontier therefore weakly dominates SSD's.
  • SPOT. S2D2 is also 8–12 points more accurate than SPOT at similar speed.

W2/Q2: Isolating AR caching from verification. We ran the full 2×2 ablation on SDAR-8B at both Table-1 operating points (200 GSM8K + 200 MBPP). Entries are accuracy (speedup over BD3):

BVariantGSM8KMBPP
16BD3 (dynamic)82.0 (1.00×)51.0 (1.00×)
16BD3 + AR caching, no verification79.0 (1.08×)56.5 (1.10×)
16S2D2 without AR caching71.5 (0.83×)45.0 (1.17×)
16S2D2 + AR caching (config-B)84.5 (0.96×)65.0 (1.15×)
4BD3 (dynamic)88.0 (1.00×)66.0 (1.00×)
4BD3 + AR caching, no verification89.5 (0.95×)65.5 (0.93×)
4S2D2 without AR caching86.5 (0.84×)60.0 (0.90×)
4S2D2 + AR caching (config-A)87.0 (0.80×)63.0 (0.82×)
  • At B = 16, the gain requires both components. Caching alone is within noise, and verification without caching hurts. Together they give +2.5 / +14.0 points.
  • Why caching helps. With AR caching, the verifier reads a causally written KV prefix that matches its block-size-1 conditionals. Accepted spans become longer (4.14 vs. 3.30 tokens).
  • At scale. The 3-seed, 500-example runs confirm the B = 16 result (C4).
  • At B = 4. The variants are within noise of each other, and S2D2 is somewhat slower. At small blocks diffusion decoding is already close to AR.

We will add this ablation to the paper.

W3: Gains rest on selected operating points. We agree the gains are not uniform across tasks, and we will say so explicitly.

  • MBPP. The collapse of dynamic decoding on MBPP (21.6 vs. 48.4 for AR on SDAR-1.7B) is exactly the failure mode S2D2 targets, and S2D2 recovers most of it.
  • GSM8K is mostly on par.
  • IFEval, never used for tuning, improves for all SDAR models.
  • HumanEval is weaker in several configurations, and we will acknowledge this.

Q3: Static decoding steps; AR timing.

  • Static decoding uses S = B = 4, i.e., one token per step. We will state this explicitly.
  • AR baseline. AR is the same model with block size 1, using the same implementation (B = 4, SB = 1 for Fast-dLLM v2, since B = SB = 1 is degenerate for its sub-block design).
  • Timing. All speedups, AR included, are end-to-end wall-clock times, measured identically at batch size 1.

References: see C1 (SSD, SPOT, BlockSpec).

Reviewer_M4XtOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper presents S2D2, a training-free decoding framework that adapts self-speculation to pretrained block-diffusion language models. The same model proposes tokens in parallel under diffusion decoding and verifies the first contiguous masked span using its autoregressive mode with a block size of 1. The main contributions are the verifier implementation for block-diffusion architectures and its integration with diffusion decoding. Results show improved accuracy–latency tradeoffs over confidence-threshold baselines in several settings.

Critical strengths

The method is easy to follow and requires no auxiliary models or additional training. It provides a unified inference-time correction mechanism for block-diffusion models with reliable block-size-1 autoregressive conditionals, enabling reuse across multiple existing model families.

Critical weaknesses
  1. Limited methodological novelty. ASSD [1] already combines same-model parallel drafting with autoregressive conditional verification, probability-ratio acceptance, and residual resampling for any-subset autoregressive models. SSD [2] also performs training-free self-speculation for diffusion language models, using batched hypothetical diffusion states for verification. S2D2's main additional contribution is adapting these ideas to block-diffusion architectures within a hybrid decoding framework.

  2. Heuristic routing selection. The paper provides no general, principled rule for choosing among routing policies or setting their hyperparameters. Acceptance proxies, cost coefficients, and routing thresholds are selected through empirical configuration sweeps, as described in the hyperparameter-selection appendix. This leaves limited guidance on selecting a suitable routing policy for a new model or deployment setting.

[1] Reviving Any-Subset Autoregressive Models with Principled Parallel Sampling and Speculative Decoding

[2] Self Speculative Decoding for Diffusion Large Language Models

Questions for authors

It would be helpful to clarify the benefits of more sophisticated routing policies over always-on verification or a simple minimum-span rule, and to provide practical guidance on choosing policies and their parameters.

3 comments

We thank the reviewer for the careful reading.

W1: Novelty relative to ASSD and SSD. Please see C2 and C1. Briefly:

  • ASSD provides a joint-distribution guarantee, but only for any-subset AR models.
  • SSD emulates the model's own slow stepwise diffusion decoder losslessly, using batched hypothetical states.
  • S2D2 instead uses the block-size-1 AR mode that block-diffusion models already have, to correct a fast decoder.

The difference matters in practice. At B = 16, SSD inherits the 7–16 point accuracy loss of its target, while S2D2 recovers AR-level accuracy at higher speed (C1). We believe showing that this works training-free, across position-aligned and right-shifted architectures and with wall-clock gains, is a useful contribution.

W2/Q: Heuristic routing; benefits over always-on or min-span; practical guidance. Please see C3:

  1. At B ≥ 16, S2D2 is insensitive to the routing choice; essentially every setting in Tables 13–14 beats dynamic decoding on both axes.
  2. Always-on is a strong default; most paper configurations verify at 88–100% of steps anyway.
  3. Selective policies mainly help when verification is relatively expensive, e.g. large sub-blocks on Fast-dLLM v2, where min-span 8 skips about 30% of verifications.

We will add this guideline to the paper.

References: see C1 (ASSD, SSD).

Thank you for your response. I decide to keep my score unchanged.

Thank you for reading our response. We will add the SSD and SPOT comparisons, the discussion of how S2D2 differs from ASSD and other dLLM speculative decoders, and the configuration-selection analysis to the revision. If any specific concern remains open, we would be happy to clarify it during the discussion period.

Reviewer_YiJFOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies fast decoding for block-diffusion language models in the few-step regime. The authors propose S2D2, a training-free self-speculative decoding framework that uses the same pretrained model as a diffusion drafter and a block-size-1 autoregressive verifier. S2D2 applies verifier-normalized speculative acceptance and residual resampling, together with lightweight routing policies that decide when verification is worthwhile. Experiments on five models from three block-diffusion families across four benchmarks show improved accuracy–speed tradeoffs over confidence-threshold decoding.

Critical strengths
  1. S2D2 is training-free and reuses the same model for drafting and verification, requiring neither an auxiliary model nor architectural changes.
  2. The paper provides useful routing ablations, throughput measurements, and failure-case analysis.
Critical weaknesses
  1. The novelty is relatively incremental given prior work such as ASSD, SSD, DiffuSpec, and BlockSpec, while direct empirical comparisons with these closely related methods are missing.
  2. Configurations are selected using GSM8K and MBPP samples and then evaluated on the same benchmark families; the separation of tuning and evaluation data is unclear, and most main results lack multi-seed uncertainty estimates.
Questions for authors

N/A

3 comments

We thank the reviewer for the positive assessment and helpful suggestions.

W1: Novelty relative to ASSD, SSD, DiffuSpec and BlockSpec; missing comparisons.

  • New comparisons. We now compare with SSD (our re-implementation) and SPOT (official code) (C1). ASSD needs a specific any-subset AR architecture, and DiffuSpec and BlockSpec have no public code.
  • Results. At large blocks S2D2 is 7–14 points more accurate at similar or better speed. At small blocks it roughly ties with SSD.
  • The settings differ (C2). These methods guarantee equivalence to AR or to a slower diffusion decoder, buying speed with extra parallel compute. S2D2 aims for speed without such compute, correcting toward AR quality.
  • Compatibility. The ideas combine well with S2D2. For example, our training-free port of D²SD further reduces NFE (response to Y4TA).

W2: Tuning/evaluation separation and multi-seed uncertainty.

  • Tuning. Decoding behavior differs across domains, so tuning on representative tasks is common practice. We searched on 100-example subsets of GSM8K (math) and MBPP (code), without per-task tuning (A.5).
  • Held-out tasks. HumanEval and IFEval were never used for tuning, and the gains transfer to them (e.g., IFEval +3 to +8 points; C3).
  • Multi-seed results. We now report 3-seed results (C4), in addition to the 5-seed LLaDA2.1 results (Table 4). Seed std is 1–1.5 points, well below the main gains.

References: see C1 (ASSD, SSD, SPOT, DiffuSpec, BlockSpec, D²SD).

Thank you for your response. I maintain my score.

Thank you for reading our response and for your continued support of the paper. We will incorporate the new comparisons (SSD, SPOT), the multi-seed results, and the clarification on configuration selection into the revision.

Reviewer_RK1aOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary
  • Setting: This paper concerns improving the inference efficiency of diffusion large language models (dLLMs) via speculative decoding, where the draft model is the dLLM itself with block-size BB and the target model is the dLLM itself with block-size 11.
  • Results: This paper shows that it is possible to be faster than pure diffusion decoding (S2D2 with "config-B") and it is also possible to achieve better text-generation quality than pure autoregressive decoding (S2D2 with "config-A").
Critical strengths
  • Interestingness: This paper discovers that, for existing open-source open-weight dLLMs, without any additional training or fine-tuning, the diffusion decoding can draft for the autoregressive decoding, the latter can refine the former, and such speculative decoding may achieve a new Pareto frontier of speed and quality. This is a very interesting discovery.
Critical weaknesses
  • Limited significance: dLLMs are usually not designed for autoregressive decoding. Although initialized with autoregressive weights, the dLLMs in this paper, namely SDAR, Fast-dLLM v2, and LLaDA2.1, are not guaranteed to preserve autoregressive decoding ability that is reliable, or at least as reliable as their diffusion decoding ability, after diffusion training. As this paper indicates, some dLLMs perform poorly in autoregressive decoding mode, such as Fast-dLLM v2 with block-size 11 and subblock-size 11. The speculative decoding with a possibly weak target model is questionable.
Questions for authors

None.

3 comments

We thank the reviewer for finding the discovery "very interesting".

W1: The AR mode of a dLLM may be a weak verifier. We agree that the AR mode is not guaranteed to be reliable. This is exactly why S2D2 does not decode globally in AR mode: the AR mode serves only as a local verifier of the first masked span, and a routing policy decides when to invoke it.

In practice, the AR mode of current block-diffusion models is a strong verifier:

  • AR is the most accurate decoder for most models in the paper (average accuracy):
    • SDAR-1.7B: 56.4 vs. 48.4 for dynamic decoding
    • SDAR-4B: 69.5 vs. 67.5
    • Fast-dLLM v2: 59.4 vs. 53.3, with B = 4, SB = 1 as its AR setting
  • The verifier is consistent with the drafts. Per-token acceptance rates are 0.80–0.96, and each verification accepts 3–11 draft tokens on average (yKG4 Q2).

We also report the boundary case explicitly. D2F (Table 8) is trained with diffusion forcing, so its AR mode is weak (20% vs. 50% on GSM8K). For such models, reject-only verification is safer.

Finally, concurrent works pursue the same pairing of diffusion drafting with AR verification:

  • UNO keeps the AR LLM frozen and trains a diffusion LoRA drafter, so quality is preserved.
  • Nemotron-Labs-Diffusion trains a single model with AR, diffusion and self-speculation modes.

This suggests the direction is promising, and AR modes will likely become more reliable as models are trained for it.

References

  • D2F: Wang et al. Diffusion LLMs Can Do Faster-Than-AR Inference via Discrete Diffusion Forcing. arXiv:2508.09192, 2025.
  • UNO: Sahoo et al. Unlocking Lossless Speedups in LLMs via Discrete Diffusion. arXiv:2609.04010, 2026.
  • Nemotron-Labs-Diffusion: Fu et al. Nemotron-Labs-Diffusion: A Tri-Mode Language Model Unifying Autoregressive, Diffusion, and Self-Speculation Decoding. arXiv:2607.05722, 2026.

Thank you for your detailed reply. I acknowledge the provided empirical evidence that several existing dLLMs have reliable autoregressive decoding ability. However, I do not think this fully addresses my concern about the general applicability of the proposed method. Therefore, I decide to keep my original rating.

Moreover, my understanding is that the routing policy mainly improves speculative-decoding efficiency instead of verifier reliability. Besides, verifier--drafter consistency does not necessarily imply verifier reliability. Neither of these points directly addresses my concern.

Thank you for the follow-up. We agree with both points and clarify how S2D2 handles them.

Consistency is not reliability. We agree: acceptance rates show that verification is usable, not that the verifier is correct. Reliability must be judged by task accuracy. For the models we evaluate, the AR mode is more accurate than the diffusion setting S2D2 builds on, and S2D2 moves accuracy toward it (average accuracy, Tables 1–3; Diffusion = the matching baseline: dynamic B = 4 for SDAR, SB = 4 for Fast-dLLM v2, quality thresholds for LLaDA2.1):

ModelAR modeDiffusionS2D2 (config-A)
SDAR-1.7B56.448.454.4
SDAR-4B69.567.568.1
SDAR-8B71.770.572.6
Fast-dLLM v259.453.354.2
LLaDA2.1-Mini (quality)78.373.777.4

Routing also controls reliance on the verifier. Routing decides at each step whether the AR mode is consulted, so it bounds how much an unreliable verifier can affect the output. If verification is disabled, S2D2 reduces exactly to the base diffusion decoder. D2F, whose AR mode is much weaker than its diffusion mode, illustrates this (Table 8, GSM8K):

D2F (LLaDA-8B)Accuracy
AR mode (B = 1)20.0
Diffusion (B = 32)50.0
S2D2, residual resampling33.3
S2D2, reject-only (re-mask rejected positions)46.7
S2D2, verification routed off50.0

Even with a weak verifier, S2D2 stays far above the AR mode, and the routing and correction rule let it fall back to diffusion quality. Whether to enable verification can be decided on the small development set already used for configuration selection (A.5). The applicability condition is therefore explicit and cheap to check: a usable block-size-1 mode. This holds for all three families we evaluate, and it is becoming a design goal of recent models (e.g., Nemotron-Labs-Diffusion, UNO). We will state this condition and the role of routing explicitly in the revision.

References

  • Nemotron-Labs-Diffusion: Fu et al. Nemotron-Labs-Diffusion: A Tri-Mode Language Model Unifying Autoregressive, Diffusion, and Self-Speculation Decoding. arXiv:2607.05722, 2026.
  • UNO: Sahoo et al. Unlocking Lossless Speedups in LLMs via Discrete Diffusion. arXiv:2609.04010, 2026.
Reviewer_Y4TAOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

S2D2 is a training-free self-speculative decoding method for block-diffusion language models. It uses the same model in block-diffusion mode for proposals and in a local autoregressive mode for verification, with routing policies to control when verification is applied. The evaluation considers accuracy and speed across several model families and tasks.

Critical strengths

The training-free use of the same model for both drafting and local verification is simple and practical. The work makes a useful incremental contribution toward improving the speed-quality trade-off of block-diffusion language models, and evaluates multiple model families.

Critical weaknesses

The improvement is modest or mixed in some settings, and the results depend on the model and selected operating point. The method is an incremental combination of existing ideas, and does not claim to preserve the global autoregressive output distribution. More complete comparisons at similar accuracy would make the practical speed-quality trade-off clearer. These limitations temper the contribution but do not outweigh the usefulness of the approach.

Questions for authors

Could the authors clarify the speed-quality trade-off at matched accuracy across their main models and tasks, including sensitivity to the routing policy?

Future-work suggestion, not a condition for acceptance: could confidence-guided multi-branch drafting or prefix-conditioned redrafting, as explored in D²SD (https://arxiv.org/abs/2606.04446), improve the draft stage here? D²SD targets an autoregressive model using a separately trained drafter, whereas S2D2 uses a diffusion model as both drafter and local verifier, so a combination would require adaptation and may give up the training-free property.

1 comment

We thank the reviewer for the positive assessment, and especially for the D²SD suggestion, which we have now explored.

W1: Modest or mixed improvements; dependence on the operating point. The size of the gain depends on the operating point, because diffusion decoding itself trades quality for speed (C3). Where the gains are substantial:

  • SDAR. Up to 4.1× average speedup over AR, and about 1.6× over tuned dynamic decoding with +4.5 average accuracy.
  • SDAR-8B config-B. Faster and more accurate than dynamic decoding.
  • Large blocks. +8.5 / +13.3 points at the same speed (C4).
  • LLaDA2.1-Mini. 3.95× faster than the static baseline at comparable accuracy (conservative setting).
  • Against recent dLLM speculative decoders. 7–14 points more accurate than SSD and SPOT (C1).

Q1: Trade-off at matched accuracy and sensitivity to routing. Tables 12–17 and Fig. 3 give this view for SDAR-8B:

  • At matched accuracy (~75.5 avg.): dynamic decoding needs B = 4 (2.5×), while S2D2 reaches the same accuracy at 3.3–4.1× with B = 16/32. That is 1.3–1.6× faster.
  • Sensitivity: at B ≥ 16, essentially every routing setting beats dynamic decoding on both axes, so S2D2 is insensitive to the exact threshold within a broad range (C3).

Q2: D²SD-style multi-branch drafting. We implemented a training-free version of D²SD's idea inside S2D2:

  • Using D²SD's posterior over the first rejection position, we open side branches (next-best draft token) at the top-K positions.
  • All branches are verified in S2D2's single verification forward, using block-diagonal masks.
  • No redraft is needed, because a block-diffusion draft at position i+1 conditions on MASK at position i, not on the token there.
  • A SpecInfer-style multi-candidate rejection rule keeps S2D2's per-position acceptance property.

Results for K = 4 vs. K = 0 (3 seeds):

Model / configGSM8K acc / NFE / timeMBPP acc / NFE / time
SDAR-8B config-B86.0 → 85.4 / −7.8% / −4.8%61.1 → 60.3 / −6.0% / −2.1%
SDAR-8B config-A85.1 → 84.8 / −5.5% / −4.7%62.1 → 62.9 / −4.5% / −3.3%
Fast-dLLM v2 config-B84.0 → 84.6 / −5.8% / −2.4%46.2 → 46.1 / −6.0% / −2.4%
Fast-dLLM v2 config-C84.1 → 84.1 / −5.0% / −0.6%47.6 → 46.2 / −3.4% / +1.2%
  • NFE drops by 3–8% everywhere, with accuracy unchanged within seed variance.
  • Placing branches by D²SD's posterior beats uniform placement: the branch lands on the actual rejection position 37% vs. 18% of the time.
  • Wall-clock gains are modest (up to ~5%). S2D2 already corrects one token at the first rejection, so a branch only adds its (short) tail.
  • Larger gains would likely need redrafted tails or D²SD's trained drafter.

We will add this extension to the appendix, and thank the reviewer for the suggestion.

References

  • D²SD: D²SD: Dual-Diffuse Speculative Decoding for Large Language Models. arXiv:2606.04446, 2026.
  • SpecInfer: Miao et al. SpecInfer: Accelerating Large Language Model Serving with Tree-based Speculative Inference and Verification. ASPLOS 2024.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Common Responses

We thank all reviewers for their constructive feedback, and we are glad S2D2 was found simple and practical (Y4TA), a very interesting discovery (RK1a), training-free with no auxiliary model (YiJF, M4Xt) and useful across model families (nh9s, yKG4).

Setup of new experiments. Unless stated otherwise, new experiments use:

  • a fixed set of 500 GSM8K test examples and the full MBPP test set;
  • the paper's decoding settings, batch size 1, on one H200;
  • wall-clock speedups, measured with the same evaluation code for all methods.

Absolute GSM8K numbers are slightly lower than in Table 1 for every method, AR included, because the example subset differs.

C1. Comparison with speculative decoders for diffusion LMs (nh9s, YiJF, M4Xt)

We compare with SSD and SPOT on SDAR-8B, using the same hardware and evaluation code for all methods.

  • SSD. No code is public, so we re-implemented it following its Algorithm 1 (N = 5). We verified that it matches its lossless target up to fp16 ties.
  • SPOT. We used the official code and the authors' SDAR-8B drafter.
  • Not compared. BlockSpec and DiffuSpec have no public code. ASSD requires an any-subset AR (XLNet-style) architecture and cannot be applied to block-diffusion models.

Entries are accuracy (wall-clock speedup over AR).

MethodBGSM8K (T=0)MBPP
AR185.6 (1.00×)65.2 (1.00×)
Stepwise diffusion (SSD's lossless target)487.8 (1.65×)63.8 (1.58×)
SSD487.6 (2.16×)63.8 (2.05×)
S2D2 config-A486.2 (2.30×)63.8 (2.12×)
Stepwise diffusion (SSD's lossless target)1679.0 (1.76×)49.4 (1.62×)
SSD1679.2 (3.33×)49.0 (2.87×)
S2D2 config-B1686.6 (4.51×)62.6 (3.19×)
Method (B = 16)GSM8K (T=1, 3 seeds)MBPP
Static diffusion (SPOT's lossless target)78.5 ± 0.6 (1.73×)49.4 (1.62×)
Dynamic diffusion77.5 ± 1.3 (3.47×)47.8 (2.76×)
SPOT (trained drafter)78.2 ± 1.1 (3.36×)49.6 (3.59×)
S2D2 config-B (training-free)86.0 ± 1.4 (3.51×)61.1 ± 1.3 (3.28×)

Takeaways:

  • SSD and SPOT are lossless with respect to stepwise diffusion decoding, not AR decoding. At B = 16 that target loses 7–16 points relative to AR.
  • S2D2 corrects toward AR quality. At B = 16, where most of the speedup is, S2D2 is 7–14 points more accurate than SSD and SPOT, at similar or better speed. SPOT is about 10% faster on MBPP, but needs a trained drafter for each model and block size.
  • At B = 4, S2D2 and SSD are roughly tied.

The methods target different goals (C2), so this comparison is not strictly like-for-like. They are also complementary: we ported D²SD's multi-branch drafting into S2D2's single verification pass, and it reduces NFE by a further 3–8% (response to Y4TA).

References

  • SSD: Gao et al. Self Speculative Decoding for Diffusion Large Language Models. arXiv:2510.04147, 2025.
  • SPOT: Shlomi et al. On the SPOT: Enhancing Speculative Decoding for Diffusion Language Models via Commit-Order Prediction. arXiv:2610.22543, 2026.
  • ASSD: Guo & Ermon. Reviving Any-Subset Autoregressive Models with Principled Parallel Sampling and Speculative Decoding. arXiv:2504.20456, 2025.
  • BlockSpec: Pan et al. BlockSpec: Blockwise Speculative Decoding for Diffusion LLMs.
  • DiffuSpec: Li et al. DiffuSpec: Unlocking Diffusion Language Models for Speculative Decoding. arXiv:2510.02358, 2025.
  • D²SD: D²SD: Dual-Diffuse Speculative Decoding for Large Language Models. arXiv:2606.04446, 2026.

C2. Positioning and novelty (YiJF, M4Xt, Y4TA)

We appreciate that novelty is partly subjective. We would like to clarify a few points that are easy to misread.

S2D2 does not aim to be global speculative decoding, as stated in the abstract and Sec. 4.4. The setting differs from prior work:

  • ASSD provably samples from an any-subset AR model, which requires a specific architecture.
  • SSD, SPOT, DiffuSpec and BlockSpec reproduce a slower multi-step diffusion decoder exactly, trading extra parallel compute for fewer sequential steps.
  • S2D2 reproduces neither.

The question S2D2 asks. A block-diffusion model has blockwise-AR structure, and at block size 1 it is an AR model, so the same weights provide a verifier for free. Can this make decoding faster without hurting quality? This is not obvious, because every verification costs an extra forward pass.

Two views of S2D2:

  • A verifier-adaptive threshold. Dynamic decoding accepts draft tokens above a fixed confidence threshold. S2D2 replaces it with min(1, q/p), so the model's own AR mode decides how much of the parallel draft to trust.
  • A dynamic block size. When the AR mode agrees with the draft, S2D2 commits a long span at once (3–11 tokens per verification on average; yKG4 Q2). When it disagrees early, decoding falls back to near-AR steps. This is why S2D2 can use large blocks, where diffusion decoding alone degrades.

What the experiments show. Across three families and five models, this improves the accuracy–speed trade-off, often making decoding both faster and more accurate than tuned dynamic decoding.

  • An empirical study. The EDLM connection (Sec. 4.4) offers a way to understand the method, not a guarantee. The hybrid diffusion/AR trajectory is hard to analyze exactly.
  • Future work. Learning a cheap routing head is a natural next step.

Since S2D2 never claims a global guarantee and targets a different goal (speed at AR-level quality), we respectfully suggest that the absence of such a guarantee should not count against it.

References

  • EDLM: Xu et al. Energy-Based Diffusion Language Models for Text Generation. arXiv:2410.21357, 2024.
  • See C1 for ASSD, SSD, SPOT, DiffuSpec and BlockSpec.

C3. Configuration selection and routing (Y4TA, M4Xt, nh9s, yKG4)

Two configurations per model. Diffusion decoding has an inherent quality–speed trade-off, so we report a quality-oriented config-A and a speed-oriented config-B, in the same way the baselines are reported at several block sizes. Sometimes one configuration wins on both axes: on SDAR-8B, config-B is faster than dynamic decoding (3.43× vs. 2.58×) and more accurate (71.3 vs. 70.5).

Selection (Appendix A.5).

  • We ran a grid search on 100 GSM8K + 100 MBPP examples (representative math and code tasks), on the largest model of each family, with no per-task tuning.
  • HumanEval and IFEval were never used for selection, and the gains carry over to them. IFEval vs. dynamic decoding: +3.2 / +7.8 / +6.9 on SDAR-1.7B/4B/8B.

Robustness (Tables 12–14, SDAR-8B). At B ≥ 16, essentially every routing setting is both faster and more accurate than dynamic decoding:

  • B = 16: minimum-span routing gives 72.6–75.7 at 3.3–3.6×, vs. 67.5 at 3.1× for dynamic decoding.
  • B = 32: 67.5–76.5 at 3.3–4.1×, vs. 63.3 at 3.0×.
  • At B = 4, S2D2 trades some speed for accuracy. This is expected, since by construction it lies between diffusion and AR decoding.

Always-on vs. selective routing. Most paper configurations verify at 88–100% of steps (yKG4 Q2). Selective routing matters mainly when verification is relatively expensive, e.g. Fast-dLLM v2 at SB = 32 (66–71%). We will add this guideline to the paper:

  • start with always-on verification, plus AR-like caching for position-aligned models;
  • switch to min-span or score-threshold routing when verification overhead dominates.

C4. Multi-seed results (YiJF, nh9s)

Mean ± std over 3 seeds:

ModelMethodGSM8KMBPP
SDAR-8BAR85.9 ± 0.765.2
SDAR-8BDynamic diffusion (B = 16)77.5 ± 1.347.8
SDAR-8BS2D2 config-A (B = 4)85.1 ± 0.862.1 ± 1.7
SDAR-8BS2D2 config-B (B = 16)86.0 ± 1.461.1 ± 1.3
Fast-dLLM v2S2D2 config-B (SB = 16)84.0 ± 1.546.2 ± 0.4
Fast-dLLM v2S2D2 config-C (SB = 32)84.1 ± 1.547.6 ± 0.8
  • Seed std is 1–1.5 points. The Fast-dLLM v2 numbers agree with Table 2 within this variance.
  • The key gains are far above noise. For example, S2D2 vs. dynamic decoding at B = 16 is +8.5 on GSM8K and +13.3 on MBPP.
  • LLaDA2.1 multi-seed results (5 seeds) are in Table 4.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.