acceptodds
Under review as a conference paper at ICLR 2027

On the SPOT: Enhancing Speculative Decoding for Diffusion Language Models via Commit-Order Prediction

25 views

Abstract

Diffusion language models (DLMs) promise parallel text generation, but in prac- tice face an unresolved speed–quality trade-off. Exact decoding fills in only one masked position per model pass, while faster multi-token decoding schemes are lossy. Speculative decoding is the standard lossless acceleration for autoregres- sive models. However, it does not transfer directly to DLMs, which must choose both what to generate and where. We show that this order is the missing piece: the model’s own predictions from the previous pass already give the right token at 92–100% of positions, but not which position comes next. Based on this ob- servation, we introduce Speculating Positions Order Trajectory (SPOT), which learns to predict this order. A 0.62M-parameter model reads only the target’s out- put distributions and proposes a tree of candidate continuations, and the target verifies all of them in a single pass by checking whether each equals its own next state. On the verification side, our analysis shows that prior block-diffusion verifi- cation approaches either assume T =0 or a sequential proposal interface. To sup- port tree-based speculation at any temperature, we propose using coupled-noise match verification for diffusion blocks, following shared-Gumbel-noise coupling approaches. Our comprehensive experiments show that SPOT cuts target passes by up to 6.8× and reaches up to 3.2× the throughput of standard decoding, beating the calibrated and training-free drafting baselines in every task and budget at T =0 and T =1. In addition, we show that the learned drafter transfers across models, demonstrating successful transfer across six different DLMs. Code is available at: https://anonymous.4open.science/r/spot-speculative-decoding-E354

Primary area:
foundation or frontier models, including LLMs
Keywords:
diffusion language models, speculative decoding, masked diffusion, lossless acceleration, drafter, tree verification, Gumbel noise coupling

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_6Tv8OpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes SPOT, a speculative decoding method for diffusion language models. Its central observation is that stale target distributions often predict future committed tokens accurately, while predicting their commit order remains difficult. SPOT uses a 0.62M-parameter model to predict positions conditioned on hypothetical commitments and constructs a tree of candidate states. A shared-Gumbel verifier preserves the target sampling distribution under the stated decoding assumptions. Experiments report up to 6.8× fewer target evaluations and approximately 3.2× higher throughput, together with cross-model drafter transfer.

Critical strengths
  1. A useful diagnosis of the drafting bottleneck. The position/token oracle experiments provide convincing evidence that commit-order prediction is particularly valuable for the studied decoding regime. The lightweight predictor directly addresses this bottleneck and improves over stale-confidence ordering. Using only probability features also makes the cross-model transfer results interesting.

  2. A principled verifier with meaningful measured acceleration. The shared-Gumbel construction cleanly separates verification correctness from draft quality and supports tree-based speculation under sampling. The proof explicitly addresses adaptive position selection. Reporting both target evaluations and wall-clock throughput, including their different responses to draft budget, strengthens the practical evaluation.

Critical weaknesses
  1. The comparisons do not fully establish the advantage over well-tuned competing methods. Appendix D.4 fixes the SSD-style chain depth at four, while SPOT can allocate substantially deeper paths. The baselines are adaptations within the authors’ decoder rather than complete original implementations. This is useful for isolating drafting, but limits broader superiority claims. Moreover, MRP is compared using published speedups measured with different hardware, block widths, implementations, and generation lengths. The assertion that these differences cannot explain the margin is insufficiently supported.

  2. Inconsistent descriptions of the evaluation protocol weaken reproducibility. Section 5.1 states 350 samples, with all 164 HumanEval problems, whereas Appendix E describes first-100 evaluation subsets and explicitly associates Table 2 with those subsets. Appendix D.5 also justifies calibration-set separation by stating that every evaluated run uses samples 0–99. If the larger evaluation sets were used, that separation requires clarification. These inconsistencies affect the interpretation of sample coverage and hyperparameter selection.

Questions for authors
  1. Protocol clarification: Which sample counts and exact splits produced Table 2? Were the MBPP samples used for mass-floor calibration excluded from every reported evaluation, including any 350-sample runs?

  2. Comparison robustness: How do the conclusions change when baseline chain depth, graph structure, and verification budget are tuned for throughput using a comparable validation allowance? Can the MRP comparison be reproduced under matched conditions, or its superiority claim qualified?

  3. Implementation exactness: Appendix A.1 assumes batch-invariant target evaluation and refers to Appendix G for finite-precision effects, but that appendix primarily discusses timing variability. What are the greedy trajectory mismatch rates between standalone and batched verification, especially for unpadded SPOT-γ? Similar downstream accuracy alone does not establish trajectory equality.

2 comments
AuthorsOpenReview ↗

Response to Reviewer 6Tv8

Dear Reviewer 6Tv8,

Thank you for the positive evaluation and for recommending acceptance. We especially appreciate your careful reading of the evaluation protocol and your questions on tuned baselines and a matched MRP comparison, which led to new experiments and clarifications that will be included in the revised paper and, we believe, improve it.

W1 / Q2. Tuned baselines and a matched MRP comparison.

Following the reviewer's suggestion, we strengthened the results with tuned baselines and added a matched MRP comparison, and tuning the baselines does not change the conclusions. Running the SSD-style chain as deep as the budget (8 and 20, first 100 problems of every task) changes its target passes by at most 4% and does not improve its speed, while SPOT still needs 16–47% fewer passes and is faster at every task and budget (table below). Spiffy uses its own offline-calibrated graph built with its published recipe, SPOT's own settings (the 0.6 runner-up threshold and SPOT-γ's mass floor) are fixed across all results, and with each method at its own best budget (8, 12 or 20), SPOT is 1.12–1.40× faster than the best baseline on every task at T=0 and T=1.

SSD-style chain at its main-table depth (4) and at depth equal to the budget, against SPOT. SDAR-8B, T=0, first 100 problems, NFE at budget 8 / 20.

NFE (D8 / D20)GSM8KMBPPHumanEvalMATH500
SSD-style, depth 492 / 8584 / 7587 / 80100 / 93
SSD-style, depth = budget95 / 8285 / 7489 / 78104 / 90
SPOT70 / 5356 / 3962 / 4484 / 65

For MRP we matched its own setting: block 16, where its released checkpoint exists, with the same prompts and the same verifier (SDAR-8B, T=0, SPOT at budget 8). SPOT still wins on every task, with 1.5–1.8× fewer target passes and 1.5–1.8× higher throughput, and running MRP's own released code with the same checkpoint confirms that our harness does not handicap it.

NFE · tok/s (speed-up)GSM8KMBPPHumanEvalMATH500
Draft-free263 · 18.0263 · 17.5263 · 17.0263 · 18.6
MRP118 · 31.9 (1.8×)113 · 34.1 (1.9×)105 · 38.3 (2.3×)132 · 30.1 (1.6×)
SPOT76 · 54.2 (3.0×)62 · 61.0 (3.5×)66 · 58.3 (3.4×)89 · 45.6 (2.5×)
SPOT-γ81 · 53.0 (2.9×)65 · 58.2 (3.3×)68 · 63.1 (3.7×)96 · 43.9 (2.4×)

We thank the reviewer for these suggestions, which strengthen the comparison, and will add the tuned-baseline and MRP results, with both tables, to the revision.

W2 / Q1. Sample counts, splits, and the calibration samples.

We agree with the reviewer that the calibration point requires clarification and verification. The main table uses the sample counts and splits stated in Section 5.1. We checked γ under two settings: one in which the calibration problems were part of the evaluation set, and one in which they were excluded, using unused MBPP train problems. Both settings select the same value, γ=0.1, confirming that this does not affect the reported results. Nevertheless, we fully acknowledge the reviewer's important point and will revise the relevant sections accordingly to ensure complete consistency, clarity and reproducibility of the experimental protocol. We thank the reviewer for raising it and for helping us improve the paper's reproducibility.

Q3. Trajectory mismatch rates, especially for unpadded SPOT-γ.

Unpadded SPOT-γ matches a standalone run as often as padded SPOT does, and as often as draft-free decoding run at a different batch shape, so the differences come from the batch shape and not from drafting. Draft-free decoding at SPOT's own batch shape reproduces SPOT's output on 96% of problems. 94–99% of first differences are two nearly tied positions committed in swapped order. Correctness flips are balanced, so accuracy does not change. We thank the reviewer for this question and will clarify this in the revision where needed.

Thank you again. We hope the corrected protocol and the new checks resolve your concerns about soundness and comparisons. If they do, we would be grateful if you would take them into account in your final rating. We would be happy to answer any follow-up questions or clarify any remaining concerns.

AuthorsOpenReview ↗

Follow-up on our response

Dear Reviewer 6Tv8,

Thank you again for your time and effort.

We hope that our rebuttal has addressed your concerns. With the discussion period nearing its end, we would appreciate hearing your thoughts.

We would be happy to address any remaining concerns or provide further clarification. If our responses have addressed your remaining concerns, we would greatly appreciate it if you could consider strengthening your support for the paper.

Reviewer_1ZDXOpenReview ↗
Rating
4
Confidence
2
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes a way to speed up speculative decoding for dLLMs by training a drafter to predict the next decoding positions, then batch verify a tree of drafts. This method achieves up to 3.2 times of throughput improvement.

Critical strengths
  1. Many existing methods for accelerating diffusion-based LLMs (dLLMs) are lossy. Speculative decoding provides lossless acceleration for auto-regressive LLMs, but it is not directly transferable to dLLMs. Unlike other speculative decoding methods for dLLMs, this paper proposes a way to do it by predicting the next decoding positions based on the current block's information, which improves verifiers acceptance rate.
  2. This paper proposes "coupled-noise match", a way to improve verifiers acceptance rate by making the drafter's and verifier's Gumbel noise the same.
Critical weaknesses

None

Questions for authors

This paper talks about speculative decoding, which is supposed to be lossless. However, you mentioned "greedy match" in section 3, which you described as a non-exact verification method. I do not understand why mention it in this paper, how does this point support your claim?

1 comment
AuthorsOpenReview ↗

Response to Reviewer 1ZDX

Dear Reviewer 1ZDX,

We warmly thank you for the positive evaluation and the clear support for acceptance. We also thank you for the question about greedy match, and the clarification below will be included in the revised paper.

Q. Why greedy match, a non-exact verifier, appears in the paper.

We agree that this clarification is important and that the paper should have explained it more clearly. Greedy match is a negative control. It is what one gets by running the T=0 verifier unchanged at a higher temperature: a draft is accepted when it equals the target's most likely token. Every accepted token is then the most likely token rather than a sample from the target, so the output is biased, and the better the drafter, the more tokens it accepts and the larger the bias (table below, from Section 3). Our verifiers instead accept a draft only when it equals the target's own sample, which keeps the output exact. No result we claim uses greedy match, and we will make this explicit in Section 3.

SDAR-8B on MBPP, SPOT drafter at budget 8. How often the emitted token is the model's most likely one, and accuracy, for the model's own sampling and for greedy match:

TMost likely token: modelMost likely token: greedy matchAccuracy: modelAccuracy: greedy match
0.598.6%97.8%0.350.36
1.097.6%97.3%0.340.38
1.586.6%95.8%0.080.28
2.05.7%75.0%0.000.01

Our exact verifiers stay within 1.6 points of the model's own rate and within 0.04 of its accuracy at every T up to 1.5.

Thank you again for your support. We would be happy to answer any follow-up questions.

Reviewer_CU3sOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces SPOT, a speculative-decoding method for diffusion language models. Its central observation is that stale target distributions usually predict future token identity accurately, while predicting the order in which positions are committed remains difficult. SPOT therefore uses a lightweight 0.62M-parameter model to predict commit order, recursively constructs a tree of hypothetical future states, and verifies all drafted states in one batched target-model forward per speculative round.

For nonzero sampling temperatures, the paper introduces coupled-noise verification: the drafter and target share position-indexed Gumbel noise, making the target transition deterministic conditional on that noise while preserving the target's marginal sampling distribution. Experiments span four tasks, multiple temperatures and block widths, and six DLM targets. However, the LLaDA-8B and Dream-7B transfer experiments use an approximate prefix cache and are not part of the lossless claims. On SDAR-8B, SPOT reduces target-model evaluations and improves throughput over the evaluated exact speculative-decoding baselines.

Critical strengths
  1. The paper identifies a consequential bottleneck and converts it into an effective lightweight method. The analysis shows that stale token predictions remain highly accurate while stale commit-order information degrades sharply. SPOT directly targets this missing information and substantially improves both NFE and throughput. For example, at T = 1 and D = 20, it uses 42.6 NFE on MBPP versus 64.8 for Spiffy, while SPOT-gamma reaches 2.86 times the draft-free throughput.

  2. The coupled-noise verifier provides a clean exactness mechanism under sampling. Conditioning on shared position-indexed Gumbel noise reduces verification to deterministic state equality, while marginalization recovers the target distribution. The temperature sweep demonstrates practical value: at T = 1.5 and D = 20, coupled verification uses 54.1 NFE for SPOT versus 62.7 with independent sampling.

Critical weaknesses
  1. The comparison does not fully establish superiority over all recent applicable methods under a common implementation. The main tables use SSD-style and BlockSpec-style implementations inside the paper's shared harness. MRP is compared only through published speedup numbers measured under different hardware and settings. Some omitted methods are not directly comparable because they use approximate verification, but the fidelity of the shared-harness reimplementations to the original methods should be validated. The broad claim of beating prior methods is therefore stronger than the available direct evidence.

  2. The wall-clock and uncertainty methodology needs stronger validation. The paper reports standard deviations or spread statistics for several settings, but it does not provide confidence intervals or paired significance tests, and some appendix results use only one seed. Several tables use the first 100 examples, while other experiments use larger subsets; the relationship between the sample counts stated in Section 5.1 and Appendix E should be clarified. Appendix G also applies a disclosed rerun rule to timing outliers, affecting 26 of approximately 1,500 cells. Although the major gains are likely robust, reporting results both with and without these reruns would make small throughput margins and "every setting" claims easier to assess.

Questions for authors
  1. Can the paper provide direct comparisons against the strongest applicable recent methods using their released implementations, or show that the SSD-style and BlockSpec-style implementations reproduce their published acceptance and throughput behavior?

  2. Can the authors report paired confidence intervals or significance tests for throughput and NFE? How do the conclusions change when all original timing repetitions, including the archived slow runs, are retained?

  3. Can the authors clarify whether inference uses one runner-up or up to four token candidates per position and provide the exact configurations used for every headline result?

  4. How often do finite-precision differences between standalone and (1 + D)L-batched target evaluation change the selected position or token? This would clarify how closely the practical implementation matches the idealized trajectory-equality proof.

2 comments
AuthorsOpenReview ↗

Response to Reviewer CU3s

Dear Reviewer CU3s,

Thank you for the positive evaluation and for supporting the acceptance of our paper. We also thank you for the thorough and insightful review. Your detailed questions on baseline fidelity and statistical rigor led to new experiments and clarifications that will be included in the revised paper. In short: our SSD-style and BlockSpec-style versions reduce NFE at least as much as their papers report, MRP now runs inside our harness, where SPOT needs 1.5–1.8× fewer target passes and is 1.5–1.8× faster, and SPOT is significantly faster than every baseline in all 48 comparisons of the main table.

W1 / Q1. Fidelity of the SSD- and BlockSpec-style baselines, and a direct MRP comparison.

Neither SSD nor BlockSpec released code, so we implemented both from their papers, with details in the appendix, and both implementations will be part of our code release. Run in each paper's own setting (LLaDA-8B-Instruct), our implementations reduce NFE at least as much as the papers report (SSD: 62–68% vs 58–63%, BlockSpec: 34–52% vs 25–33%). For BlockSpec this is its threshold variant, since the paper gives no step counts for the exact tree we use. We compare NFE because, unlike throughput, it does not depend on the GPU or cache implementation. MRP now runs inside our harness with its released checkpoint, the same prompts and the same verifier (SDAR-8B, block 16, T=0, 350 problems × 3 seeds, SPOT at budget 8):

NFE · tok/s (speed-up)GSM8KMBPPHumanEvalMATH500
Draft-free263 · 18.0263 · 17.5263 · 17.0263 · 18.6
MRP118 · 31.9 (1.8×)113 · 34.1 (1.9×)105 · 38.3 (2.3×)132 · 30.1 (1.6×)
SPOT76 · 54.2 (3.0×)62 · 61.0 (3.5×)66 · 58.3 (3.4×)89 · 45.6 (2.5×)
SPOT-γ81 · 53.0 (2.9×)65 · 58.2 (3.3×)68 · 63.1 (3.7×)96 · 43.9 (2.4×)

MRP's own released code with the same checkpoint gives a 1.52× speed-up on GSM8K (1.40× in its paper), against 1.8× in our harness, so our harness does not handicap MRP. We thank the reviewer for this suggestion and will add the MRP comparison and these additional experiments to the revision.

W2 / Q2. Significance of the gains, and the effect of the timing reruns.

Paired 95% bootstrap intervals over all 48 SPOT-vs-baseline comparisons of the main table show that SPOT is faster and needs fewer passes in every one. SPOT-γ is faster in every one and uses slightly more passes in a few cells, because it verifies fewer drafts per pass.

Pooled over 16 comparisonsSpeed ratio: range (lowest 95% bound)NFE ratio: range (lowest 95% bound)
SPOT vs Spiffy1.12–1.48× (1.12)1.18–1.55× (1.17)
SPOT vs SSD-style1.13–1.97× (1.12)1.07–1.92× (1.06)
SPOT vs BlockSpec-style1.54–2.96× (1.53)1.61–2.81× (1.60)
SPOT-γ vs Spiffy1.26–2.66× (1.25)0.92–1.37× (0.92)
SPOT-γ vs SSD-style1.26–3.61× (1.25)0.99–1.59× (0.98)
SPOT-γ vs BlockSpec-style1.68–5.13× (1.67)1.30–2.33× (1.28)

Keeping every archived timing run changes one row (SPOT-γ on GSM8K), which stays at or above 1.33× over every baseline, and pass counts are unaffected. The main table uses 350 problems per task (all 164 for HumanEval) and 3 seeds, and we will update Appendix E to match, state n and seeds in every table, and add these analyses to the revised version.

Q3. One runner-up or four token candidates, and the exact configurations.

One runner-up at most: a position gets a second token only when its top-1 probability is below 0.6. The "4 token candidates" mentioned in the paper refer to a training setting of the drafter. We will clarify this and add a configuration table for each headline result.

Q4. How often finite precision changes the selected position or token.

The differences come from the batch shape, not from drafting. Draft-free decoding at SPOT's padded batch shape reproduces SPOT's output on 96% of problems (100% on GSM8K, HumanEval and MATH500, and 85% on MBPP, whose long prompts change the last bits), and SPOT matches an unpadded standalone run as often as draft-free decoding at a different shape does. Correctness flips are balanced (19 vs 14 problems for SPOT, 20 vs 19 for SPOT-γ, out of 1,214), so accuracy does not move. We will state the batch-invariance assumption in Appendix A.1 together with this measurement.

Thank you again for the careful review. We believe your questions and suggestions improve the paper, and all of them will be included in the revised version. We hope the new checks resolve your soundness concerns and would be grateful if you took them into account in your final rating. We would be happy to answer any follow-up questions or clarify any remaining concerns.

Thank you for the detailed response. It addressed my concerns and I would keep my positive rating.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response to the AC and Reviewers

Dear AC and Reviewers,

We thank the reviewers for their positive evaluations and constructive reviews. We addressed every point with new experiments and clarifications, which strengthen the comparisons and leave the paper's conclusions unchanged. We have uploaded a revised PDF with every change marked in blue. Line numbers below refer to the revised PDF.

The gains are statistically significant: paired 95% bootstrap intervals put SPOT ahead of every baseline in all 48 comparisons of the main table, in both speed and target passes (NFE) (lines 416–419, 1992–1998, Table 27).

On the baselines (CU3s, 6Tv8), MRP now runs inside our harness under matched conditions, where SPOT outperforms MRP by 1.5–1.8× on every task, in both speed and target passes (lines 421–423, 1312–1317, Table 7). In each paper's own setting, our SSD-style and BlockSpec-style implementations reduce NFE at least as much as the papers report, and both will be part of our code release. Tuning the baselines' depth and budget does not change the conclusions (lines 1248–1257, Table 5).

On the protocol (CU3s, 6Tv8), the main table uses the sample counts stated in Section 5.1 and the appendix now matches it (lines 1378–1386). Every table states its sample count and seeds (e.g. lines 216, 382, 435–436), a new table lists the configuration behind each headline number (lines 402–403, 1225–1236, Table 4), and γ is calibrated on problems from the MBPP training split (line 1271, Table 6).

On finite precision (CU3s, 6Tv8), the appendix now reports the requested measurement. Output differences come from the batch shape and not from drafting, and accuracy does not change (lines 724–735). On greedy match (1ZDX), Section 3 now states that it is a negative control and that no claimed result uses it (line 233).

We sincerely appreciate again the time and effort of the AC and the reviewers, and their valuable feedback, which helped us improve the paper. Detailed answers are in the individual replies, and we would be happy to answer any follow-up questions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.