acceptodds
Under review as a conference paper at ICLR 2027

Accelerating Speculative Diffusions via Block Verification

Abstract

Speculative decoding speeds up LLM inference by using a draft model to generate tokens, with an acceptance-rejection scheme that ensures that the output matches the target distribution. Adapting this to continuous diffusions is difficult because speculative sampling requires drawing from a residual distribution. While straightforward in discrete spaces, efficiently sampling this residual in continuous space is non-trivial. Consequently, existing diffusion adaptations either use computationally inefficient sampling techniques or rely on an alternative scheme. In this work, we introduce a novel scheme that efficiently implements the original speculative sampling mechanism for diffusion models. Our approach offers a critical advantage over current methods: it enables us to adapt block verification from LLMs to diffusions—which provably improves the acceptance rate of drafts. Furthermore, we formalize and analyze the Free Drafter, a heuristic self-speculative drafter for diffusions that requires no training. By enabling block verification, our Free Drafter yields up to a 6.3% speedup over existing speculative methods with no additional training and negligible overhead beyond the existing parallel verification pass.

Primary area:
generative models
Keywords:
diffusion, sampling, speculative sampling, coupling

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_KrgROpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper develops a residual-sampling procedure for speculative diffusion by decomposing an equal-covariance Gaussian residual into a one-dimensional sampling problem and an orthogonal Gaussian component, enabling block verification in continuous spaces.

Critical strengths
  1. The residual decomposition provides a concrete technical mechanism for implementing block verification, supported by a sampling proposition and an analysis of why reflection-style deterministic corrections are insufficient.

  2. The experiments distinguish verification improvements from drafter improvements and include quality checks using FID for images and trajectory MMD² for molecular dynamics.

Critical weaknesses
  1. The incremental latency benefit of block verification is limited in some settings, and its statistical reliability could be assessed more directly. On ImageNet LDM, the gain over reflection-based verification is 2.32% at 250 steps and only 0.09% at 50 steps; the reported uncertainties concern each method’s speedup over DDPM rather than the paired improvement between the two methods.
  2. The relationship between theoretical exactness and the numerical sampler requires further clarification. The distribution-preservation argument assumes correct residual sampling, whereas the experimental implementation limits lower-bound search to three iterations and bisection to four, leaving the effect of this approximation on the sampling distribution unclear.
Questions for authors
  1. Could the authors report paired latency comparisons and confidence intervals for the improvement of block verification over reflection-based verification, particularly where the reported gains are small?
  2. How sensitive are the residual samples, output quality, and runtime to the numerical-search budget, and could the authors quantify the sampling error introduced by the three-iteration bound search and four-iteration bisection?
2 comments

We thank the reviewer for their precise comments. Please see our general comments.

Thanks for the authors' responses, which address my concerns. I keep my original positive rating.

Reviewer_squ8OpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies speculative sampling for diffusion models. Its main contribution is a residual-sampling procedure for Gaussian transitions with shared covariance. It reduces the sampling problem to a one-dimensional component and an independent Gaussian component. This construction enables the adaptation of block verification from autoregressive language models to diffusion sampling. The paper also analyzes the computational trade-offs between Frozen and Free drafting strategies. Experiments report additional wall-clock speedups of up to approximately 6.3% over reflection-based speculative diffusion, with similar FID scores.

Critical strengths

The paper aims to addresses an obstacle to applying block verification in continuous spaces

Training-free: The approach does not require training an additional draft model when using self-speculative drafting.

The efforts go beyond image generation to i.e. the molecular-dynamics experiments

Critical weaknesses

Limited evaluation beyond U-Net architectures and comparison with transformer-based speculative methods. All image-generation experiments use U-Net backbones. Can the proposed method work effectively onto the transformer based arehictectures? Are there comparisons between the method with Free[1]? Incremental gains: particularly with fewer denoising steps.

[1] FREE: Uncertainty-Aware Autoregression for Parallel Diffusion Transformers

The overall speedup over sequential DDPM should be distinguished from the additional benefit of block verification over existing speculative sampling. Exp (tab3) shows ~0.09% at 50 steps, 1.47% at 100 steps, and 2.32% at 250 steps. It somehow leave the practical significance uncertain for shorter sampling chains.

Implementation of lossless: The theoretical distribution-preservation guarantee assumes exact residual sampling. However, Appendix J limits the bound search to three iterations and the bisection search to four iterations. How is the resulting approximation error controlled, and could it accumulate over successive denoising steps? Does the implemented algorithm retain an exact distribution-preservation guarantee, or should its guarantee be described as approximate?

Questions for authors

See weakness

1 comment

We thank the reviewer for their insightful questions.

Can the proposed method work effectively onto the transformer based arehictectures?

Although we do not have experiments on transformer based architectures for image generation, our molecular dynamics (MDs) experiments do. The draft model is an attention-augmented GNN, Orb-v3 (Rhodes et al., 2025). The target model is an E(3)-equivariant GNN (Wood et al., 2025). Thus, our block verification scheme not only works effectively for a different target architecture, but also over different drafters and generation tasks.

Are there comparisons between the method with Free[1]?

We do not have specific experiments to compare against Free, but we note that their modifications for speculative sampling are complimentary to our paper. Indeed, the drafter considered in Free can be freely utilized instead of the discussed FreeDrafter with block verification --- block verification is drafter agnostic. We however, note, some of our theory will translate to the drafter of Free: particularly, Corollary 3.5 would hold if generating a draft is roughly the same runtime as a single target model call.

We also note that although we do not explore the (relaxed) schedule of Free, in Appendix K we consider a relaxed temperature variant of speculative diffusion (with block verification).

However, we do note that more specific drafters which could exploit the structure of block verification and specific temperature (relaxation) schedules are interesting future directions of work.

It somehow leave the practical significance uncertain for shorter sampling chains.

We hope our general comment addresses the significance uncertainty concerns. We do however concede that block verification in comparison to vanilla reflection sampling verification (and speculative diffusion for image generation in general) have limitation for small sampling chains. This is ultimately a limitation of speculative sampling for diffusion in general, see discussion around Eq (45). To make a long story short, due to the noise schedule required for diffusion models, at the end of a sampling chain, acceptance will be very small to zero. This will happen regardless of how good the drafter is for the verification methods explored and is exacerbated when the chains are short.

Implementation of lossless

Please see our general response on bisection search parameters. We agree that on the practical side, we should caveat that we are approximately lossless. Despite this, in practice, we find that the bisection search parameter do not change quality metrics by much.

Reviewer_rYNxOpenReview ↗
Rating
2
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes an efficient residual-sampling method for speculative diffusion models, enabling block verification in continuous state spaces. The key observation is that, for Gaussian draft and target transitions with identical isotropic covariance, the positive-part residual can be decomposed into a one-dimensional non-Gaussian component along the difference between the transition means and an independent Gaussian component in the orthogonal subspace. This reduces high-dimensional residual sampling to one-dimensional inverse-CDF sampling.

Building on this decomposition, the authors adapt block verification from speculative decoding to diffusion sampling, allowing more drafted denoising steps to be retained without changing the target distribution in the idealized algorithm. The paper also analyzes the computational trade-offs between the Frozen Drafter and a cached-score Free Drafter.

Experiments compare reflection-based sample verification, decomposition-based sample verification, and decomposition-based block verification across pixel-space and latent-space image generation tasks. The proposed block verification generally improves block efficiency and achieves up to approximately 6.3% higher wall-clock speedup relative to existing speculative methods, although the gains vary across settings. Additional experiments extend the approach to stochastic molecular-dynamics integrators.

Critical strengths

The main strength of the paper is the Gaussian residual sampling method in Proposition 3.1. It turns a difficult high-dimensional sampling problem into a much simpler one-dimensional problem. This makes continuous block verification easier to implement without expensive rejection sampling. The mathematical derivation is also clear and convincing.

The experiments are another strength. The R/D/B comparisons help show where the efficiency improvements come from. The comparison between Frozen and Free Drafters also shows that higher acceptance rates don't always lead to faster inference. Finally, the molecular dynamics experiments suggest that the method can work beyond image generation, although the overall speed improvements are relatively small.

Critical weaknesses

The speed improvements are quite small. The proposed method is usually only a few percent faster than reflection-based verification, and the differences may be within the experimental variation. Without confidence intervals or statistical tests, it's hard to tell whether the improvements are significant.

Block verification improves efficiency by around 8%, but the actual runtime improvement is only about 2%. It's unclear how much time is spent on residual sampling and other calculations. A breakdown of the computational costs would help explain this gap.

The experimental evaluation is somewhat limited. Most experiments use self-speculation with U-Net image models. It would be useful to test the method with a separate, smaller drafter or a more modern diffusion architecture. This would help show whether the method generalizes beyond the current settings.

Questions for authors

If all methods sample from the same target distribution, why do they produce different FID scores? Could this be due to numerical errors or random variation?

The speed improvement over reflection-based verification is only a few percent. Is this difference statistically significant, and does it remain when each method uses its best draft length under the same computing budget?

2 comments

We thank the reviewer for their detailed comments.

If all methods sample from the same target distribution, why do they produce different FID scores? Could this be due to numerical errors or random variation?

We would like to clarify that the different FIDs are due to "numerical variation" from GPU parallelization. There is some numeric errors from the bisection search, but this isn't significant (see the general comment "Bisection"). We note that prior work utilizing speculative diffusions also have such variation of FIDs, eg, De Bortoli et al. (2025); Hu et al. (2025).

The speed improvement over reflection-based verification is only a few percent. Is this difference statistically significant, and does it remain when each method uses its best draft length under the same computing budget?

Please see our general comment "Significance". We would also like to note that it is typical in the speculative sampling literature to fix the draft length, and then test different speculative approaches, eg, original block verification paper Sun et al. (2025).

Block verification improves efficiency by around 8%, but the actual runtime improvement is only about 2%. It's unclear how much time is spent on residual sampling and other calculations. A breakdown of the computational costs would help explain this gap.

Although we do not have explicit experiments / timing for this, Appendix H dives into the theoretical breakdown of computational cost.

The experimental evaluation is somewhat limited. Most experiments use self-speculation with U-Net image models. It would be useful to test the method with a separate, smaller drafter or a more modern diffusion architecture. This would help show whether the method generalizes beyond the current settings.

Given the theory focus of the paper, we hoped to provide a diverse set of experimental settings. To deviate from the self-speculative U-Nets, we thought that the molecular dynamics (MDs) experiments would be an interesting test-bed. As the MD experiments not only differ in the aforementioned ways, they also test whether block verification has usefulness outside of image generation. More concretely, the MD experiments are not self-speculative and utilizes draft and target models Orb-v3 (Rhodes et al., 2025) and UMA-S-1.1 (Wood et al., 2025), which correspond to attention-augmented GNNs and E(3)-equivariant GNNs, respectively, which deviate greatly from U-Nets used in image generation. Indeed, Orb-v3 has a (equivariant) transformer architecture (in reference to other reviewer's comments about transformer based architectures).

We would just like to quickly follow up your comment about a breakdown in computational cost

Block verification improves efficiency by around 8%, but the actual runtime improvement is only about 2%. It's unclear how much time is spent on residual sampling and other calculations. A breakdown of the computational costs would help explain this gap.

Please see "Timing Breakdown". Notice the low runtime of draft, verification, and residual components compared to the target model evaluation. When the "target eval" cost is lower, the effects of block efficiency on runtime are diminished; the converse is expected under comparable acceptance and parallel execution.

Comments on the paper

3 comments
AuthorsOpenReview ↗

General Response

We thank the reviewers for their thoughtful comments. We are glad to see that the reviews found that our work provides a "concrete technical mechanism for implementing block verification" (KrgR) with mathematical derivations which are "clear and convincing" (rYNx). We are also glad to see the appreciation of the diverse set of experiments "beyond image generation" (squ8).

In the following, shared comment, we would like to address two points shared between different reviews. Namely, we discuss (1) significance / error bars and (2) bisection search parameters.


Significance

For the former, we thank all reviewers for flagging the need for a rigorous statistical evaluation. The particular issue comes with the informal usage of the "R↑B%" ratio improvement metric. As the improvements are relatively small, ratios do not yield nice statistical intervals. As such, we agree with reviewers that something more formal should be used to ensure gains are not artifacts of, e.g., random wall-clock variation. To do so, we have added paired speedup differences "BΔR\text{B}\Delta\text{R}" with the corresponding 95% CIs. These are calculated via the standard Welch’s tt-test. The resulting CIs verify that block verification improves runtime, with large number of denoising steps having better (delta more positive) intervals. We are propagating these changes to the manuscript to the relevant tables (see updated tables in the submission), where we make the difference between block and reflection speedups the main focus "BΔR\text{B}\Delta\text{R}", while de-emphasizing the previously expressed ratios.

We would like to further comment about speedups. We believe that our draft honestly states the limitations of speculative sampling (even beyond our approach), e.g., Section 5 "Number of sampling steps" and Section 6; also see discussion around (45). Speculative diffusion as currently stated in the literature has limited speedup opportunities when the number of denoising steps are low. In-fact the probability of accepting a draft will theoretically vanish as we get close to the end of the denoising process (see Eq (45) + discussion; Figure 3). Despite this, we believe that merits of speculative diffusion and block verification are still useful contributions to the literature when we have large number of denoising steps. The adjusted changes above show that improvements are statistically "visible" for large number of denoising steps. And in-general, as our approach is drafter agnostic, the original reflection sample verification can be readily switched for block verification to yield a, possibly small but, consistent runtime improvement. Nevertheless, in general, we believe that given that improvements speculative diffusion can be easily made by switching verification, even with small improvements over reflection sample verification, our block verification method still has merits.


AuthorsOpenReview ↗

General Response cnt.

Bisection

For the latter, we also thank all reviewers for their interest in the details of the bisection search. The following consists of complimentary experiments which show the (lack-of) changes caused by different bisection parameters. These experiments are based on the setup of Hu et al. (2025) on Stable Diffusion v2 which also examines the CLIP score of images. Other results based on Stable Diffusion v2 were started post submission (but are not ready due to compute constraints). We expedited the experiments relevant to bisection parameters.

Specifically, we use the following configuration:

  • Compute resources: Same as molecular dynamics experiments, ie, 8 GH200 GPUs.
  • Target model: Stable Diffusion v2.
  • Drafter: FreeDrafter.
  • Denoising steps: 500.
  • Draft length: 7.
  • Generated images: 2,000 per method.
  • Classifier-free guidance scale: 7.5.
  • FID evaluation: Against COCO reference images.

We compare runtimes, CLIP score, and FIDs over DDPM versus block verification with different bisection parameters. In particular we consider nn-lower bound doubling steps and (n+1)(n+1)-bisection steps, signified by n/(n+1)n/(n+1). We yield the following table:

ApproachRuntime (s) [CI 95%]CLIP [CI 95%]FID
DDPM17.40 [17.40, 17.41]30.19 [30.07, 30.31]48.29
Block 1/212.13 [12.07, 12.18]30.18 [30.06, 30.29]47.62
Block 3/412.11 [12.05, 12.16]30.19 [30.01, 30.31]47.89
Block 7/812.12 [12.06, 12.17]30.19 [30.06, 30.30]48.16
Block 31/3212.11 [12.05, 12.16]30.19 [30.07, 30.31]47.93

Runtime is per sample generated. We note that the FIDs are relatively high (although the CLIP scores more or less match Hu et al. (2025)). This is likely due to the smaller number of denoising steps (one may want 1000) and the limited number of generated images used (2000 instead of 5000). Nevertheless, what is important is the relative performance and consistency of block verification with different bisection parameters.

Summarizing the results:

  • Even low(est) bisection search parameters do not affect the CLIP scores much;
  • Time nor quality does not change significantly over these ranges.

Although it may seem surprising, this is not a bug. First, in comparison to the evaluation of the target model, increasing the number of bisection steps is insignificant. Hence, the runtime does not change much (Eq (42) + Huang et al. 2025 gives the formal picture of where speedups would come from). Second, recall from Proposition 3.1 that the residual have the form Y=mq+σ(Ue+Z⊥)Y=m^q+\sigma(Ue+Z_\perp). Bisection approximates only the scalar UU. The target mean mqm^q and perpendicular Gaussian component Z⊥Z_\perp are exact. Assume that the lower bound search succeeds (and the bisection search range is uniformly bounded for the sampling chain). Writing U^(B)\hat{U}^{(B)} for the output of BB bisection iterations, then the difference between exact and bisection outputs are given by

=\sigma \vert \hat{U}^{(B)}-U \vert =O\left(\frac{\sigma}{2^B}\right).$$ We have the usual rate of error decrease with bisection steps, but also have that the error is weighted by the noise schedule $\sigma_k$. The behavior of the noise schedule and how it interacts with speculative diffusion acceptance yields a couple of insights, see Eq (45) + Figure 3. - The noise schedule has that as $k \to K$, then $\sigma_k \to 0$. - From Eq (45) + Figure 3, draft acceptance is highest at the start of the sampling chain, and then vanishes at the end. - At the start of sampling, residual calculations are avoided as drafts are more likely to get accepted. - At the end of sampling, residual calculations are most common but their error vanishes due to the above equation. Hence, residual approximation errors are introduced less frequently at high noise levels and are diminished by smaller noise scales when residual corrections are most frequent, providing a plausible explanation for the similar image quality across bisection budgets. Although this only holds for a single step, the above observed stability of CLIP and FID scores provides empirical evidence that error in the residuals do not result in a large impact on final image quality. Nevertheless, we will make sure to include such discussions in the next version of the paper while also carefully caveating that in practice our approach is not lossless (in the sense some approximation must be made via bisection).
AuthorsOpenReview ↗

Timing Breakdown

Dear reviewers, we would just like to add the following table. This is intended to complement the previous general comments.

Timing Breakdown

We present a timing breakdown utilizing the same settings as our "Bisection" comment above, with the following changes:

  • The following results use the 31/32 bisection setting. We check that the numerical CDF discrepancy "tol", ie ∣Ψα(U^)−u∣|\Psi_{\alpha}(\hat U)-u|, is at most 10−710^{-7}, where uu is the sampled uniform random variable in our inverse sampling.
  • The statistics are over 55 images generated per method, using the same five prompt/seed pairs over different methods.
  • K=1000 denoising steps.

Entries are mean seconds per complete trajectory, with 95% percentile-bootstrap CIs (2,000 resamples of the five pairs).

ComponentReflection: sec (% total) [95% CI]Block: sec (% total) [95% CI]
Draft0.59 (3.73%) [0.54, 0.65]0.56 (3.69%) [0.55, 0.59]
Target eval14.24 (89.92%) [12.96, 15.63]13.32 (87.07%) [12.89, 13.81]
Verification (arithmetics / non-model calls)0.71 (4.50%) [0.65, 0.79]0.99 (6.47%) [0.96, 1.02]
Residual0.051 (0.33%) [0.046, 0.058]0.158 (1.03%) [0.148, 0.168]
Total15.84 (100.00%) [14.44, 17.34]15.30 (100.00%) [14.81, 15.83]

Takeaways:

  • From reflection to block, there is an increase in residual sampling cost, but it is small compared to the target model evaluation.
  • The block bisection residual cost is small even with the 31/32 bisection setting. Other parts of speculative sampling which reflection / "regular" verification share with block verification are larger than the residual time in either case.
  • At fixed latent dimension and search budget, the cost per individual residual correction does not directly depend on the target model size. For more expensive target evaluations, this overhead should therefore become relatively smaller (assuming similar block efficiency, draft costs, parallel costs etc).
  • Given the cheapness of the residual calculation, our sampling scheme (Prop 3.1) could be a key step in more sophisticated drafting strategies (ie, future work, see Sec 6 "Discussion" in the paper).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.