acceptodds
Under review as a conference paper at ICLR 2027

Stein Guidance without Gradients: Discretization Uniform Diffusion Inference Time Scaling beyond Particle SMC

Abstract

Diffusion inference can scale through more solver steps or larger trajectory ensembles. These define the numerical depth and statistical width . In Sequential Monte Carlo (SMC) with full resampling at every solver step, the resampling depth also equals . A scalable method should decouple depth and width: the number of trajectories needed for a prescribed sampling accuracy should remain bounded as numerical depth increases. Repeated resampling can violate this principle: we construct a Brownian example with single-output total variation (TV) error for . This raises a central question: how can we decouple sequential and statistical complexity, so that increasing does not force to grow merely to maintain accuracy? We propose \method, a family of derivative-free guidance methods based on Gaussian Stein's identity, with whole-path independence Metropolis–Hastings (MH) correction. Each proposal uses value-weighted candidate selection along one trajectory. The value function need not be exact. Approximate values steer trajectories toward promising regions, while a derivative free MH correction applied after generation can asymptotically remove the bias introduced by the approximation. Generating proposal paths independently avoids the shared ancestry caused by resampling across particles. Under uniform value and spatial bounds, our theory guarantees that the number of complete proposals needed for a prescribed finite-grid sampling accuracy stays bounded or even decreases as numerical depth increases. Stein guidance also parallelizes value queries within each trajectory. At a fixed per-proposal query budget, the pool-noise term in the error bound has the same order for every pool size .

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
Diffusion Model, Stein's Identity, Guidance, Inference Time Scaling, Sequential Monte Carlo

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_GmBDOpenReview ↗
Rating
2(+1)
Confidence
3(+1)
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper introduces a novel particle-based diffusion guidance method, which leverages Stein's identity for scaling diffusion guidance problem in a derivative-free manner.

Critical strengths

With the current presentation, I can not identify any major strengths of this paper. Even though the paper raises some interesting problems such as training-free guidance, derivative-free guidance methods, particle-based diffusion guidance, scalability, etc. Some of them are in my area of expertise, but I can not recommend that they are their strengths because I can not clearly position their work compared to existing literature within their current paper version.

Critical weaknesses

I think that there are several major weaknesses of this paper.

  1. The way the authors present their research is too ambiguous. When reading through the paper, there are paragraphs that make sense, but there are also several disconnected paragraphs. The writing is extremely difficult to follow, with very complex logical reasoning flow and abundant unclear math formulation, which make very hard to digest the paper's information. The authors introduced different complex terminology without enough sufficient background. I would recommend the authors make an effort to restructure their paper to clarify the background/ existing literature and their contribution with respect to existing work.

2. Another major concern is the paper formatting. The paper seems to be written with different character fonts and font sizes. Especially, we can observe this in the paper title, sections, and math equations. In addition, a weird running header with the title "Manuscript prepared for ICLR 2027" but not "Under review as a conference paper at ICLR 2027", which somehow shows that the authors did not even use the main conference paper format. Eventually, the paper has no conclusion section, Section 2 is too short, and Section 3 and Subsection 3.1 run with the same title.

  1. There is an important related work, Stein Diffusion Guidance (SDG) [1], that proposes a very similar work. SDG also leverages Stein variational inference methods to correct biased approximate diffusion posterior samples and remains completely training-free without fine-tuning pretrained models. Even though the paper was released on arXiv in July 2025 and later accepted to ICML 2026, the authors seem to completely ignore this related work.
Questions for authors

I don't have any particular technical questions about this paper since I can not truly understand the paper, even from its abstract, after spending hours reading.

References

[1] Stein Diffusion Guidance: Training-Free Posterior Correction for Sampling Beyond High-Density Regions, ICML 2026. Nguyen et al.

5 comments

We thank the reviewers for the detailed feedback. We have substantially revised the manuscript to address the concerns about clarity, readability, organization, and positioning, and have posted the updated version (with changes in red).

On presentation

Overall revision. We were short on time before submission and did not polish the presentation as carefully as we should have. Following your feedback, we have substantially revised the motivation and technical explanations to make the paper clearer and easier to follow. We expanded the background, clarified our contribution and its relation to existing algorithms, and made the connection between guidance and correction more explicit. We also reduced auxiliary notation, repeated definitions, and technical digressions. The changes below illustrate this overall direction.

Specific changes include:

  • Abstract and background: The abstract previously opened with depth and width terminology; it now starts with the sampling problem and our guidance and correction design. We also expanded Section 2's brief introduction to define Bellman error and explain how resampling fluctuations become terminal bias.

  • Stein derivation (Section 3.1.2): We replaced several auxiliary estimators with a direct derivation of the selected candidate's mean and its guidance drift, reducing notation and making the connection to Stein's identity explicit.

  • Weighting argument (Section 3.2): The original text referred to Appendix C.1 for why the weight yields the target marginal. We now spell out the path weight and telescoping identity in the main text, with an explicit reference to the one-step identity in Appendix B.1. This directly shows why we are using the correct importance weight for the correction.

  • Formatting and organization: We replaced the mixed font setup and custom running header with the ICLR template settings, corrected the repeated section title, and added a concise conclusion summarizing the main takeaway.

On Stein Diffusion Guidance (SDG)

We thank the reviewers for pointing out this omission and bringing SDG [1] to our attention. We have added an intuitive discussion and a schematic comparison in Appendix A.

SDG uses kernelized Stein variational inference to regularize clean predictions, aiming to mitigate reward hacking associated with approximate guidance surrogates. Our method uses the classical Gaussian Stein identity (Gaussian integration by parts) for value-based candidate selection, while correction is performed by whole-path MH. Under the stated assumptions, increasing the number of MH updates makes the output arbitrarily close in total variation to the prescribed finite-grid target, even with approximate intermediate values.

References

[1] Van Khoa Nguyen, Lionel Blondé, and Alexandros Kalousis. Stein Diffusion Guidance: Training-Free Posterior Correction for Sampling Beyond High-Density Regions. ICML, 2026.

I appreciate the updates. While Stein Diffusion Guidance [1] is the most relevant work, the authors only cited it in the Appendix and never discussed it in the main text. However, the citation to this paper remains very skeptical (lines 601-602); I can not find any available .bibtex files that correspond to this citation. In addition, Stein Diffusion Guidance also performs a similar benchmarking on image tasks, but the authors never compare against it. Regarding their experiments, I consider they are not significant. In Figure 3, classical FK steering already performs competitively with the proposed approach. In the text-to-image generation (Section 5.3), the authors only showed some cherry-picked photos; there are no reported quantitative metrics, i.e., CLIP score [2], which makes it difficult to judge their actual model performance. Given the improperly formatted initial manuscript, improper discussion of related work, citation, and comparison, and weak evaluation protocol, I don't think this manuscript is ready for publication.

References

[1] Stein Diffusion Guidance: Training-Free Posterior Correction for Sampling Beyond High-Density Regions, ICML 2026. Nguyen et al.

[2] TFG: Unified Training-Free Guidance for Diffusion Models, NeurIPS 2024. Ye et al.

Thank you for the follow-up. We have to disagree with the assessment that our contribution is insignificant.

Our main experiments already demonstrate 36–39% lower critic-QQ KS than FK across the three target strengths, which is far from being "little improvement". We further compare independently tuned FK and DAS on the Hopper 80% target, using the same Late-5 guidance window and runtimes matched within 5%. Entries report KS (ms/output):

LLOursFKDAS
160.0931 (0.458–0.471)0.1557 (0.463)0.1618 (0.485)
320.0600 (0.914–0.916)0.1117 (0.895)0.1117 (0.881)

Times are median batched ms per output, including proposal generation, model evaluations, and correction. At L=32L=32, our method achieves approximately 46% lower observed KS than both baselines at matched runtime.

We also showed strict improvement for the SMC-type algorithms analysed (like FK or DAS): our superpolynomial convergence versus 1/N1/N of SMCs, our bound uniform in guidance depth nn, versus the growing-in-nn phenomenon of SMCs, under the stated assumptions. Specifically, the SMC error can grow with nn, meaning that smaller step sizes can worsen sampling accuracy at a fixed particle budget. This means that even if FK were, to some extent, competitive at a particular discretization, this would not negate the theoretical improvement or the significance of the contribution.

Furthermore, DAS and SDG require reward gradients, whereas our method uses only value queries. Our empirical gains therefore come without the additional gradient access available to these baselines. These empirical gains and theoretical advances establish a substantive contribution; the assessment of insignificance does not adequately account for either.

We also disagree that SDG is the closest methodological baseline. The shared word “Stein” in the two titles does not, by itself, justify identifying SDG as the most relevant prior work. That judgment must rest on the problem setting, sampling objectives, algorithmic mechanisms, and guarantees. Its kernelized SVGD regularizes denoising posteriors; our Gaussian Stein construction enables value-only guidance with whole-path correction to a prescribed tilted distribution. It is especially clear to see that SDG is not itself an asymptoticaly consistent algorithm under our setting. Our most relevant baselines such as SMC-type algorithms, DOIT were all designed based on asymptotic consistency. We explain these distinctions in the appendix. The previous entry referred to arXiv v3 of the same paper; we have now replaced it with the official ICML 2026 entry, we believe that this was a minor, fixable factor.

Nevertheless, we have directly evaluated paper-based SDG adaptations under our main experiment settings, with validation-based parameter selection and independent confirmation. The table below reports critic-QQ KS for the best-performing SDG adaptation among those tested in each setting, with N=32N=32:

Method30%60%80%
Ours, K=2,L=32K=2,L=320.03060.06970.1128
SDG, N=32N=320.17090.24750.3337

Even the strongest SDG adaptation tested at each target incurs approximately 3.0–5.6 times our KS error. Thus it is far from being competitive with other strong baselines even if the reviewer insists that it should be compared.

About our text-to-image experiment: In our updated draft, our revised image evaluation also reports CLIP score, HPSv2, and ImageReward over 40 GenEval two-object prompts. Our K=2K=2 variant achieves the highest mean on all three metrics, providing quantitative evidence beyond the displayed examples.

MethodCLIP score ↑HPSv2 ↑ImageReward ↑
FK-Steering, N=4N=40.28830.25840.3489
SVDD, K=4K=40.27610.24510.0098
URGE, N=4N=40.28640.24980.5723
Ours, K=2K=20.29610.26220.8290
Ours, K=4K=40.29250.25990.3445

Finally, we appreciate the reviewer identifying the initial formatting issues. These are relatively straightforward to resolve and have already been corrected. These minor issues should not be conflated with the scientific significance of the contribution. We believe that the final assessment should address the revised manuscript and the evidence above.

Reviewer_zKE2OpenReview ↗
Rating
4
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper samples from tilted diffusion measures using a local branching algorithm combined with selection using local importance weights. It exploits Stein's identity to avoid calculating the gradient of the intermediate potentials. The proposed trajectories are corrected with Metropolis-Hastings updates. The authors presented this algorithm as an alternative to SMC that they claim to degrade with the number of resampling steps (statistical depth). The paper is completed by several numerical experiments, in particular one on prompted images.

Critical strengths
  1. The proposed algorithm is elegant. Stein Guidance and local branching are really in synergy here. The former allows unbiased un-normalized proposals to be constructed without knowing the gradients, and the latter allows to choose the next value of the trajectory. The Metropolis-Hastings correction is indeed invariant with respect to the target distribution.
Critical weaknesses
  1. The criticism of SMC is in my opinion largely unfair. The authors say that SMC doesn't scale well with the number of resampling steps, but only considers multinomial resampling. I believe that with, e.g. systematic resampling, this problem would largely disappear. Additionally, equation (12) claims that the proposed algorithm converges faster than the O(1/N) rate of SMC. In fact, conditional SMC (also known as Particle Gibbs) also enjoy MCMC-like rate of convergence, which can be exponential in compact spaces.

  2. When KK is big, most particles at each iteration of Step 1 is thrown away.

Questions for authors

I'm inclined to accept because of the algorithm is conceptually nice. A question and a suggestion:

  1. I would like more precision on how you develop the estimates to the potential HkH_k. In which situations do we have estimates of HkH_k but don't want the gradient? Why can't we estimate the gradients directly?

  2. The presentation can be significantly improved. Appendix B gives a methodic presentation but is never referenced to in the main text. Having worked with SMC regularly, it still took me considerable time to understand the invariance of Algorithm 2.

3 comments

We thank the reviewers for the thoughtful comments and for the recognization of our work. Taking the advices, we have made updates to our paper.

On SMC's alternative resampling schemes

We agree that systematic resampling can substantially mitigate the problem, and our main-text Θ(n/N)\Theta(n/N) theorem specifically concerns multinomial resampling.

However, reducing resampling noise alone does not guarantee depth-uniform accuracy. SMC redistributes mass through interaction among particles: resampling replicates some trajectories and discards others, allowing dependence through shared ancestry to accumulate. Systematic resampling reduces the randomness in offspring counts, but does not generally eliminate replication or shared ancestry under unequal weights.

We looked into this problem with more details, and our additional analysis considers systematic resampling as the reviewers have mentioned, with candidates sorted by state before each resampling step, in a fixed smooth Brownian problem with approximate guidance. For the law of a uniformly selected terminal particle, we found the upper bound

\le C\log(eN)\left(\frac1N+\frac{\color{red}\sqrt n}{N^{3/2}}\right), \qquad n,N\ge1,$$ and a lower bound (under a constructed example) $$\\|\nu_{n,N}^{\mathrm{sys}}-\eta\\|_{\mathrm{TV}}\ge d > 0, \qquad N\ge N_0,\quad n\ge AN^3,$$ with constants independent of $n,N$. The upper bound gives consistency when $N\to\infty$ and $n=o(N^3/\log^2(eN))$, while the lower bound shows persistent error once $n\ge AN^3$. **These bounds identify that despite systematic resampling can improve the scaling substantially, a dependence on guidance depth can remain.** > On comparison with conditional SMC and Particle Gibbs We agree that conditional SMC/Particle Gibbs can also achieve **superpolynomial convergence** in the number of MCMC iterations. Our comparison concerns the finite-particle error of conventional SMC used commonly for inference-time scaling, instead of other general family of SMC variates. We've mentioned the superpolynomial convergence of these MCMC-type SMCs in the footnote of our updated version. Our analysis additionally characterizes **how the error bound improves as approximate value functions approach the discrete Doob values**, separating accumulated Bellman error from finite-pool variance. Establishing analogous quantitative dependence for Particle Gibbs in this setting requires a separate analysis beyond the scope of this paper. Though we agree that such methods may admit similar guarantees. Finally, **fast convergence at a fixed guidance depth does not by itself imply depth-uniform mixing**: the mixing constants may still deteriorate as depth increases at a fixed particle count. Our theorem explicitly provides a depth-uniform (in fact improving) TV bound under the stated assumptions. > On discarded candidates at large $K$ We agree that only one of the $K$ candidates is propagated at each step. **Using multiple auxiliary samples for one output is common in diffusion inference**: SVDD selects one candidate from a pool, DOIT uses $M$ Monte Carlo samples to estimate one guidance update, and SMC can select one terminal particle when a single output is requested. From a query-complexity perspective, the relevant question is **how many value queries are needed to obtain an accurate sample**, rather than how many candidates are retained. In our method, all $K$ candidate values contribute to both the selection probabilities and the pool normalizer used in the MH correction. Thus, candidates that are not propagated still contribute to the sampling procedure.

On why need estimating intermediate value functions without gradients

Intermediate values, which are idealy the Doob's hh-transforms hk(x)=E[ϕ(XT)∣Xk=x]h_k(x)=\mathbb E[\phi(X_T)|X_k=x] can be approximated through Monte Carlo.

For a Monte Carlo estimate of the Doob value, direct pathwise differentiation generally requires differentiating the terminal reward and backpropagating through all remaining rollout steps. This is extremely costly and becomes unavailable for black-box rewards, such as LLM-as-a-judge, human evaluations in images or task-success indicators in RLs and robotics.

Even for a differentiable neural value surrogate, standard backpropagation requires storing intermediate activations or additional recomputation. Our K=2K=2 construction uses two parallel value evaluations to recover the leading guidance drift in expectation, without explicitly differentiating the value function. These settings motivate using value queries even when gradient estimation is possible in principle.

On presentation and the invariance of Algorithm 2

We acknowledge that limited time for final editing before submission left the original presentation insufficiently polished. We have since substantially revised the manuscript, with changes highlighted in red in the updated PDF. We have revised Section 3.2 and added explicit references to Appendices B.1 and C.1. The main text now explains the proper weighting identity, defines the augmented target and its target path marginal, and derives the MH acceptance probability 1∧R′/R1\wedge R'/R. The key reasoning behind Algorithm 2’s invariance is now stated directly in the main text, supported by showing that R/ZnR/Z_n is indeed the importance weight for the change of measure we are looking for.

We thank the reviewer again for the constructive feedback, which has helped us improve the clarity and presentation of the paper.

Reviewer_zKE2OpenReview ↗

Acceptance

I'd like to thank the authors for taking into account my points. I keep my positive ratings.

Reviewer_SnWpOpenReview ↗
Rating
3
Confidence
1
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies sampling from reward-tilted diffusion models. The authors construct an example in which SMC with resampling at every step has a TV error of order n/Nn/N, where nn is the discretization depth and NN is the number of particles. In this example, maintaining accuracy as depth increases requires more particles. The proposed method combines centered Gaussian candidate selection, motivated by Stein’s identity, with whole-path Metropolis–Hastings correction. Under boundedness and smoothness assumptions, the authors show that the number of complete proposals needed for a fixed accuracy remains bounded independently of depth. Offline reinforcement learning experiments show that MH correction brings the sampled QQ-value distributions closer to their targets, while text-to-image examples show better generation of scenes containing both requested objects.

Critical strengths
  • This paper proposes a simple method for important task of diffusion inference with reward with provable theoretical convergence guarantees and experiments validation.
Critical weaknesses
  • under assumption 3, it is unclear whether existing methods, such as standard path-wise MH, has similar convergence.

  • The paper does not report runtime comparisons, leaving the practical efficiency unclear. The lack of an unguided MH baseline also makes it difficult to understand the benefits of stein guidance.

Questions for authors
  • Could you compare your method with standard MH to isolate the contribution of stein guidance?

  • Could you report distribution accuracy and runtime under matched computational budgets for these methods?

2 comments

We thank the reviewer for the positive assessment of our work and for raising these questions. Below, we clarify the role of guidance and present new comparisons with unguided MH and other baselines.

On standard path-wise MH and the contribution of Stein guidance

We agree that standard whole-path MH also admits superpolynomial convergence with constants uniform in guidance depth under Assumption 3. Unguided MH can be viewed as the constant-guidance case of the same proposal-and-correction framework, while Stein guidance allows us to use intermediate value estimates to improve this process.

This additional information matters because our analysis quantifies how the correction error decreases as the intermediate values become more accurate. Specifically, our bound takes the form

\lesssim_p \frac{1}{m^p} \left( \mathcal A_{2p+2}^{\\,2} +\frac{Th}{K-1} \right)^{(p+1)/2}.$$ As the intermediate values approach the discrete Doob values, **the Bellman-error contribution $\mathcal A_{2p+2}$ vanishes, reducing the coefficient of the correction error**. In the ideal Doob limit, the proposal samples the target directly and requires **no correction at all**; our finite-pool implementation retains only the additional $Th/(K-1)$ contribution. **Base MH, however, does not use intermediate values, so its convergence constant remains unchanged as these values become more accurate.** This is the purpose of guidance: using informative intermediate values to reduce the mismatch between the proposal and the tilted target, leaving less work for MH correction. Unguided MH must instead correct the full variability of the terminal importance weight $\phi(X_n)/Z_n$. **Stein centering enables effective guidance with small candidate pools**, reducing the local pool-average variance from $O(h/K)$ for independent candidates to $O(h^2/(K-1))$. The benefit is therefore particularly relevant under stronger tilts, where the base distribution is less well matched to the target and informative guidance has more room to improve the proposal. > On isolating the contribution of guidance We added an unguided whole-path MH baseline and compared accuracy at the same number of MH updates $m$. We report the KS distance between the sampled and target critic-Q distributions; lower is better. | Target | MH updates $m$ | Base MH KS | Guided MH KS | | --- | --- | --- | --- | | Hopper 80% | 7 | 0.3738 | **0.1779** | | Hopper 80% | 31 | 0.1714 | **0.0611** | | Walker $\beta=2$ | 7 | 0.2876 | **0.0425** | It can be found that **our stein guidance substantially improves accuracy beyond MH correction alone.**

On distribution accuracy and runtime

We expanded the runtime evaluation to include independently tuned FK and DAS and five additional baseline families. All methods use the same guidance windows: Late-5 for Hopper and Late-8 for Walker. FK and DAS are matched to our runtime within 5%. The other five baselines use validation-selected higher-effort configurations, fixed per dataset and repeated across the displayed LL values.

Each entry reports mean KS (ms/output); lower KS is better. Here, L=m+1L=m+1 includes the initial proposal. Runtime includes sampling and correction, measured over five independent timing blocks on one RTX 5070.

Hopper, 80% target

LLOursFKDASDOIT (native)DOIT (fixed)SVDD-PMTFGTTS-local
40.2477 (0.152–0.154)0.2345 (0.152)0.2930 (0.158)0.6697 (1.861)0.3520 (1.872)0.4322 (1.272)0.3628 (1.640)0.4472 (1.174)
160.0931 (0.458–0.471)0.1557 (0.463)0.1618 (0.485)0.6697 (1.861)0.3520 (1.872)0.4322 (1.272)0.3628 (1.640)0.4472 (1.174)
320.0600 (0.906–0.916)0.1117 (0.895)0.1117 (0.881)0.6697 (1.861)0.3520 (1.872)0.4322 (1.272)0.3628 (1.640)0.4472 (1.174)

Walker, β=2\beta=2

LLOursFKDASDOIT (native)DOIT (fixed)SVDD-PMTFGTTS-local
20.0840 (0.296)0.0983 (0.296)—0.4673 (5.369)0.1844 (5.390)0.1134 (4.274)0.1572 (2.566)0.0846 (3.691)
40.0530 (0.556)——0.4673 (5.369)0.1844 (5.390)0.1134 (4.274)0.1572 (2.566)0.0846 (3.691)
80.0456 (1.094–1.108)0.0404 (1.125)0.0534 (1.093)0.4673 (5.369)0.1844 (5.390)0.1134 (4.274)0.1572 (2.566)0.0846 (3.691)
160.0386 (2.235–2.251)0.0470 (2.293)0.0359 (2.351)0.4673 (5.369)0.1844 (5.390)0.1134 (4.274)0.1572 (2.566)0.0846 (3.691)

Times are batched costs per output. “—” indicates an unavailable strict time match.

Our method is competitive with validation-tuned FK and DAS at matched runtimes. On Hopper, it achieves substantially lower observed KS at L=16L=16 and L=32L=32, with approximately 46% reductions against both baselines at L=32L=32. On Walker, all three methods reach a very similar accuracy range, with the ordering varying across budgets. Our independent-proposal structure also provides additional opportunities for parallel execution. All complete trajectory proposals used by the outer MH correction can be generated independently and in parallel, alongside parallel candidate evaluations within each step. Once the proposals are available, only a lightweight ordered MH accept/retain sweep remains. This avoids population-wide resampling barriers and creates further opportunities to improve the accuracy–latency tradeoff on larger parallel systems.

We thank the reviewer again for these suggestions, which helped us clarify the role of guidance and strengthen the practical evaluation.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.