acceptodds
Under review as a conference paper at ICLR 2027

Gradient-Free Sampling from Generative Models via Stochastic Bounded Extremum Seeking

10 views

Abstract

We introduce a sampling approach for energy-based and score-based generative models that requires no gradient evaluations of the model. Replacing the drift term that would normally contain the score with a high-frequency dithered cosine of the model's , , produces, in the high-frequency averaging limit, Langevin Markov chain Monte Carlo for energy-based models and the reverse-time SDE of score-based diffusion. We prove that trajectories of the dithered Itô SDE converge to those of the target SDE, driven by the same Brownian motion, uniformly on compact time intervals in probability, by an averaging argument that extends bounded extremum seeking (ES) to Itô processes, with an explicit mean-square rate under global bounds. The algorithm does not require derivatives: only forward evaluations are used, one per time step regardless of dimension. The approach is not confined to smooth targets: it extends to energies with discontinuous curvature (without ellipticity requirement) and to Sobolev energies whose Hessians exist only off measure zero sets; for Lipschitz energies with gradient kinks the averaged limit remains well posed, and the convergence proof uses an -uniform occupation estimate for the dithered process, which we prove by a near-identity change of variables that removes the dither pathwise before Girsanov's theorem is applied; the Krylov-Röckner integrability class is the boundary of provability. The approach provides a hard bound on the per-step update rate and applies to explicitly time-varying targets on finite horizons. Gradient-free pixel-space sampling is not competitive with well-tuned backpropagation-based samplers at practical evaluation budgets; the regime where the approach offers an advantage is latent-space sampling when the model is a black box and the target drifts in time. We demonstrate latent-space tracking for time-varying images on CelebA-HQ () from limited 1D projection measurements and latent-space EBM sampling on CIFAR-10.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
gradient-free sampling, zeroth-order methods, Langevin Monte Carlo, energy-based models, score-based diffusion models, stochastic averaging, extremum seeking, stochastic differential equations, latent-space sampling, black-box models, time-varying targets, online tracking, non-smooth energies

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_3wa6OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces a gradient-free sampling method for energy-based and certain score-based generative models using stochastic bounded extremum seeking (ES). The main idea is to replace the gradient-dependent drift in Langevin dynamics or a reverse-time SDE with high-frequency, coordinate-dependent cosine dithers whose phase depends only on a scalar energy or log-density evaluation. In the high-frequency averaging limit, the resulting stochastic dynamics recover the desired gradient drift. The paper proves convergence of the dithered Itô process to the target Langevin or reverse-time diffusion on finite time intervals, including quantitative rates under stronger regularity assumptions and extensions to several classes of nonsmooth energies. It also studies the finite-frequency and dimension dependence of the method. Empirically, the method is evaluated on synthetic sampling tasks, latent-space EBM sampling, and time-varying latent tracking on CIFAR-10 and CelebA-HQ. The paper explores an interesting connection between extremum-seeking control and generative-model sampling. However, I have concerns about the degree of methodological novelty, the narrow practical access model, and the limited evidence that the proposed approach provides a meaningful advantage over mainstream diffusion sampling methods.

Critical strengths
  1. Using high-frequency ES dynamics to recover a Langevin or reverse-SDE drift from scalar function evaluations is conceptually interesting. The paper opens an alternative sampling route for settings in which automatic differentiation and direct score access are unavailable.

  2. The extension of bounded-ES averaging to Itô processes appears to be a meaningful theoretical contribution. The paper establishes finite-horizon convergence, provides a mean-square rate under stronger assumptions, and considers increasingly nonsmooth energy functions. The analysis of frequency crowding and dimension-dependent evaluation complexity is also useful and appropriately exposes an important limitation of the method.

  3. In the dynamic latent-tracking experiments, bounded ES performs better than several derivative-free baselines, including SPSA, coordinate finite differences, and Gaussian-smoothing estimators under a matched scalar-evaluation budget.

Critical weaknesses
  1. The algorithmic novelty is limited by its reliance on established ES machinery.

The core gradient-recovery mechanism—injecting high-frequency coordinate-wise dithers and obtaining an effective gradient through averaging—is inherited from bounded extremum seeking. The main novelty is applying this idea to Langevin and diffusion sampling and extending its averaging analysis to stochastic Itô dynamics. This theoretical extension may be valuable, but the generative-model methodology itself appears closer to a transfer of an established control technique than a fundamentally new sampling principle. The paper should more clearly separate what follows directly from prior bounded-ES results from what is technically new in the stochastic sampling setting.

  1. The practical access model is narrow.

The method requires repeated access to a scalar energy or log-density J(x,t)J(x,t), while assuming that neither its gradient nor a score vector is available. This situation may occur for physical systems, nondifferentiable simulators, compiled models, or restricted APIs, but it is not the typical setting for modern diffusion models. Standard DDPM/EDM-type models directly output a score, noise, or velocity vector through a forward pass, while locally available energy models usually permit automatic differentiation. Moreover, ordinary score-only diffusion models do not directly provide the scalar JJ required by ES; the proposed line-integral surrogate requires additional evaluations and can be biased for a nonconservative score. The paper therefore appears substantially less general for score-based models than its title and abstract may initially suggest.

  1. The experiments do not demonstrate an advantage over mainstream diffusion sampling.

The paper acknowledges that gradient-free pixel-space sampling is not competitive with well-tuned gradient-based samplers. In latent tracking, bounded ES improves over other zeroth-order methods but remains worse than the AutoDiff oracle. The CIFAR-10 and CelebA-HQ experiments mainly show that the method can operate in carefully constructed black-box tracking tasks; they do not demonstrate an advantage in standard unconditional generation quality, sampling efficiency, or wall-clock cost. Furthermore, the tracking experiments use near-deterministic, mode-seeking dynamics and are therefore closer to online black-box optimization than to distributional sampling. A real nondifferentiable simulator or physical measurement system would provide stronger evidence of practical value.

  1. There are ambiguities in the experimental specification and evaluation.

In the CIFAR-10 tracking experiment, Table 5 reports Ns=200N_s=200, whereas the accompanying text states that the tracker uses Ns=100N_s=100 cost evaluations per frame. Since this directly changes the evaluation budget, the inconsistency must be resolved. The table also appears to be missing its bottom horizontal rule. More broadly, the paper should report the exact training objective for the diffusion energy model, clarify the parameterization and scaling of σt\sigma_t, and provide the values or selection procedures for tpt_p, λ\lambda, μp\mu_p, sps_p, and CscaleC_{\mathrm{scale}}.

  1. Although the method uses only one scalar evaluation per step, this should not be interpreted as dimension-independent efficiency. Distinct coordinate frequencies become increasingly crowded as dd grows, and resolving the resulting beat frequencies requires a higher base frequency and a smaller integration step. The paper proves an at-least-linear evaluation cost but does not establish practical scalability beyond the d=256d=256 latent-space experiments.
Questions for authors

What is the practically important use case that cannot be addressed by either direct score evaluation or automatic differentiation? Can the authors demonstrate the method on a genuinely nondifferentiable simulator, physical measurement system, compiled black-box pipeline, or realistic restricted API, including latency and wall-clock costs?

  1. How much of the contribution is new beyond existing bounded ES? Could the authors explicitly identify which components of the algorithm and averaging argument follow from prior deterministic bounded-ES theory and which mathematical steps are fundamentally required by the stochastic Itô setting?

  2. How should the method be fairly compared with mainstream diffusion samplers? In particular, can the authors report sampling quality and total computational or wall-clock cost against gradient-based latent samplers using the same underlying energy model? For score-only diffusion models, how many score evaluations are required to construct the line-integral surrogate, and how large is the resulting nonconservativeness bias?

  3. Which evaluation budget was used for Table 5? Table 5 reports Ns=200N_s=200, while the text reports Ns=100N_s=100 evaluations per frame. Please clarify the setting used to obtain the reported results and provide the missing implementation and energy-training details needed for reproduction.

Reviewer_1ChSOpenReview ↗
Rating
4
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

Score matching while a powerful technique used for sampling relies on estimation of \grad{x}log(p(x)) which requires a proxy parameterization through a NN as in Diffusion models, flow matching etc or through explicit assumptions on the parameter family. This makes it difficult to utilize in settings where access to gradients may be impractical. The authors propose a weighted cosine expectation based zeroth order scheme that converges in the limit to the true reverse SDE. They then showcase empirical evidence centered on latent space dynamics and theoretical justifications on the applicability of their scheme.

Critical strengths
  1. The formulation is novel and interesting and is a useful result for sensitive settings where true gradients may not be obtainable. In general zeroth order sampling schemes have been little explored in this context and this work is a promising step in this direction.

  2. The broader generality to EBMs, CNFs etc make it potentially useful for tertiary applications where score computation can be quite hard or needs to be bootstrapped for instance intrinsic dimension estimation where score matching has been shown to provide more accurate results (see e.g. https://proceedings.mlr.press/v235/stanczuk24a.html. ).

Critical weaknesses
  1. As the authors themselves acknowledge in practice for score matching the problem of estimating the score itself is bypassed by using learned proxies but for such cases log p(x) itself is unknown and requires a separate integration scheme conditioned on data. For models where log p(x) can be computed explicitly however such as EBMs or NFs the studies are unable to showcase the advantage strongly perhaps because the setup does not use settings such as resource constraints / obfusation etc which may justify the usage of zeroth order scheme in the first place.

  2. Tying into the earlier point the greater utility of this work is more on auxiliary downstream tasks where generation is but a part of the problem and a tradeoff in generation capability is acceptable as against a broader objective. It would have been better to have one such concrete problem highlighted and shown.

Questions for authors
  1. I am curious to know if the authors have extended their method to broader lanegvin sampling schemes eg SGLD where it may be of greater use ?

  2. The strategy used potentially seems related to feynman kac which has several works associated with it in this broader context as well I would like to know how do the authors feel their scheme fits in this broader picture ?

Reviewer_Tn3kOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

This paper introduces the Extremum Seeking (ES) mechanism for sampling with zero-order information on the target potential. This idea comes from optimization. They prove that the proposed Langevin dynamics follow the same dynamic of the classical order-one SDE when the dither frequency goes to infinity. The method is tested on latent sampling.

Critical strengths
  • Using the ES idea in the sampling area for zero-order sampling is innovative and a promising direction.
  • The sampling theoretical guarantees are clearly stated and the extensions for different JJ regularities are clearly stated.
Critical weaknesses

Major weaknesses:

  • No intuitions on the ES mechanism are given, making the paper are to follow at the first read.
  • The experimental goals are unclear and I hardly follow the experimental conclusions.
  • The proposed Langevin dynamics is not compared to other Langevin dynamics such as PnP-ULA [1] or zero-order samplers [2, 3].

Minor weaknesses/typos:

  • line 81 : The notation ωi\omega_i and wiw_i for the dither frequency and the Brownian motion are very closed, could you modify this for more clarity ?
  • line 85 : According to your notation just after, it should be pθp_{\theta} not pp.
  • line 86 : It should be J=−log⁡pthetaJ = -\log p_{theta}.
  • line 96 : The term "ES" has not been introduced.
  • line 174 : I suggest to add in the beginning of Assumption 3 "It exists C≥0C \ge 0 such that".
  • line 212 : There is an extra "&".
  • line 266 : I do not understand this sentence. Could you explain to me the frequency gap that you are refer to ?
  • line 316 : The notation x/yx/y is unclear, please use an index. Moreover, if you sum on cc, the result PxI(c,k)P_x I(c,k) can not depend of cc, could you clarify these definitions. You should also recall the definitions of I^\hat I and ItgtI^{\text{tgt}}.
  • line 710 : You define N=dN=d, I suggest to keep the same notation through the paper.
  • line 715 : I suggest to add "for n∈{1,…,N}n \in \{1, \dots, N\}.
  • line 730-766 : This is very interesting for giving an intuition on the ES mechanism, I suggest to add a short version of this paragraph in the main paper.
  • line 787 : tjτt_j^{\tau} is not defined.

[1] Bayesian imaging using plug & play priors: when langevin meets tweedie, Laumont, R{'e}mi and Bortoli, Valentin De and Almansa, Andr{'e}s and Delon, Julie and Durmus, Alain and Pereyra, Marcelo, SIAM Journal on Imaging Sciences, 2022 [2] A. Roy, L. Shen, K. Balasubramanian, and S. Ghadimi, Stochastic zeroth-order discretizations of Langevin diusions for Bayesian inference, Bernoulli, 2022 [3] He Ye, Kevin Rojas, and Molei Tao, Zeroth-order sampling methods for non-log-concave distributions: Alleviating metastability by denoising diffusion, NeurIPS, 2024

Questions for authors
  • paragraph 3.5 : The goal of this experiment is unclear and the choice of definition of JJ is not motivate, could you detail the goal and give me intuitions on this experiment ?
  • line 399-400 : How the sampling performance of ES in the latent space in term of FID compare with other methods ?
  • What is the computational cost of the proposed method ? Especially compared with baseline.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.