acceptodds
Withdrawn from ICLR 2027

DDPM-based generation of tilted samples

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Given independent samples from a -dimensional probability distribution, our aim is to generate diffusion-based samples from a distribution obtained by tilting the original, where the degree of tilt is parameterized by . We define a plug-in estimator and show that it is minimax-optimal. We develop an algorithm that, given original samples, generates exponentially tilted samples using a kerneled score estimator. The sample complexity of this algorithm is provably polynomial for bounded random variables, and avoids the curse of dimensionality. The proof uses tight Wasserstein bounds between the plug-in estimator distribution and the true distribution as a function of and , and a novel score estimation bound. Furthermore, we also demonstrate sample complexity bounds for neural network-learned scores. Our theoretical results are supported by extensive simulations. Applications of our work include finance, weather and climate modeling, and many other domains, where the aim may be to generate samples from a tilted distribution that satisfies practically motivated moment constraints.

Primary area:
generative models
Keywords:
Diffusion, DDPM, Tilting, Score Function, Neural Networks

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_kK5WOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies the problem of generating samples from an exponentially tilted distribution when only independent samples from the original distribution are available. The authors construct a self-normalized importance-weighted empirical distribution and use its Gaussian-smoothed score in a DDPM reverse sampling procedure.

Critical strengths

The paper addresses a practically relevant problem of sampling from exponentially tilted distributions using only samples from the original distribution, without requiring access to its density.

Critical weaknesses
  1. The proposed algorithm performs exponential tilting entirely through importance reweighting of the original samples, while DDPM is subsequently used merely to generate samples approximating the resulting weighted empirical distribution. Since this empirical distribution already admits exact sampling through importance resampling, the paper does not establish what additional statistical benefit the diffusion procedure provides, nor does it introduce a mechanism to address the fundamental weight degeneracy and sample coverage issues of exponential tilting. Consequently, the necessity and distinctive algorithmic contribution of the proposed DDPM framework remain insufficiently justified.
  2. The proposed framework largely combines established techniques in importance-weighted empirical approximation [1], DDPM convergence theory [2], low-dimensional score approximation [3], and Wasserstein analysis [4]. Its main novelty therefore relies on the claimed theoretical guarantees, yet several central results contain substantial mathematical gaps that undermine their validity (see questions). Moreover, the paper does not establish a clear statistical or computational advantage of its DDPM procedure over direct importance resampling, leaving its incremental contribution insufficiently justified.

[1] Iyer S R, Mandal H, Gupta D, et al. Fundamental limits for weighted empirical approximations of exponentially tilted distributions[C]// Proceedings of the 29th International Conference on Artificial Intelligence and Statistics. PMLR, 2026, 300: 5239–5247.

[2] Li G, Yan Y. O(d/T) convergence theory for diffusion probabilistic models under minimal assumptions[J]. Journal of Machine Learning Research, 2025, 26(292): 1–55.

[3] Mu X, Shang Z, Zhou Z, et al. Score approximation for diffusion models on arbitrary low-dimensional structures[J]. arXiv preprint arXiv:2606.19894, 2026.

[4] Weed J, Bach F. Sharp asymptotic and finite-sample rates of convergence of empirical measures in Wasserstein distance[J]. Bernoulli, 2019, 25(4A): 2620–2648.

Questions for authors
  1. Since the importance-weighted empirical distribution Pn,θP_{n,\theta} already admits exact sampling through direct resampling, what motivates introducing an additional DDPM procedure to approximate this distribution? What fundamental sampling difficulty does DDPM address that cannot be handled by direct importance resampling?
  2. Can the authors demonstrate that DDPM-generated samples approximate the true tilted distribution PθP_\theta more accurately than samples drawn directly from Pn,θP_{n,\theta}? In particular, does the diffusion procedure provide any improvement in Wasserstein distance or other meaningful distributional metrics under a matched computational budget?
  3. When the tilt strength increases, the importance weights may concentrate on very few observations, substantially reducing the effective sample size. Since the proposed DDPM score is constructed entirely from these weighted samples, how does the algorithm mitigate this degeneracy or recover regions of the tilted distribution that are poorly represented in the original dataset?
  4. Theorem 2 claims a distribution-independent constant Cdint,pC_{d_{\mathrm{int}},p}, but the proof appears to omit the covering constant CkC_k from Definition 2, which can be arbitrarily large even for fixed intrinsic dimension. How can the stated uniform bound hold for distributions supported on arbitrarily many well-separated low-dimensional components?
  5. The derivative formula in Lemma 6 appears to have an incorrect sign, as the simple case P=δ0P=\delta_0 and Q=δvQ=\delta_v yields the opposite derivative. Can the authors provide a corrected derivation and rigorously justify the conditional-expectation and covariance bounds used in Lemma 8 to establish Proposition 2?
  6. Theorem 5 claims a sample complexity bound without explicit dependence on θ\theta, although sufficiently strong tilting can concentrate the target distribution on regions with arbitrarily small probability under the original distribution. How can such a uniform learning guarantee hold when these regions are unlikely to appear in the training samples, particularly since the proof uses the unknown probabilities πi=P(X∈Ai)\pi_i=P(X\in A_i)?
Reviewer_RhHNOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

Given a probability distribution, the aim of this paper is to generate samples from an exponentially tilted version of this distribution. The authors use self-normalized importance sampling (SNIS) to construct a discrete approximation of the tilted distribution and show that this estimator is minimax in a particular sense. They then use diffusion-based sampling to sample from a smoothed version of this discrete approximation and prove polynomial complexity for this sampling scheme.

Critical strengths
  • Important research direction.
    • Drawing samples from tilted versions of distributions induced by pretrained models is an active and important research direction, and finds practical application in regimes like Bayesian inference, reward-tilting of diffusion models, and Bayesian optimization.
  • Asymptotic Minimaxity and Wasserstein bounds of a SNIS estimator.
    • In Theorem 1, the authors show that the well-known SNIS estimator is asymptotically minimax, which seems to be a novel result.
    • In Theorem 2, the authors extend known results by showing bounds for the expected Wasserstein distance between the true tilted law and the random SNIS measure.
Critical weaknesses
  • The main algorithm is irrelevant.
    • In the kernelized score setting, Algorithm 1 uses a DDPM reverse process to sample from a known, finite Gaussian mixture model pn,θ,1p_{n,\theta,1}. Exact draws from this GMM are easy to obtain via ancestral sampling. Thus, Algorithm 1 is not only unnecessary, but introduces error due to both the discretization of the reverse process, and the mismatch between the terminal distributions of YTY_T and XTX_T.
    • Even more so, the practical justification of drawing from a smoothed version of the law of Xn,θX_{n,\theta}, namely ∑i=1nγθ,iδXi\sum_{i=1}^n \gamma_{\theta,i} \delta_{X_i}, is unclear. Hence, drawing directly from this discrete measure as defined in Eq. 3 would yield exact samples and is trivial to do.
  • The kernelized score sampler will not produce "novel" tilted samples.
    • As Xn,θX_{n,\theta} only has support on a set of nn samples X1,…,XnX_1,\dots,X_n drawn from the original distribution, the proposed sampler can only ever reproduce (noisy versions of) the original model's samples XiX_i, which may be an issue in high dimensions and for moderate nn, as one cannot hope to explore the state space sufficiently well by just drawing from the unconditional original model.
    • Indeed, in contrast to approaches like guidance and DPS, the proposed sampler does not leverage the original diffusion model at all, as it merely needs samples from the original distribution.
  • Overstated contribution.
    • The contribution list includes "We create an estimator Xn,θX_{n,\theta} of XθX_\theta" (L066), which makes it sound like a novel estimator is proposed, when in fact the authors use a well-known SNIS estimator.
  • Lack of experimental validation.
    • There are no experiments that compare the author's approach to any of the existing methods, like guidance methods and DPS.
    • Moreover, there's no comparison to a naive baseline which directly samples the discrete SNIS measure directly (or the smoothed version via ancestral sampling).
  • Factual errors, undefined symbols, and typos
    • Notation keeps changing. Examples include Rn\mathbb{R}^n vs. RnR^n, Ca,b\mathcal{C}_{a,b} vs. Ca,bC_{a,b}, PP vs. P\mathbb{P}, X\mathcal X vs. XX (Section 3.1)
    • After introducing P(Xθ∈A)\mathbb{P}(X_\theta \in A) in Eq. 1, L041 reads "for all A⊂RdA \subset \mathbb{R}^d", which is not well-defined in general, as some AA are not measurable. Presumably the authors care about events in the Borel σ\sigma-algebra of Rd\mathbb{R}^d.
    • L184 reads "Observe that Mθ,ρθ<∞M_\theta, \rho_\theta < \infty since XX is bounded", but this is not true without further regularity assumptions on gg.
    • L196 mentions μn,θ,μθ\mu_{n,\theta}, \mu_\theta, without ever defining them.
    • Sum in Eq. 10 should run over tt, not ii
    • Theorem 3: pp is undefined. Do the authors mean pdatap_{\mathrm{data}}? Also, E∣p∣2\mathbb E \\|p\\|_2 should probably read Ep[∣X∣2]\mathbb E_p[\\|X\\|_2]. It also says "for the forward process given by (11)", although (11) specifies the reverse process. A constant is introduced as cMc_M and then (presumably) used as cc in the Theorem.
    • L293: φK\varphi_K is not the density of N(0,K2I)N(0, K^2I): σt\sigma_t should be replaced with KK.
    • sn,θ,ts_{n,\theta,t} is incorrect: It should read φ1−αˉt\varphi_{\sqrt{1-\bar{\alpha}_t}}, not φ(1−αˉt)\varphi_{(1-\bar{\alpha}_t)}, as (1−αˉt)I(1 - \bar{\alpha}_t)I is the covariance of p(Xt∣X0)p(X_t \mid X_0).
    • dBLd_{BL} is not defined in the main text
Questions for authors

None

Reviewer_4gbUOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies the problem of generating samples from an exponentially tilted distribution using independent samples from an unknown original distribution. The authors construct a self-normalized importance-sampling estimator and derive Wasserstein convergence bounds that depend on the intrinsic dimension and the variance of the importance weights. They then use a Gaussian-smoothed version of this estimator to construct a DDPM sampler, obtaining theoretical sample complexity guarantees. The paper additionally considers neural-network score approximation and evaluates the approach on several low-dimensional synthetic distributions.

Critical strengths

a) The problem of generating tilted samples from an unknown distribution is interesting. The paper attempts to provide theoretical guarantees for this setting without requiring explicit knowledge of the original density or differentiability of the tilting function.

b) I find the connection between self-normalized importance sampling and Wasserstein convergence particularly interesting. The dependence on the quantity ρθ\rho_\theta provides a natural way to characterize the difficulty introduced by tilting.

c) The paper makes an interesting attempt to connect statistical estimation error with diffusion sampling guarantees through score stability. The neural-network extension is also potentially interesting because it attempts to exploit intrinsic dimensionality rather than ambient dimensionality.

Critical weaknesses

a) I am concerned about Theorem 5, which claims sample complexity independent of the tilt parameter. Consider a distribution consisting of two well-separated components, one of which initially has probability (q≪1)(q\ll 1). We can choose a linear exponential tilt that makes the rare component have probability (1/2)1/2) under the target distribution. However, with substantially fewer than (1/q)(1/q) original samples, we are unlikely to observe this component at all. It is therefore unclear how a neural-network estimator learned from these samples could approximate the tilted distribution accurately, regardless of its architecture. This seems to contradict the theorem as stated. Perhaps the authors intend an approximation result assuming access to the population score rather than a statistical learning guarantee?

b) I also have a concern about Theorem 2. The intrinsic dimension in Definition 2 is defined through an asymptotic covering exponent, without controlling the corresponding covering constant. However, the theorem claims a convergence bound with a constant independent of the underlying distribution. For example, consider a uniform distribution over m3m^3 very short, well-separated line segments arranged on a three-dimensional grid. This support has intrinsic dimension one, but with n≍m3n\asymp m^3 samples, its empirical Wasserstein error can remain of order n−1/3n^{-1/3}, rather than the claimed n−1/2n^{-1/2} rate for p=1p=1. I think some quantitative covering-number assumption is needed. Could the authors clarify how their bound handles this example?

c) My other concern is the motivation for using diffusion rather than simply sampling from the weighted empirical distribution. Once the importance weights are computed, we can sample an index according to those weights and obtain an exact sample from the empirical estimator. We can also sample directly from its Gaussian-smoothed version. The proposed DDPM algorithm instead approximately samples from this smoothed distribution through a reverse process. I would like to understand what additional advantage diffusion provides, either statistically or computationally. The experiments do not appear to compare against these direct sampling baselines.

Questions for authors

Please see above. I would also appreciate clarification of the precise assumptions required to apply the cited Li and Yan DDPM convergence theorem, particularly regarding the diffusion noise schedule and finite-resolution covering numbers.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.