acceptodds
Under review as a conference paper at ICLR 2027

Robustness of Diffusion Models under Distribution Shift

Abstract

Score-based diffusion models are increasingly considered in settings where the underlying data distribution may differ from the training distribution, yet existing theoretical guarantees largely focus on the no-shift setting. In this work, we study robust score estimation under Wasserstein perturbations of a reference distribution. For the Ornstein–Uhlenbeck diffusion, we show that robust estimation decomposes into two fundamental components: the statistical cost of learning the reference distribution and the intrinsic cost of distribution shift. The latter scales quadratically with the Wasserstein radius, and this dependence is minimax optimal. We construct an explicit finite-sample estimator achieving the resulting robust minimax rate without knowing the shift radius. When the reference distribution lies on an unknown low-dimensional subspace, the statistical term adapts to the intrinsic dimension while the shift cost remains unchanged. Finally, we show that the same decomposition governs positive-time reverse sampling and obtain matching minimax guarantees in KL divergence. Together, these results characterize how finite data, intrinsic dimension, and distribution shift affect the robustness of score-based diffusion models.

Primary area:
learning theory
Keywords:
Diffusion models, distribution shift, score estimation, minimax optimality, Wasserstein robustness

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_kgYbOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper studies score estimation for the Ornstein–Uhlenbeck (OU) diffusion when the score is learned from nn samples of a reference distribution p0p_0 but evaluated on a shifted distribution q0q_0 in the W2W_2-ball Qε(p0)\mathcal Q_\varepsilon(p_0). The risk is the time-averaged shifted score loss Rε,p0(s^)=sup⁡q0E1T−t0∫t0T∥s^−∇log⁡qt∥L2(qt)2dt\mathcal R_{\varepsilon,p_0}(\hat s) = \sup_{q_0}\mathbb E\frac{1}{T-t_0}\int_{t_0}^T \|\hat s-\nabla\log q_t\|^2_{L^2(q_t)}dt. The main results are: (1) Proposition 3.1, a comparison principle for "OU-compatible" estimators s^=∇log⁡Ptμ^0\hat s = \nabla\log P_t\hat\mu_0 giving R≤2ε2(T−t0)τ0+2T−t0ED2(pt0∥p^t0)\mathcal R \le \frac{2\varepsilon^2}{(T-t_0)\tau_0} + \frac{2}{T-t_0}\mathbb E D_2(p_{t_0}\|\hat p_{t_0}); (2) Theorem 3.2, a matching ε2/(t0(T−t0))\varepsilon^2/(t_0(T-t_0)) lower bound; (3) a "Gaussian-blanket" estimator (empirical measure mixed with a Gaussian, propagated through the OU semigroup) with rate 1/(n(T−t0)τ0d/2)+ε2/((T−t0)τ0)1/(n(T-t_0)\tau_0^{d/2}) + \varepsilon^2/((T-t_0)\tau_0), shown minimax optimal over all compactly supported p0p_0 (Theorems 4.2–4.4); (4) an extension to p0p_0 supported on an unknown kk-dimensional linear subspace with dd replaced by kk (Theorem 5.1); and (5) the same decomposition for the KL error of positive-time reverse sampling (Theorem 6.1). A three-dimensional numerical illustration with a four-atom p0p_0 and translation shifts is given in Appendix H.

Critical strengths
  1. Clean problem formulation. Casting robustness as a worst-case risk over a W2W_2 ball, with the estimator trained on p0p_0 and evaluated against ∇log⁡qt\nabla\log q_t, is a natural and well-defined question that had not been formalized in the diffusion-theory literature.
  2. Elegant comparison principle. Proposition 3.1 combines the Gaussian-channel transport inequality (Lemma B.1), a three-density KL/Rényi inequality (Lemma B.3) and the relative de Bruijn identity (Lemma B.2) to reduce the entire time-integrated score risk to a single divergence at time t0t_0. The proof is short, correct, and the ingredients are well cited.
  3. Matching lower bounds. The oscillatory two-point construction for the shift term (Appendix D) and the Fano construction for the statistical term (Appendix C) are carefully executed, with explicit constants and an explicit handling of persistence over the time window.
  4. Self-contained and readable. Despite 58 pages, the main text is clear, the appendix is well organized, and the authors are upfront about limitations (OU only, linear subspaces only, realizability vs. trainability).
Critical weaknesses

Theory

  1. The "robust estimation" problem collapses to "nominal estimation + a stability modulus," so the main theorem is less informative than it sounds. The upper bound in Proposition 3.1 is additive, the Gaussian-blanket estimator does not depend on ε\varepsilon, and the lower bound in Theorem 3.2 is just the two-point statement that no estimator can distinguish q0±q_0^\pm in the ball. Consequently "minimax optimality over Qε\mathcal Q_\varepsilon" is automatic once one has (a) a nominal minimax estimator for p0p_0 and (b) the worst-case sensitivity of q0↦∇log⁡Ptq0q_0\mapsto\nabla\log P_t q_0 in W2W_2. The paper does not identify any regime in which robustness changes what an estimator should do, nor any trade-off between the two terms. The authors should state plainly that the shift cost is a property of the score map, not of learning, and position the contribution accordingly.

  2. The statistical term is minimax only over an enormous class, and the "optimal" estimator is the smoothed empirical measure. The class PD\mathcal P_D is all compactly supported distributions with no smoothness. Over this class the minimax rate 1/(nτ0d/2)1/(n\tau_0^{d/2}) is attained by propagating the empirical measure through the OU kernel, i.e. by memorizing the training set. This is the opposite of what one wants from a theory of generative models: the resulting "generator" reproduces the training points plus Gaussian noise. The rate is also exponentially bad in dd through both τ0−d/2\tau_0^{-d/2} and the constant Cd,D=2ed/2(1+D2/d)d/2C_{d,D} = 2e^{d/2}(1+D^2/d)^{d/2}. Prior minimax work on score estimation (Zhang et al. 2024; Dou et al. 2024; Stéphanovitch et al. 2025) obtains rates that improve with Hölder/Sobolev smoothness of p0p_0; the present theory cannot exploit smoothness at all. A result over a smoothness class would be substantially more meaningful.

  3. The shift cost ε2/t0\varepsilon^2/t_0 is a worst case driven by singular or oscillatory perturbations and is pessimistic for smooth references. The 1/τ01/\tau_0 factor comes from the Gaussian-channel inequality KL(qt∥pt)≤at2W22/(2σt2)\mathrm{KL}(q_t\|p_t)\le a_t^2W_2^2/(2\sigma_t^2), which is tight only for perturbations at the noise scale (the lower-bound instance uses 1±αcos⁡(πkx)1\pm\alpha\cos(\pi k x) on [−1,1][-1,1] with frequency k≍t0−1/2k\asymp t_0^{-1/2}). For a smooth p0p_0 and a smooth shift (e.g. a translation of a Gaussian), the actual score discrepancy is O(ε2)O(\varepsilon^2) with no 1/t01/t_0 blow-up. The paper never distinguishes these regimes, so the headline message "the intrinsic cost is ε2/t0\varepsilon^2/t_0" is a statement about the worst element of Qε\mathcal Q_\varepsilon rather than about typical distribution shifts. The optimality claim in Theorem 4.4 is also restricted to ε≤t0\varepsilon\le\sqrt{t_0}, i.e. to shifts smaller than the noise level at the earliest time; the regime ε≫t0\varepsilon\gg\sqrt{t_0}, arguably the interesting one for "distribution shift," is not covered.

  4. The comparison principle only applies to OU-compatible estimators, which excludes score networks trained by denoising score matching. Proposition 3.1 requires s^(⋅,t)=∇log⁡Ptμ^0\hat s(\cdot,t) = \nabla\log P_t\hat\mu_0 for a single initial law μ^0\hat\mu_0. A neural score trained by the standard loss does not satisfy this. Appendix G shows a ReLU network can approximate the Gaussian-blanket score uniformly, and Corollary G.4 transfers the bound for L∞L^\infty-close perturbations, but the authors themselves note (Appendix G.5) that this "does not imply that a particular score-matching training procedure recovers the constructed parameters" and that constants "may deteriorate as t0→0t_0\to0." The connection to diffusion models as actually trained is therefore thin.

  5. The reverse-sampling results in Section 6 are essentially restatements of the score results. With the "self-consistent" initialization νT=p^n,Tϑ\nu_T = \hat p^\vartheta_{n,T}, the reverse process driven by ∇log⁡p^n,t\nabla\log\hat p_{n,t} has law p^n,t0\hat p_{n,t_0} at time t0t_0 by construction, so (6.1) is literally Proposition 3.1 plus Proposition 4.1 with no sampling analysis. The Gaussian-initialization variant (Corollary F.4) adds only e−Te^{-T} terms. Moreover, the KL is evaluated at time t0>0t_0>0 between OU-smoothed laws, not between the generated distribution and q0q_0. Discretization is not treated. Calling this "robust generation" overstates what is shown.

  6. The 1/(T−t0)1/(T-t_0) normalization makes the risk and the rates partly an artifact of averaging. Both the upper and lower bounds scale as 1/(T−t0)1/(T-t_0), so letting T→∞T\to\infty sends every risk to zero. The standard denoising score-matching objective weights time by the diffusion coefficient and does not have this property. The authors should either use the unnormalized integral or explain why the averaged risk is the right object.

  7. Low-dimensional structure is linear only, with kk known. Assumption 2 requires an exact linear subspace and known kk; the data-splitting estimator uses the empirical span of n1n_1 samples. This is much weaker than the manifold-type results the paper cites (Chen et al. 2023a; Azangulov et al. 2024; Tang & Yang 2024), and the extension is straightforward given Proposition 3.1 (the only new ingredient is the missed-energy bound kD2/(n1+k)kD^2/(n_1+k)). Adaptivity to unknown kk is not addressed.

Numerical study (Appendix H)

  1. The experiment is too small to illustrate the claims. d=3d=3, p0p_0 is a four-atom discrete measure, so the Gaussian-blanket estimator is exactly a five-component Gaussian mixture whose only randomness is a multinomial count. Nothing is learned in any meaningful sense; the "finite-sample effect" is the fluctuation of four empirical frequencies. The shift family is twelve fixed translations, which are far from the worst case in Qε\mathcal Q_\varepsilon (see Weakness 3); the plotted maximum over twelve directions is not an approximation of the supremum in (2.2). The τ0−d/2\tau_0^{-d/2} dependence, which is the distinctive part of the rate, is not probed at all (the authors say so).

  2. "Consistent with quadratic dependence" is asserted, not measured. Figure 1 overlays empirical curves on the theoretical upper bounds but reports no fitted slopes in ε\varepsilon, no tightness ratio, and no comparison with a non-OU-compatible baseline (e.g. a small score network trained by denoising score matching, or the plug-in empirical score without the blanket). Since the shift lower bound is only shown for an oscillatory family, a direct simulation of that family would have been more convincing than translations.

  3. No neural-network or real-data experiment. Given that the motivation is pretrained diffusion models deployed under shift, even a modest experiment with a trained score network on a synthetic shift would test whether the predicted additive decomposition and ε2\varepsilon^2 scaling appear in practice.

Presentation

  1. Section 4.1 should state explicitly that the estimator interpolates the training data, and Section 4.3 should state that the minimax class contains no smoothness constraint and that the rate is therefore not comparable to smoothness-adaptive rates. The phrase "intrinsic cost of distribution shift" is used throughout for what is a worst-case stability constant.

Scores

  • Soundness: 3 (good). The proofs are careful and, as far as I checked (Appendices B, D.1, F.1–F.2, H), correct. The claims are stated precisely; the issue is scope, not rigor.
  • Presentation: 3 (good). Clear writing; some overclaiming in the framing ("robust generation," "intrinsic cost").
  • Contribution: 2 (fair). The comparison principle is nice, but the robustness problem decouples trivially into nominal estimation plus a stability modulus; the statistical term is minimax only over a smoothness-free class and is attained by memorization; the shift cost is a worst-case over singular perturbations; and the generation and intrinsic-dimension extensions follow directly from Proposition 3.1.
  • Rating: 2 — marginally below the acceptance threshold (weak reject). I would reconsider if the authors (a) establish a shift cost and matching lower bound over a smoothness class where the two terms genuinely interact, or otherwise demonstrate a regime where robustness changes the estimator, (b) analyze the regime ε>t0\varepsilon>\sqrt{t_0}, and (c) strengthen the numerical study along the lines of Question 7.
  • Confidence: 4. I am familiar with the minimax score-estimation literature and with transport–entropy inequalities for Gaussian channels; I read the main text and the proofs of Proposition 3.1, Lemma B.1–B.3, Proposition F.2 and the numerical appendix in detail, and skimmed the lower-bound constructions.
Questions for authors
  1. Is there any setting in which the minimax-optimal estimator under shift differs from the nominal minimax estimator for p0p_0? If not, what does the robustness framework add beyond the stability bound of Lemma B.1?
  2. What is the shift cost for smooth references? For instance, if p0p_0 has a β\beta-Hölder density bounded below, is sup⁡q0∈Qε∫t0T∥∇log⁡qt−∇log⁡pt∥L2(qt)2dt\sup_{q_0\in\mathcal Q_\varepsilon}\int_{t_0}^T\|\nabla\log q_t-\nabla\log p_t\|^2_{L^2(q_t)}dt still of order ε2/t0\varepsilon^2/t_0, or does it drop to O(ε2)O(\varepsilon^2)? Can Theorem 3.2 be adapted to such a class?
  3. What happens for ε>t0\varepsilon>\sqrt{t_0}? Is the upper bound still tight, and if not, what is the correct behavior?
  4. Can the statistical term be improved for smooth p0p_0 within your framework, e.g. by replacing the empirical measure in μ^0\hat\mu_0 with a kernel or wavelet density estimator while keeping OU compatibility? This would make the "robust" rate adaptive to smoothness.
  5. The Gaussian blanket uses ϑ=D/d\vartheta = D/\sqrt d and the calibration Mϑ,t0M_{\vartheta,t_0}, both of which require knowing DD. How sensitive is the bound to misspecification of DD, and is there an adaptive choice?
  6. In Theorem 5.1 the sample is split in half. Does the rate degrade if the empirical span is built from all nn samples without splitting, or is the splitting only a proof device?
  7. For the numerical study, could you (a) report the fitted exponent in ε\varepsilon on log–log axes, (b) add the oscillatory family from Appendix D as a shift family, and (c) include a non-OU-compatible estimator for comparison?
5 comments
AuthorsOpenReview ↗

Reply to Reviewer kgYb part1

We thank the reviewer for acknowledging the novelty of our problem formulation, the soundness of our comparison principle, and the careful establishment of our matching lower bounds. Yet, in Critical Weakness 1, the reviewer claims that robust minimax optimality is "automatic" from nominal estimation and a stability modulus, and further describes the robustness problem as one that "decouples trivially." We strongly disagree with this assessment. Our paper establishes a sharp minimax characterization of a previously unstudied score estimation problem under Wasserstein distribution shift. An additive decomposition does not make the result trivial, nor does a meaningful minimax characterization require fundamentally new proof techniques or a different optimal estimator.

We are also concerned about the length and scope of the review. It contains 11 extensive weaknesses and 7 additional questions, several of which call for substantial new theoretical results. In particular, the reviewer explicitly makes reconsideration of the rating conditional on deriving minimax results over smoothness classes or identifying a regime where robustness changes the optimal estimator, analyzing the large-shift regime, and conducting additional experiments. These demands go well beyond what can reasonably be expected during a conference rebuttal.

We therefore focus our response on the main theoretical objections, rather than addressing every proposed extension individually.

1. Novelty and significance of the robust minimax characterization (W1, Q1)

The reviewer's central assertion that our robust minimax result is "automatic" from nominal score estimation and a stability modulus is not justified.

Nominal score-estimation guarantees control errors under the reference distribution ptp_t, whereas our robust risk involves simultaneous changes in both the target score and the evaluation distribution. A nominal L2(pt)L^2(p_t) guarantee does not directly imply a uniform L2(qt)L^2(q_t) bound over Wasserstein perturbations. Likewise, a Gaussian-channel stability inequality alone does not establish the required finite-sample robust upper bound.

Our analysis addresses this issue through Proposition 3.1 and the Gaussian-blanket construction. In particular, Proposition 4.1 establishes the quantitative Rényi-divergence estimate

\leq \log\left(1+\frac{M_{\vartheta,t_0}-1}{n+1}\right).$$ Combining this estimate with the comparison principle, we obtain the finite-sample robust upper bound.Matching lower bounds then establish the optimal statistical and distribution-shift dependence in Theorem 4.4. The reviewer's argument also overlooks the significance of the resulting characterization. The additive decomposition and the fact that the estimator requires no knowledge of the shift radius are **conclusions of our analysis**. They identify an irreducible distribution-shift error, characterize its dependence on the smoothing time, and determine when statistical estimation or distributional uncertainty dominates the robust risk. We also disagree with the premise that robustness must change the optimal estimator or produce a nontrivial interaction between error terms to constitute a meaningful theoretical contribution. Neither is a mathematical requirement for a minimax characterization. The absence of such an interaction is itself a conclusion about the statistical problem under consideration.
AuthorsOpenReview ↗

Reply to Reviewer kgYb part2

2. Statistical complexity and worst-case distribution shift (W2–3, Q2–4)

Our framework deliberately considers compactly supported distributions without additional smoothness assumptions. This is not an overlooked limitation but a choice of statistical model. It allows us to characterize robustness for a broad distribution class, including discrete and singular measures.

The reviewer describes our estimator as "memorizing" the training samples. However, the Gaussian-blanket estimator is an explicitly regularized, OU-smoothed empirical construction, and its performance is established through quantitative risk bounds. The relevant mathematical question is whether the estimator achieves the minimax rate over the specified distribution class, which our results establish. Rates under additional Hölder or Sobolev assumptions concern different statistical problems and do not invalidate this characterization.

Similarly, the reviewer conflates worst-case robustness with behavior under particular structured perturbations. The observation that a Gaussian translation may yield a smaller score discrepancy does not contradict a sharp lower bound over the full Wasserstein neighborhood. In particular, smoothness of the reference distribution p0p_0 alone does not impose smoothness restrictions on the shifted distribution q0q_0.

We emphasize that the matching distribution-shift lower bound is established in the small-shift regime specified in Theorem 3.2, including ε≤t0\varepsilon\leq\sqrt{t_0}. We do not claim matching minimax optimality for larger shifts or over every restricted smoothness class. These are distinct theoretical questions requiring additional analysis, not counterexamples to the results established in the paper.

The reviewer is entitled to prefer a different distribution class or a broader parameter regime. However, such preferences should be distinguished from mathematical deficiencies in the present minimax characterization.

3. OU compatibility, neural estimators, and reverse sampling (W4–5)

The reviewer's criticism of OU compatibility overlooks a fundamental distinction between the construction used for the upper bound and the estimator class considered in the minimax problem.

Our minimax lower bounds apply to arbitrary measurable, possibly randomized estimators. OU compatibility is a sufficient structural condition used to construct an estimator achieving the upper bound, and it is not imposed on all competing estimators. Furthermore, Corollary G.4 and our constructive neural approximation results extend the robustness guarantee to score fields that need not be exact scores of OU-smoothed densities, provided the approximation error is controlled.

We do not claim that standard denoising score-matching training automatically achieves the required accuracy. Establishing guarantees for a particular neural training algorithm is a separate problem and is not necessary for the validity of our minimax results.

Regarding Section 6, we explicitly state that self-consistent initialization recovers the estimated OU trajectory. The contribution is the corresponding positive-time KL minimax characterization, including a matching lower bound over arbitrary output estimators. We also analyze standard Gaussian initialization separately in Appendix F.2. We do not claim a new discretization algorithm or convergence guarantees for sampling a potentially singular distribution at time zero.

AuthorsOpenReview ↗

Reply to Reviewer kgYb part3

4. Time normalization, intrinsic dimension, and estimator calibration (W6–7, Q5–6)

The factor 1/(T−t0)1/(T-t_0) defines a time-averaged score-estimation risk. Removing this normalization simply multiplies the corresponding upper and lower bounds by T−t0T-t_0, without changing their dependence on the sample size, shift radius. Our results concern finite diffusion horizons. Moreover, the OU process considered here has a constant diffusion coefficient, so weighting the integral by its squared diffusion coefficient does not introduce a different time dependence.

Theorem 5.1 considers an unknown linear subspace with known intrinsic dimension kk. The unknown-linear-subspace geometry is also the setting considered by Chen et al. (2023a), whose network architecture uses a prescribed intrinsic dimension. Importantly, the estimator does not require exact subspace recovery or non-degeneracy assumptions on the latent distribution, and the shifted distribution is allowed to leave the reference subspace. The statistical dependence on the ambient dimension dd is replaced by dependence on kk. We do not claim general manifold adaptation or adaptation to unknown kk.

Regarding calibration, any valid upper bound on the support radius DD can be used in the Gaussian-blanket construction, although a looser bound may worsen the constants. Underestimating DD may invalidate the required domination property.

Sample splitting in Theorem 5.1 provides the conditional independence used to analyze the projected estimator. Our theorem establishes the stated rate for this construction, and we do not claim an analogous result for an unsplit estimator without further analysis.

5. Numerical evaluation and presentation (W8–11, Q7)

This submission is under the learning theory track, and its central contributions are theoretical minimax upper and lower bounds. The numerical study is intended to illustrate selected predictions, particularly the dependence on the sample size and distribution-shift radius, rather than provide an exhaustive empirical evaluation of diffusion-model training.

The finite-support example is a legitimate statistical estimation problem: fluctuations in empirical frequencies give rise to finite-sample estimation error. The experiments examine this effect and the behavior under controlled distribution shifts. We do not claim that the finite set of translation directions approximates the worst case over the entire Wasserstein ball or that the experiments independently establish minimax sharpness.

The review requests fitted exponents, oscillatory perturbations, additional estimator comparisons, neural-network training, and real-data evaluations. Some of these could provide useful complementary illustrations. However, the minimax optimality established in this work follows from matching mathematical upper and lower bounds, not empirical curve fitting. An exhaustive experimental study is not necessary to establish the principal theoretical contribution.

We will clarify the illustrative purpose of the experiments and the worst-case interpretation of the theoretical results. The absence of the additional experiments requested by the reviewer does not undermine the stated minimax guarantees.

Reviewer_y7CrOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper studies how distribution shift affects the score estimation error and sampling error in diffusion models. It develops theory to bound the worst-case score estimation error given a fixed Wasserstein tolerance for distribution shift. It also characterizes this error considering finite training sample size and the case where the reference distribution is supported on a low-dimensional subspace, as well as lower bounds for such errors. Finally, it translates score estimation results to KL bounds on the sample distribution.

Critical strengths
  • Distribution shift widely exists in deployment of diffusion models. The paper identifies this problem and aims at developing new theory to address it.
Critical weaknesses
  • The significance of the results is not very clear. The bound on worst-case score estimation error as a function of the Wasserstein distance between reference and shifted distributions seems interesting, but the additional finite-sample bound appears orthogonal to the former effect. The paper claims to characterize the effects of both distribution shift and finite training samples on score estimation error, but does not seem to any reveal interesting interplay between them. If so, the latter effect has been widely studied in the literature, and the contribution is limited to the first effect.
  • Furthermore, the ultimate quantity of interest is the distributional error in the generated samples. If the distribution shift is already bounded in Wasserstein distance, this mismatch can be characterized, in a very straightforward way, by combining the existing error of the sampler under no distribution shift with the known distribution shift in Wasserstein distance. Therefore, it's unclear how this paper and its Theorem 6.1 provide additional information for this question.
  • Throughout the paper, the estimated score s^\hat{s} is assumed to be the score function of some estimated distribution. However, this is generally not ture: s^\hat{s} may not be a score function at all, such as when it's parameterized by a neural network. How would the results change when we loosen the assumption on s^\hat{s} to arbitrary measurable functions?
Questions for authors

Please see weaknesses. Additionally, "Gaussian blanket" does not seem to be a standard term. The authors may consider renaming it.

4 comments
AuthorsOpenReview ↗

Reply to Reviewer y7Cr part1

We thank the reviewer for the feedback. Please find our point-by-point responses below.

  1. W1:The significance of the results is not very clear. The bound on worst-case score estimation error as a function of the Wasserstein distance between reference and shifted distributions seems interesting, but the additional finite-sample bound appears orthogonal to the former effect. The paper claims to characterize the effects of both distribution shift and finite training samples on score estimation error, but does not seem to any reveal interesting interplay between them. If so, the latter effect has been widely studied in the literature, and the contribution is limited to the first effect.

We disagree with the assessment that our contribution is limited to the distribution-shift term simply because the nominal statistical rate has been studied previously. Our paper considers a new statistical problem, namely score estimation under Wasserstein distribution shift, where only source samples are available while the unknown target distribution determines both the score to be estimated and the measure under which estimation error is evaluated. To our knowledge, this setting has not been previously studied, and establishing its sharp minimax characterization is a central contribution of our work.

Specifically, Theorem 4.4 establishes the minimax rate

\asymp \frac{1}{n(T-t_0)\tau_0^{d/2}} + \frac{\varepsilon^2}{t_0(T-t_0)}.$$ The fact that the first term coincides with the known nominal statistical rate does not mean that its achievability and optimality under distribution shift were already established. Existing nominal $L^2(p_t)$ guarantees do not directly imply uniform control of the shifted $L^2(q_t)$ risk. Nor does a Wasserstein stability inequality alone establish that the statistical and shift-dependent rates can be achieved simultaneously without an additional penalty. Proposition 3.1 provides the required comparison principle, while Proposition 4.1 establishes the quantitative Rényi-divergence control needed for the robust finite-sample upper bound. Together with the matching lower bounds, these results show that a single estimator achieves the optimal statistical and distribution-shift rates without knowing the shift radius $\varepsilon$. We also disagree that the absence of a nontrivial interaction between the two terms diminishes the significance of the result. In our framework, the Wasserstein shift radius is specified independently of the number of source samples. However, this does not imply that their effects on the minimax risk must be additive. Since distribution shift changes both the target score and the evaluation distribution, it could in principle amplify finite-sample estimation errors and introduce an additional penalty depending jointly on $n$ and $\varepsilon$. **Our analysis establishes that no such additional penalty is necessary at the minimax-rate level in the stated regime.** The additive decomposition is therefore a theoretical conclusion. It also identifies when additional source samples improve performance and when the irreducible distribution-shift error becomes dominant. Furthermore, Section 5 shows that unknown low-dimensional source structure improves the statistical dependence from the ambient dimension $d$ to the intrinsic dimension $k$, while the worst-case shift cost remains unchanged (Theorem 5.1 and Corollary E.3). This further illustrates how statistical estimation and distribution-shift uncertainty contribute differently to the robust minimax risk.
AuthorsOpenReview ↗

Reply to Reviewer y7Cr part2

  1. W2:Furthermore, the ultimate quantity of interest is the distributional error in the generated samples. If the distribution shift is already bounded in Wasserstein distance, this mismatch can be characterized, in a very straightforward way, by combining the existing error of the sampler under no distribution shift with the known distribution shift in Wasserstein distance. Therefore, it's unclear how this paper and its Theorem 6.1 provide additional information for this question.

The proposed argument is valid when generation error is measured in Wasserstein distance, but it does not apply to the KL divergence studied in Section 6. Our objective is to control KL(qt0∣μ^n,t0)\mathrm{KL}(q_{t_0}\\|\widehat\mu_{n,t_0}), which cannot be bounded by simply combining a nominal sampling error with the Wasserstein distance between the source and target distributions.

In particular, although Gaussian smoothing provides a bound on KL(qt0∣pt0)\mathrm{KL}(q_{t_0}\\|p_{t_0}) in terms of W2(q0,p0)\mathsf W_2(q_0,p_0), this bound and a nominal guarantee on KL(pt0∣μ^n,t0)\mathrm{KL}(p_{t_0}\\|\widehat\mu_{n,t_0}) do not imply a bound on KL(qt0∣μ^n,t0)\mathrm{KL}(q_{t_0}\\|\widehat\mu_{n,t_0}), since KL divergence does not satisfy the triangle inequality. The proposed combination of existing results therefore does not establish our KL guarantee. Our analysis instead uses a change-of-reference inequality involving Rényi divergence, together with the quantitative finite-sample control established in Proposition 4.1.

Theorem 6.1 establishes the sharp minimax rate

\asymp \frac{1}{n\tau_0^{d/2}}+\frac{\varepsilon^2}{\tau_0}$$ in the stated regime. It shows that a single reverse sampler constructed solely from source samples achieves the optimal statistical and distribution-shift rates simultaneously, with matching lower bounds over all measurable estimators. Corollary F.4 further provides a bound under standard Gaussian initialization, including the additional initialization error, while Corollary F.6 establishes the corresponding intrinsic-dimensional minimax rate. Thus, Theorem 6.1 provides a sharp KL characterization of robust generation that cannot be obtained through the Wasserstein triangle inequality suggested by the reviewer. 3. > `W3: Throughout the paper, the estimated score $\hat s$ is assumed to be the score function of some estimated distribution. However, this is generally not true: $\hat s$ may not be a score function at all, such as when it's parameterized by a neural network. How would the results change when we loosen the assumption on $\hat s$ to arbitrary measurable functions?` We clarify that OU compatibility is a property of our explicitly constructed estimator used to prove the upper bound, not an assumption imposed on the class of admissible estimators. Both the robust risk in (2.2) and the minimax formulation in (4.6) allow arbitrary measurable, possibly randomized score estimators, regardless of whether they correspond to the score of any probability distribution. In particular, the lower bounds in Theorems 3.2 and 4.3 hold over this unrestricted class. **Therefore, the relaxation suggested by the reviewer is already incorporated into our minimax theory, and the optimal rate in Theorem 4.4 remains unchanged.** Furthermore, our analysis is not limited to exactly OU-compatible estimators. Section 4.2 and Appendix G explicitly address neural network approximations. Proposition G.1 constructs a ReLU network approximation to the Gaussian-blanket score, while Corollary G.4 establishes that a measurable estimator approximating an OU-compatible score retains the robust risk guarantee up to an additional approximation-error term, without requiring exact OU compatibility. Choosing this error sufficiently small preserves the minimax rate. Thus, **our minimax guarantees already cover arbitrary measurable estimators, and we also provide a constructive extension to neural score approximations.**

Dear Reviewer y7Cr,

With less than three days remaining in the discussion period, we would appreciate your feedback on whether our responses have addressed your concerns. Please let us know if any further clarification is needed.

Best regards, The Authors

Thank the authors for their responses. I am convinced that the characterization of score error and the analysis of KL divergence have merits. I will increase my score from 2 to 3. I'd encourage the authors to clarify these points in the paper.

Reviewer_wy7BOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The manuscript studies score estimation and sampling for diffusion models when the target measure lies in a Wasserstein neighborhood of the training data distribution. It introduces a Gaussian blanket estimator and derives statistical and distribution-shift bounds and extends the analyses when the source is supported on an unknown linear subspace. The problem and the robustness characterization is interesting. However, the methodolgical contribution is limited and practically trained diffusion models under for the proposed method needs further study.

Critical strengths
  • The proposed framework takes into account deviations in both the target score and the distribution used to evaluate the score error.
  • Matching lower bound show the cost of distribution shift.
Critical weaknesses

Novelty The upper bound combines known results such as Gaussian smoothing inequalities, entropy dissipation etc. The lower bound construction adapts Dou et al. (2024). Extending the separation arguments require some additional work but the basic proof strategies are well-known.

Statistical Improvement The statistical rate follows by integrating existing fixed-time score-estimation bounds, as noted after Theorem 4.4. The main contribution is the sharp characterization of distribution-shift error. The paper could better explain what broader insights it provides

The guarantees rely on a specially constructed estimator The analysis uses scores generated by evolving a single estimated distribution under the OU process. However, a trained score network may not satisfy this consistency property. Appendix G establishes a neural approximation of the estimator, but does not show that standard score-matching training attains the required accuracy.

Experiments The finite-support translation examples are useful. However, more detailed experments are necessary for understanding neural training, intrinsic dimensional adaptation, and dependence on the smoothing time. A comparison with unregularized mixture and a blanket regularized mixture using true score weights will be helpful.

Questions for authors
  1. What precisely does the Gaussian blanket enable? A concrete example or comparison would help isolate the main technical contribution, whether it ensures the finiteness of the Renyi divergence, or helps in attaining minimax rates should be clearly explained.

  2. What is the new ingredient in the statistical lower bound?

8 comments
AuthorsOpenReview ↗

Reply to Reviewer wy7B part1

We thank the reviewer for the feedback. Please find our point-by-point response below.

  1. W1: Novelty: The upper bound combines known results such as Gaussian smoothing inequalities, entropy dissipation etc. The lower bound construction adapts Dou et al. (2024). Extending the separation arguments require some additional work but the basic proof strategies are well-known. Q2: What is the new ingredient in the statistical lower bound?

We disagree with the assessment that the use of established proof techniques limits the novelty of our contribution. The main novelty of our work lies in formulating and establishing a sharp minimax characterization of score estimation under Wasserstein distribution shift, a problem that, to our knowledge, has not been previously studied. Unlike nominal score estimation, only samples from the reference distribution are available, while the unknown distribution shift affects both the target score and the measure under which estimation error is evaluated. We characterize the unavoidable statistical and distribution-shift errors, establish matching lower bounds, and construct a single estimator attaining the resulting minimax rate without knowing the shift radius.

The fact that individual proof ingredients are well known does not imply that the resulting theoretical guarantees follow directly from existing results. In particular, nominal L2(pt)L^2(p_t) guarantees do not directly yield uniform control of the shifted L2(qt)L^2(q_t) risk. Proposition 3.1 establishes a comparison principle that reduces the estimation component to Rényi-divergence control, while our Gaussian-blanket construction provides the quantitative finite-sample bound required to achieve the robust minimax rate. Together with the matching lower bounds, these results provide a characterization that is not available from existing nominal score-estimation theory.

Regarding Q2, although packing arguments and Fano's inequality are standard, the key additional ingredient is the construction of a single family of hard source distributions whose score separation persists over a common, nontrivial time interval. Existing fixed-time lower bounds, including those of Dou et al. (2024), cannot simply be integrated to establish a time-averaged minimax lower bound, since the hard instances may depend on the diffusion time. Our construction establishes persistent separation for the same family, which is essential for obtaining the integrated lower bound.

We emphasize that our contribution is not the introduction of entirely new proof techniques, but rather the formulation and sharp theoretical characterization of a previously unstudied robustness problem. We believe the novelty and significance of these results should be evaluated accordingly.

  1. W2: Statistical Improvement: The statistical rate follows by integrating existing fixed-time score-estimation bounds, as noted after Theorem 4.4. The main contribution is the sharp characterization of distribution-shift error. The paper could better explain what broader insights it provides.

As explicitly acknowledged after Theorem 4.4, the nominal statistical rate agrees with that obtained by integrating existing fixed-time bounds. However, this observation alone does not establish our robust minimax characterization. Our results show that a single estimator constructed solely from source samples simultaneously attains the optimal statistical and distribution-shift rates, without knowledge of the shift radius (Theorems 4.2–4.4). Moreover, the time-integrated statistical lower bound (Theorem 4.3) does not follow directly from existing fixed-time lower bounds, since the hardest distribution may vary with diffusion time.

Beyond these rates, our results reveal a fundamental distinction between statistical and distributional uncertainty. In the unknown-subspace setting, the statistical rate depends on the intrinsic dimension kk rather than the ambient dimension dd, even when the shifted distribution leaves the source subspace, whereas the unavoidable distribution-shift cost remains unchanged (Section 5). Theorem 6.1 further establishes that the same statistical–shift decomposition persists in positive-time reverse sampling under KL divergence. Thus, increasing sample size or exploiting intrinsic structure can reduce statistical error, but cannot eliminate the intrinsic cost of distribution shift.

AuthorsOpenReview ↗

Reply to Reviewer wy7B part 2

  1. W3: The guarantees rely on a specially constructed estimator: The analysis uses scores generated by evolving a single estimated distribution under the OU process. However, a trained score network may not satisfy this consistency property. Appendix G establishes a neural approximation of the estimator, but does not show that standard score-matching training attains the required accuracy.

OU compatibility is a property of our explicit estimator used to establish the upper bound, not a restriction on the class of estimators considered in our minimax analysis. In particular, our lower bounds (Theorems 3.2 and 4.3) apply to all measurable, possibly randomized score estimators, regardless of whether they satisfy OU compatibility. Therefore, our minimax optimality results hold over the unrestricted estimator class, rather than only among OU-compatible estimators.

Our objective is to characterize the fundamental statistical limits of robust score estimation and construct an estimator attaining these limits, rather than analyze a specific neural training procedure. Importantly, Appendix G further establishes that the Gaussian-blanket score admits a constructive ReLU-based approximation with explicit network complexity bounds (Proposition G.1). Section G.5 shows that the resulting neural estimator retains the robust statistical guarantees up to the approximation error, without requiring exact OU compatibility. Thus, our results establish both minimax optimality and neural realizability. Proving that standard score-matching training achieves these guarantees is a separate algorithmic question and is not necessary for our minimax characterization.

  1. W4: Experiments: The finite-support translation examples are useful. However, more detailed experments are necessary for understanding neural training, intrinsic dimensional adaptation, and dependence on the smoothing time. A comparison with unregularized mixture and a blanket regularized mixture using true score weights will be helpful.

We respectfully disagree that the suggested extensive experiments are necessary to substantiate the main contributions of this paper. Our work is primarily a learning-theoretic study aimed at establishing sharp minimax guarantees for score estimation and sampling under distribution shift, rather than developing an empirically competitive training algorithm. The minimax optimality and intrinsic-dimensional adaptation results are rigorously established through matching upper and lower bounds, independently of numerical experiments. Neural-training benchmarks and extensive comparisons with alternative estimators address separate algorithmic questions and are not required to validate these theoretical results. In particular, our minimax guarantees do not claim that the proposed estimator outperforms every alternative on each individual distribution.

Nevertheless, to further illustrate our theoretical predictions, we conducted an additional experiment examining the dependence on the smoothing cutoff t0t_0. Using the same experimental setup as in our original submission, we consider 18 cutoffs t0∈[0.002,0.2]t_0\in[0.002,0.2], recalibrating Mϑ,t0M_{\vartheta,t_0} for each cutoff while fixing ϑ=D/3\vartheta=D/\sqrt{3}. For n=5×107n=5\times10^7, representative time-averaged score risks are:

t0t_0ε=0\varepsilon=0ε=0.05\varepsilon=0.05
0.0021.2986×10−31.2986\times10^{-3}1.251509
0.0203.4392×10−53.4392\times10^{-5}0.125344
0.0503.3893×10−63.3893\times10^{-6}0.050332
0.2003.6599×10−83.6599\times10^{-8}0.011762

Log–log fits over the six smallest cutoffs, t0∈[0.002,0.008434]t_0\in[0.002,0.008434], yield slopes of −1.524 without shift and −1.000 with shift, consistent with the t0−3/2t_0^{-3/2} and t0−1t_0^{-1} dependencies in our theoretical upper bound. The zero-shift behavior also reflects the cutoff-dependent Gaussian-blanket calibration. All observed risks remain below the explicit theoretical upper bound.

These results provide additional numerical illustration of the predicted smoothing-time dependence. We have incorporated this in our revision.

AuthorsOpenReview ↗

Reply to Reviewer wy7B part3

  1. Q1: What precisely does the Gaussian blanket enable? A concrete example or comparison would help isolate the main technical contribution, whether it ensures the finiteness of the Renyi divergence, or helps in attaining minimax rates should be clearly explained.

The Gaussian blanket is introduced to obtain sharp quantitative control of the Rényi divergence, rather than merely ensuring its finiteness. Under compact support, the unregularized empirical OU mixture already has finite Rényi divergence, but it may assign very small density to poorly sampled regions, making sharp finite-sample control difficult.

The calibrated Gaussian component provides a deterministic lower envelope for the estimated density and, crucially, a uniform domination bound for the component kernels (Lemma B.4). This enables the dominated-kernel argument in Lemma B.5, yielding the explicit bound in Proposition 4.1:

\leq \log\left(1+\frac{M_{\vartheta,t_0}-1}{n+1}\right).$$ Combined with Proposition 3.1, this establishes the optimal statistical contribution to the robust minimax rate in Theorem 4.4. Thus, the Gaussian blanket is not simply a device for ensuring finiteness, it also provides the quantitative control needed to construct an estimator attaining the minimax rate. We do not claim that Gaussian-blanket regularization is necessary for every minimax-optimal estimator.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.