- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper studies score estimation for the Ornstein–Uhlenbeck (OU) diffusion when the score is learned from samples of a reference distribution but evaluated on a shifted distribution in the -ball . The risk is the time-averaged shifted score loss . The main results are: (1) Proposition 3.1, a comparison principle for "OU-compatible" estimators giving ; (2) Theorem 3.2, a matching lower bound; (3) a "Gaussian-blanket" estimator (empirical measure mixed with a Gaussian, propagated through the OU semigroup) with rate , shown minimax optimal over all compactly supported (Theorems 4.2–4.4); (4) an extension to supported on an unknown -dimensional linear subspace with replaced by (Theorem 5.1); and (5) the same decomposition for the KL error of positive-time reverse sampling (Theorem 6.1). A three-dimensional numerical illustration with a four-atom and translation shifts is given in Appendix H.
Critical strengths
- Clean problem formulation. Casting robustness as a worst-case risk over a ball, with the estimator trained on and evaluated against , is a natural and well-defined question that had not been formalized in the diffusion-theory literature.
- Elegant comparison principle. Proposition 3.1 combines the Gaussian-channel transport inequality (Lemma B.1), a three-density KL/Rényi inequality (Lemma B.3) and the relative de Bruijn identity (Lemma B.2) to reduce the entire time-integrated score risk to a single divergence at time . The proof is short, correct, and the ingredients are well cited.
- Matching lower bounds. The oscillatory two-point construction for the shift term (Appendix D) and the Fano construction for the statistical term (Appendix C) are carefully executed, with explicit constants and an explicit handling of persistence over the time window.
- Self-contained and readable. Despite 58 pages, the main text is clear, the appendix is well organized, and the authors are upfront about limitations (OU only, linear subspaces only, realizability vs. trainability).
Critical weaknesses
Theory
-
The "robust estimation" problem collapses to "nominal estimation + a stability modulus," so the main theorem is less informative than it sounds. The upper bound in Proposition 3.1 is additive, the Gaussian-blanket estimator does not depend on , and the lower bound in Theorem 3.2 is just the two-point statement that no estimator can distinguish in the ball. Consequently "minimax optimality over " is automatic once one has (a) a nominal minimax estimator for and (b) the worst-case sensitivity of in . The paper does not identify any regime in which robustness changes what an estimator should do, nor any trade-off between the two terms. The authors should state plainly that the shift cost is a property of the score map, not of learning, and position the contribution accordingly.
-
The statistical term is minimax only over an enormous class, and the "optimal" estimator is the smoothed empirical measure. The class is all compactly supported distributions with no smoothness. Over this class the minimax rate is attained by propagating the empirical measure through the OU kernel, i.e. by memorizing the training set. This is the opposite of what one wants from a theory of generative models: the resulting "generator" reproduces the training points plus Gaussian noise. The rate is also exponentially bad in through both and the constant . Prior minimax work on score estimation (Zhang et al. 2024; Dou et al. 2024; Stéphanovitch et al. 2025) obtains rates that improve with Hölder/Sobolev smoothness of ; the present theory cannot exploit smoothness at all. A result over a smoothness class would be substantially more meaningful.
-
The shift cost is a worst case driven by singular or oscillatory perturbations and is pessimistic for smooth references. The factor comes from the Gaussian-channel inequality , which is tight only for perturbations at the noise scale (the lower-bound instance uses on with frequency ). For a smooth and a smooth shift (e.g. a translation of a Gaussian), the actual score discrepancy is with no blow-up. The paper never distinguishes these regimes, so the headline message "the intrinsic cost is " is a statement about the worst element of rather than about typical distribution shifts. The optimality claim in Theorem 4.4 is also restricted to , i.e. to shifts smaller than the noise level at the earliest time; the regime , arguably the interesting one for "distribution shift," is not covered.
-
The comparison principle only applies to OU-compatible estimators, which excludes score networks trained by denoising score matching. Proposition 3.1 requires for a single initial law . A neural score trained by the standard loss does not satisfy this. Appendix G shows a ReLU network can approximate the Gaussian-blanket score uniformly, and Corollary G.4 transfers the bound for -close perturbations, but the authors themselves note (Appendix G.5) that this "does not imply that a particular score-matching training procedure recovers the constructed parameters" and that constants "may deteriorate as ." The connection to diffusion models as actually trained is therefore thin.
-
The reverse-sampling results in Section 6 are essentially restatements of the score results. With the "self-consistent" initialization , the reverse process driven by has law at time by construction, so (6.1) is literally Proposition 3.1 plus Proposition 4.1 with no sampling analysis. The Gaussian-initialization variant (Corollary F.4) adds only terms. Moreover, the KL is evaluated at time between OU-smoothed laws, not between the generated distribution and . Discretization is not treated. Calling this "robust generation" overstates what is shown.
-
The normalization makes the risk and the rates partly an artifact of averaging. Both the upper and lower bounds scale as , so letting sends every risk to zero. The standard denoising score-matching objective weights time by the diffusion coefficient and does not have this property. The authors should either use the unnormalized integral or explain why the averaged risk is the right object.
-
Low-dimensional structure is linear only, with known. Assumption 2 requires an exact linear subspace and known ; the data-splitting estimator uses the empirical span of samples. This is much weaker than the manifold-type results the paper cites (Chen et al. 2023a; Azangulov et al. 2024; Tang & Yang 2024), and the extension is straightforward given Proposition 3.1 (the only new ingredient is the missed-energy bound ). Adaptivity to unknown is not addressed.
Numerical study (Appendix H)
-
The experiment is too small to illustrate the claims. , is a four-atom discrete measure, so the Gaussian-blanket estimator is exactly a five-component Gaussian mixture whose only randomness is a multinomial count. Nothing is learned in any meaningful sense; the "finite-sample effect" is the fluctuation of four empirical frequencies. The shift family is twelve fixed translations, which are far from the worst case in (see Weakness 3); the plotted maximum over twelve directions is not an approximation of the supremum in (2.2). The dependence, which is the distinctive part of the rate, is not probed at all (the authors say so).
-
"Consistent with quadratic dependence" is asserted, not measured. Figure 1 overlays empirical curves on the theoretical upper bounds but reports no fitted slopes in , no tightness ratio, and no comparison with a non-OU-compatible baseline (e.g. a small score network trained by denoising score matching, or the plug-in empirical score without the blanket). Since the shift lower bound is only shown for an oscillatory family, a direct simulation of that family would have been more convincing than translations.
-
No neural-network or real-data experiment. Given that the motivation is pretrained diffusion models deployed under shift, even a modest experiment with a trained score network on a synthetic shift would test whether the predicted additive decomposition and scaling appear in practice.
Presentation
- Section 4.1 should state explicitly that the estimator interpolates the training data, and Section 4.3 should state that the minimax class contains no smoothness constraint and that the rate is therefore not comparable to smoothness-adaptive rates. The phrase "intrinsic cost of distribution shift" is used throughout for what is a worst-case stability constant.
Scores
- Soundness: 3 (good). The proofs are careful and, as far as I checked (Appendices B, D.1, F.1–F.2, H), correct. The claims are stated precisely; the issue is scope, not rigor.
- Presentation: 3 (good). Clear writing; some overclaiming in the framing ("robust generation," "intrinsic cost").
- Contribution: 2 (fair). The comparison principle is nice, but the robustness problem decouples trivially into nominal estimation plus a stability modulus; the statistical term is minimax only over a smoothness-free class and is attained by memorization; the shift cost is a worst-case over singular perturbations; and the generation and intrinsic-dimension extensions follow directly from Proposition 3.1.
- Rating: 2 — marginally below the acceptance threshold (weak reject). I would reconsider if the authors (a) establish a shift cost and matching lower bound over a smoothness class where the two terms genuinely interact, or otherwise demonstrate a regime where robustness changes the estimator, (b) analyze the regime , and (c) strengthen the numerical study along the lines of Question 7.
- Confidence: 4. I am familiar with the minimax score-estimation literature and with transport–entropy inequalities for Gaussian channels; I read the main text and the proofs of Proposition 3.1, Lemma B.1–B.3, Proposition F.2 and the numerical appendix in detail, and skimmed the lower-bound constructions.
Questions for authors
- Is there any setting in which the minimax-optimal estimator under shift differs from the nominal minimax estimator for ? If not, what does the robustness framework add beyond the stability bound of Lemma B.1?
- What is the shift cost for smooth references? For instance, if has a -Hölder density bounded below, is still of order , or does it drop to ? Can Theorem 3.2 be adapted to such a class?
- What happens for ? Is the upper bound still tight, and if not, what is the correct behavior?
- Can the statistical term be improved for smooth within your framework, e.g. by replacing the empirical measure in with a kernel or wavelet density estimator while keeping OU compatibility? This would make the "robust" rate adaptive to smoothness.
- The Gaussian blanket uses and the calibration , both of which require knowing . How sensitive is the bound to misspecification of , and is there an adaptive choice?
- In Theorem 5.1 the sample is split in half. Does the rate degrade if the empirical span is built from all samples without splitting, or is the splitting only a proof device?
- For the numerical study, could you (a) report the fitted exponent in on log–log axes, (b) add the oscillatory family from Appendix D as a shift family, and (c) include a non-OU-compatible estimator for comparison?
5 comments
Reply to Reviewer kgYb part2
2. Statistical complexity and worst-case distribution shift (W2–3, Q2–4)
Our framework deliberately considers compactly supported distributions without additional smoothness assumptions. This is not an overlooked limitation but a choice of statistical model. It allows us to characterize robustness for a broad distribution class, including discrete and singular measures.
The reviewer describes our estimator as "memorizing" the training samples. However, the Gaussian-blanket estimator is an explicitly regularized, OU-smoothed empirical construction, and its performance is established through quantitative risk bounds. The relevant mathematical question is whether the estimator achieves the minimax rate over the specified distribution class, which our results establish. Rates under additional Hölder or Sobolev assumptions concern different statistical problems and do not invalidate this characterization.
Similarly, the reviewer conflates worst-case robustness with behavior under particular structured perturbations. The observation that a Gaussian translation may yield a smaller score discrepancy does not contradict a sharp lower bound over the full Wasserstein neighborhood. In particular, smoothness of the reference distribution alone does not impose smoothness restrictions on the shifted distribution .
We emphasize that the matching distribution-shift lower bound is established in the small-shift regime specified in Theorem 3.2, including . We do not claim matching minimax optimality for larger shifts or over every restricted smoothness class. These are distinct theoretical questions requiring additional analysis, not counterexamples to the results established in the paper.
The reviewer is entitled to prefer a different distribution class or a broader parameter regime. However, such preferences should be distinguished from mathematical deficiencies in the present minimax characterization.
3. OU compatibility, neural estimators, and reverse sampling (W4–5)
The reviewer's criticism of OU compatibility overlooks a fundamental distinction between the construction used for the upper bound and the estimator class considered in the minimax problem.
Our minimax lower bounds apply to arbitrary measurable, possibly randomized estimators. OU compatibility is a sufficient structural condition used to construct an estimator achieving the upper bound, and it is not imposed on all competing estimators. Furthermore, Corollary G.4 and our constructive neural approximation results extend the robustness guarantee to score fields that need not be exact scores of OU-smoothed densities, provided the approximation error is controlled.
We do not claim that standard denoising score-matching training automatically achieves the required accuracy. Establishing guarantees for a particular neural training algorithm is a separate problem and is not necessary for the validity of our minimax results.
Regarding Section 6, we explicitly state that self-consistent initialization recovers the estimated OU trajectory. The contribution is the corresponding positive-time KL minimax characterization, including a matching lower bound over arbitrary output estimators. We also analyze standard Gaussian initialization separately in Appendix F.2. We do not claim a new discretization algorithm or convergence guarantees for sampling a potentially singular distribution at time zero.
Reply to Reviewer kgYb part3
4. Time normalization, intrinsic dimension, and estimator calibration (W6–7, Q5–6)
The factor defines a time-averaged score-estimation risk. Removing this normalization simply multiplies the corresponding upper and lower bounds by , without changing their dependence on the sample size, shift radius. Our results concern finite diffusion horizons. Moreover, the OU process considered here has a constant diffusion coefficient, so weighting the integral by its squared diffusion coefficient does not introduce a different time dependence.
Theorem 5.1 considers an unknown linear subspace with known intrinsic dimension . The unknown-linear-subspace geometry is also the setting considered by Chen et al. (2023a), whose network architecture uses a prescribed intrinsic dimension. Importantly, the estimator does not require exact subspace recovery or non-degeneracy assumptions on the latent distribution, and the shifted distribution is allowed to leave the reference subspace. The statistical dependence on the ambient dimension is replaced by dependence on . We do not claim general manifold adaptation or adaptation to unknown .
Regarding calibration, any valid upper bound on the support radius can be used in the Gaussian-blanket construction, although a looser bound may worsen the constants. Underestimating may invalidate the required domination property.
Sample splitting in Theorem 5.1 provides the conditional independence used to analyze the projected estimator. Our theorem establishes the stated rate for this construction, and we do not claim an analogous result for an unsplit estimator without further analysis.
5. Numerical evaluation and presentation (W8–11, Q7)
This submission is under the learning theory track, and its central contributions are theoretical minimax upper and lower bounds. The numerical study is intended to illustrate selected predictions, particularly the dependence on the sample size and distribution-shift radius, rather than provide an exhaustive empirical evaluation of diffusion-model training.
The finite-support example is a legitimate statistical estimation problem: fluctuations in empirical frequencies give rise to finite-sample estimation error. The experiments examine this effect and the behavior under controlled distribution shifts. We do not claim that the finite set of translation directions approximates the worst case over the entire Wasserstein ball or that the experiments independently establish minimax sharpness.
The review requests fitted exponents, oscillatory perturbations, additional estimator comparisons, neural-network training, and real-data evaluations. Some of these could provide useful complementary illustrations. However, the minimax optimality established in this work follows from matching mathematical upper and lower bounds, not empirical curve fitting. An exhaustive experimental study is not necessary to establish the principal theoretical contribution.
We will clarify the illustrative purpose of the experiments and the worst-case interpretation of the theoretical results. The absence of the additional experiments requested by the reviewer does not undermine the stated minimax guarantees.
Reply to Reviewer kgYb part1
We thank the reviewer for acknowledging the novelty of our problem formulation, the soundness of our comparison principle, and the careful establishment of our matching lower bounds. Yet, in Critical Weakness 1, the reviewer claims that robust minimax optimality is "automatic" from nominal estimation and a stability modulus, and further describes the robustness problem as one that "decouples trivially." We strongly disagree with this assessment. Our paper establishes a sharp minimax characterization of a previously unstudied score estimation problem under Wasserstein distribution shift. An additive decomposition does not make the result trivial, nor does a meaningful minimax characterization require fundamentally new proof techniques or a different optimal estimator.
We are also concerned about the length and scope of the review. It contains 11 extensive weaknesses and 7 additional questions, several of which call for substantial new theoretical results. In particular, the reviewer explicitly makes reconsideration of the rating conditional on deriving minimax results over smoothness classes or identifying a regime where robustness changes the optimal estimator, analyzing the large-shift regime, and conducting additional experiments. These demands go well beyond what can reasonably be expected during a conference rebuttal.
We therefore focus our response on the main theoretical objections, rather than addressing every proposed extension individually.
1. Novelty and significance of the robust minimax characterization (W1, Q1)
The reviewer's central assertion that our robust minimax result is "automatic" from nominal score estimation and a stability modulus is not justified.
Nominal score-estimation guarantees control errors under the reference distribution pt, whereas our robust risk involves simultaneous changes in both the target score and the evaluation distribution. A nominal L2(pt) guarantee does not directly imply a uniform L2(qt) bound over Wasserstein perturbations. Likewise, a Gaussian-channel stability inequality alone does not establish the required finite-sample robust upper bound.
Our analysis addresses this issue through Proposition 3.1 and the Gaussian-blanket construction. In particular, Proposition 4.1 establishes the quantitative Rényi-divergence estimate
\leq \log\left(1+\frac{M_{\vartheta,t_0}-1}{n+1}\right).$$ Combining this estimate with the comparison principle, we obtain the finite-sample robust upper bound.Matching lower bounds then establish the optimal statistical and distribution-shift dependence in Theorem 4.4. The reviewer's argument also overlooks the significance of the resulting characterization. The additive decomposition and the fact that the estimator requires no knowledge of the shift radius are **conclusions of our analysis**. They identify an irreducible distribution-shift error, characterize its dependence on the smoothing time, and determine when statistical estimation or distributional uncertainty dominates the robust risk. We also disagree with the premise that robustness must change the optimal estimator or produce a nontrivial interaction between error terms to constitute a meaningful theoretical contribution. Neither is a mathematical requirement for a minimax characterization. The absence of such an interaction is itself a conclusion about the statistical problem under consideration.