acceptodds
Under review as a conference paper at ICLR 2027

Efficiency of Diffusion Models for Infinite-dimensional Data Generation: Dimension Independent Approximation and Estimation Errors

Abstract

Despite the empirical success of diffusion models in generation of extremely high dimensional data such as images and videos, existing theoretical guarantees are largely confined to finite-dimensional spaces and suffer from the curse of dimensionality. This work provides the first rigorous statistical analysis of diffusion models in infinite-dimensional Hilbert spaces. To handle the absence of a Lebesgue measure in , we reformulate the score function using Radon-Nikodym derivatives relative to a Gaussian base measure, thereby eliminating the need for restrictive density lower-bound assumptions. By adopting the *-smooth function space* with different degrees of smoothness depending on directions and a novel Gaussian CDF transform, we derive polynomial-order, "dimension-independent” estimation error bounds for neural network score estimators. Our results establish a formal theoretical foundation for the scalability of diffusion models to infinite-dimensional data.

Primary area:
learning theory
Keywords:
diffusion models, score-based generative models, approximation theory, infinite-dimensional

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_bynAOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper studies score-based diffusion models on a separable Hilbert space HH (identified with R∞\mathbb{R}^\infty). Because there is no Lebesgue measure, the score is defined through the Radon–Nikodym derivative ρt=dPt/dμCt\rho_t = dP_t/d\mu_{C_t} with respect to a Gaussian reference measure, giving s(x,t)=2C∇log⁡ρt(x)−2CCt−1xs(x,t) = 2C\nabla\log\rho_t(x) - 2CC_t^{-1}x. The initial density ratio ρ0\rho_0 is assumed to lie in a γ\gamma-smooth function class (Okumoto & Suzuki, 2022) transported from [0,1]∞[0,1]^\infty to R∞\mathbb{R}^\infty by a coordinatewise Gaussian-CDF change of variables, with smoothness ai≍iηa_i \asymp i^\eta increasing along coordinates (mixed or anisotropic). The main results are (i) Theorem 4.4, a ReLU-network approximation bound for the first doutd_{out} coordinates of the score with error Γ2−2(1−v/a†)Γ\Gamma 2^{-2(1-v/a^\dagger)\Gamma} plus a truncation term dout1−2/r0d_{out}^{1-2/r_0}, and (ii) Theorem 5.1, bounds on En[W12(P0,Q^proj)]\mathbb{E}_n[W_1^2(P_0,\hat Q^{proj})] for an empirical-risk-minimizing score network under two loss weightings (Option 1: identity; Option 2: C−1C^{-1}). The rates are polynomial in nn with exponents that depend on the effective smoothness a†a^\dagger, the covariance decay r0r_0, and a construction-dependent constant κ\kappa, and they saturate at (2−r0)/4(2-r_0)/4 (Option 1) and (2−r0)/(2+r0)(2-r_0)/(2+r_0) (Option 2). The paper is purely theoretical; there are no experiments.

Critical strengths
  1. Relevant and timely question. Explaining why diffusion models scale to ultra-high-dimensional data is important, and a framework that avoids the ambient dimension entirely is a sensible way to attack it. Infinite-dimensional (functional) diffusion models exist in practice, and statistical theory for them is indeed lacking.
  2. Clean reformulation. Working with the density ratio ρt\rho_t relative to a Gaussian reference, the OU-semigroup representation of ρt\rho_t (Lemma A.6), and the Gaussian-CDF transform that pulls the [0,1]∞[0,1]^\infty trigonometric theory over to R∞\mathbb{R}^\infty is a natural and well-executed technical device.
  3. End-to-end argument. The paper goes from function-class assumptions to an explicit network class, approximation bounds, a covering-number/Bernstein oracle inequality (Proposition H.3), and a Wasserstein error decomposition (Lemma 5.2 / H.2) with Girsanov, early stopping, truncation and terminal projection all accounted for. The authors are candid about where the rates lose relative to finite-dimensional results.
  4. Honest discussion. The limitations paragraph correctly identifies the entropy factor κ\kappa, the KL→TV→W1W_1 conversion, and the missing lower bound as the sources of looseness.
Critical weaknesses

Theory

  1. Assumption 4.2 is a strong density lower bound in disguise, which undercuts the headline claim. The abstract and contribution list emphasize "eliminating the need for restrictive density lower-bound assumptions." But Assumption 4.2 requires ∥ρ0−1∥L∞≤1/2\|\rho_0 - 1\|_{L^\infty} \le 1/2, i.e. 1/2≤dP0/dμC0≤3/21/2 \le dP_0/d\mu_{C_0} \le 3/2 everywhere. This is a two-sided, uniform bound on the density ratio and it is used throughout (positivity of the denominator in ∂iρt/ρt\partial_i\rho_t/\rho_t, Gaussian domination P~t≤32N(0,Ct)\tilde P_t \le \frac32 N(0,C_t) in Lemma H.1, the exponential-moment bounds, the envelope GAG_A). Substantively, it says the data distribution is a mild bounded perturbation of a fixed Gaussian: it excludes any P0P_0 with compact support, with heavier or lighter tails than μC0\mu_{C_0}, with low-dimensional support, or with multimodal structure where the ratio departs from 1 by more than 50%. This is far more restrictive than the finite-dimensional lower-bound assumptions the paper contrasts itself with (e.g. Oko et al. 2023 assume a density bounded below on a compact support). The claim should be reworded, and the paper should discuss which realistic data models satisfy Assumption 4.2.

  2. The covariance assumptions tie the data to the noise process. Assumption 2.2 requires sup⁡ic0,i/ci\sup_i c_{0,i}/c_i and sup⁡ici/c0,i\sup_i c_i/c_{0,i} to be bounded and both sequences to decay as i−2/r0i^{-2/r_0}. Combined with Assumption 4.2, this means the second-order structure of P0P_0 is essentially dictated by the noise covariance CC chosen by the practitioner (or vice versa). In the latent-diffusion / Fourier-coefficient motivation given after Assumption 4.1, the decay of the data coefficients is a property of the data, not something that can be matched to CC without knowing it. How robust are the results to a mismatch, e.g. c0,i≍i−2/r0c_{0,i}\asymp i^{-2/r_0} but ci≍i−2/r1c_i \asymp i^{-2/r_1} with r1≠r0r_1\ne r_0?

  3. The rates are slow and saturate in a way that makes "dimension-independent" somewhat misleading. The paper reports bounds on En[W12]\mathbb{E}_n[W_1^2]. Taking a square root, the W1W_1 rate is at best n−(2−r0)/8n^{-(2-r_0)/8} (Option 1) or n−(2−r0)/(2(2+r0))n^{-(2-r_0)/(2(2+r_0))} (Option 2), i.e. never faster than n−1/4n^{-1/4}, regardless of smoothness. The saturation comes from the trade-off between the truncation term dout1−2/r0d_{out}^{1-2/r_0} and the doutd_{out}-linear entropy term doutΓ6log⁡n/nd_{out}\Gamma^6\log n/n, so the "ambient dimension" has been replaced by a free truncation dimension doutd_{out} that must be grown with nn at a rate governed by r0r_0. The paper's own finite-dimensional sanity check (Section 5) shows the exponent is worse than the TV-based s/(2s+d)s/(2s+d) of Oko et al. even before accounting for the W1W_1 conversion. Without a lower bound, it is unclear whether any of this is intrinsic. For a paper whose main message is "polynomial, dimension-independent rates," the fact that smoothness cannot push the rate past n−1/4n^{-1/4} in W1W_1 deserves a more prominent, quantitative discussion.

  4. The Wasserstein guarantee is obtained by a lossy KL→TV→W1W_1 route plus terminal projection. Lemma H.1 bounds W12(P~T‾,Q^proj)≲n−2+log⁡n⋅KLW_1^2(\tilde P_{\underline T}, \hat Q^{proj})\lesssim n^{-2} + \log n\cdot \mathrm{KL} via Pinsker on a ball of radius log⁡n\sqrt{\log n}. This is why the bound is for W12W_1^2 rather than W1W_1 and why the exponent is halved. Finite-dimensional work (Oko et al. 2023, Thm 5.4) obtains W1W_1 directly from the score error. The paper acknowledges this, but it means the "dimension-independent" rates are not comparable to existing finite-dimensional W1W_1 rates in a like-for-like way. The W2W_2 results in Appendix H.2 either need an extra regularity assumption (Assumption H.4, which includes an a.s. Lipschitz condition on the fitted estimator, condition (26), that is not implied by the ERM definition) or go through the same projection/Pinsker route.

  5. Anisotropic case: the stated parameter range is inconsistent. Assumption 4.3 allows any 0<η<∞0<\eta<\infty, but for anisotropic smoothness a†:=(∑iai−1)−1a^\dagger := (\sum_i a_i^{-1})^{-1} with ai≍iηa_i\asymp i^\eta is zero unless η>1\eta>1, in which case the condition v/a†<1v/a^\dagger<1 cannot hold and the exponents in Theorems 4.4 and 5.1 degenerate. Appendix B.1 also writes "under a1<a2a_1<a_2" while Assumption 4.3 allows a1≤a2a_1\le a_2. Please state the admissible range precisely for each smoothness type.

  6. Parts of Assumption 4.2 appear to be unused. The LL-Lipschitz condition on C∇ρ0C\nabla\rho_0 is stated in Assumption 4.2, but I could not locate where it enters the proofs of Theorem 4.4 or Theorem 5.1 (the only Lipschitz regularity used in the estimation argument is the separate Assumption H.4 for Case (iii)). If it is not needed, it should be removed; if it is needed, the step should be pointed to.

  7. The hypothesis class depends on unknown quantities. The clipping thresholds bgrad,i=Rgradci i−1/2−δ0b_{grad,i} = R_{grad}\sqrt{c_i}\,i^{-1/2-\delta_0} require knowledge of δ0\delta_0 from Assumption 4.2, and Γ\Gamma, doutd_{out}, T‾\underline T, TT are tuned using a†a^\dagger, vv, κ\kappa and r0r_0 (equation (20), Appendix H). The estimator is therefore not adaptive, and the paper does not discuss how to choose these in practice or what happens under misspecification. The Option 1 vs Option 2 comparison also relies on the crude conversion ∥z∥C−12≲dout2/r0∥z∥2\|z\|_{C^{-1}}^2\lesssim d_{out}^{2/r_0}\|z\|^2, which is where Option 1 loses; it is not clear this reflects a real difference between the losses rather than a proof artifact.

  8. The entropy constant κ\kappa is large and construction-specific. κ=9\kappa = 9 (mixed) comes from an eighth power of (Kρ+K′)≍2Γ/a1(K_\rho + K')\asymp 2^{\Gamma/a_1} in the covering bound (18)–(19). This dominates the exponent 2(a†−v)/(2(a†−v)+κ)2(a^\dagger-v)/(2(a^\dagger-v)+\kappa) unless a†a^\dagger is very large. The paper presents this as a feature of the construction, but it means that for moderate smoothness the bound is far from the approximation-theoretic limit 2−2(1−v/a†)Γ2^{-2(1-v/a^\dagger)\Gamma} suggested by Theorem 4.4.

  9. No time discretization and no lower bound. The generative guarantee is for the continuous-time reverse SDE with exact simulation of the top doutd_{out} coordinates. Pidstrigach et al. (2024) and Wang & Wang (2024), which the paper cites as motivation, already treat discretization in infinite dimensions, so combining with them should be possible. More importantly, the absence of a minimax lower bound for the class defined by Assumptions 2.2, 4.1–4.3 makes it impossible to judge whether the saturation and the κ\kappa loss are real.

Presentation / scope

  1. No numerical illustration at all. Even a toy experiment (e.g. a Gaussian-perturbed functional dataset satisfying Assumptions 2.2 and 4.2, comparing Option 1 and Option 2 and varying doutd_{out}) would help readers assess whether the predicted saturation and the Option 1/2 gap are visible, and whether the class in Assumption 4.2 is reasonable.

  2. The main text is hard to read on its own. The exponents in Theorem 5.1 involve κ\kappa, a†a^\dagger, vv, r0r_0 with the definitions spread across Sections 3–5 and Appendix B; the choice of doutd_{out} is only in Appendix H. A short table listing the rates for a few concrete (η,r0,p)(\eta, r_0, p) configurations would make the contribution concrete. There are also notational collisions (CC for both the operator on HH and C[1:dout]C_{[1:d_{out}]}; Φ\Phi for both the Gaussian CDF and the network class).

  3. The AI-use statement says generative AI was used "for formulating mathematical claims, providing critical ingredients for proving mathematical claims." Given the 60-page appendix, I spot-checked Lemma A.6, Lemma H.1, Proposition H.3 and the rate derivation in H.1 and found them coherent, but I was not able to verify the full approximation construction (Appendices E–G) line by line.

Scores

  • Soundness: 3 (good). The proofs I checked are correct, but the stated assumptions are inconsistent in the anisotropic case and one assumption appears unused.
  • Presentation: 2 (fair). The main text is dense and the rates are hard to interpret without the appendix; the "no density lower bound" claim is misleading.
  • Contribution: 2 (fair). The framework is natural and the end-to-end analysis is nontrivial, but the restrictive Assumption 4.2, the slow saturating rates, the lack of a lower bound, and the absence of any empirical illustration limit the significance.
  • Rating: 2 — marginally below the acceptance threshold (weak reject). I would raise my score if the authors (a) correct the claim about density lower bounds and characterize the class in Assumption 4.2 honestly, (b) fix the anisotropic parameter range, (c) either tighten the W1W_1 conversion or give a lower bound indicating the saturation is intrinsic, and (d) add at least a toy numerical illustration.
  • Confidence: 4. I am familiar with the finite-dimensional diffusion estimation literature and with the γ\gamma-smooth function class, and I read the main text and the estimation-error appendix carefully; I did not verify every step of the 40-page approximation construction.
Questions for authors
  1. Can you give a concrete family of non-Gaussian data distributions on HH satisfying Assumptions 2.1, 2.2, 4.1 and 4.2 simultaneously? In particular, is there any P0P_0 with non-full support, or with a density ratio exceeding 3/23/2 somewhere, for which the results still hold (perhaps with constants depending on ∥ρ0∥∞\|\rho_0\|_\infty and inf⁡ρ0\inf\rho_0)?
  2. Where exactly is the LL-Lipschitz condition on C∇ρ0C\nabla\rho_0 (Assumption 4.2) used?
  3. For anisotropic smoothness, what is the admissible range of η\eta, and does the condition v/a†<1v/a^\dagger<1 restrict pp in a way the main text should state?
  4. Is the saturation at (2−r0)/4(2-r_0)/4 and (2−r0)/(2+r0)(2-r_0)/(2+r_0) an artifact of the doutd_{out}-linear entropy term doutΓ6log⁡nd_{out}\Gamma^6\log n? Could a different parametrization of the linear branch −2CCt−1x-2CC_t^{-1}x (which is known up to the scalar mt2m_t^2, since CC and C0C_0 are assumed known) remove the doutd_{out} dependence entirely?
  5. What happens under covariance mismatch between C0C_0 and CC (Weakness 2)? Is K<∞K<\infty really needed, or only a one-sided bound?
  6. Can you comment on how your bound compares with simply projecting the data to the top doutd_{out} coordinates and applying a finite-dimensional result (e.g. Oko et al. 2023) with d=dout(n)d = d_{out}(n), keeping track of constants? If the finite-dimensional route gives comparable or better exponents under your assumptions, the added value of the infinite-dimensional analysis needs to be articulated more sharply.
  7. Why state Theorem 5.1 for En[W12]\mathbb{E}_n[W_1^2] rather than En[W1]\mathbb{E}_n[W_1]? The latter is what the finite-dimensional literature reports, and the Jensen step hides a factor of 2 in the exponent.
4 comments

Thank you for your detailed review. Your comments help clarify the assumptions, proof, and rate comparisons. We address them below.

A1. Distribution assumptions. Providing a concrete family modeling real data and satisfying all assumptions is difficult. However, our smoothness assumption is grounded in learning theory: Oko et al. (2023) analyze diffusion estimation over Besov spaces, and Okumoto and Suzuki (2022) establish learnability in the infinite-dimensional γ\gamma-smooth extension of mixed/anisotropic smoothness. Building on these structural assumptions, we establish polynomial convergence for infinite-dimensional generation.

The bounds 1/21/2 and 3/23/2 are not essential: keeping the other assumptions, any fixed 0<m≤ρ0≤M<∞0<m\leq\rho_0\leq M<\infty is allowed. Smoothing preserves m≤ρt≤Mm\leq\rho_t\leq M; adjusting clipping ranges and constants preserves convergence exponents. Ratios above 3/23/2 are allowed, but the positive ratio lower bound excludes distributions without full support. We remove a uniform positive lower bound on the Lebesgue density, not the Gaussian-relative ratio. Fu et al. (2024, Assumption 3.3) similarly use a Gaussian factor times a smooth function bounded above and away from zero.

A2. Lipschitz condition. We will remove the LL-Lipschitz condition on C∇ρ0C\nabla\rho_0 from Assumption 4.2: Theorems 4.4 and 5.1 and their rates remain valid without it. The ratio and coordinatewise derivative bounds remain; Gaussian smoothing at T‾>0\underline T>0 provides local regularity for the finite-dimensional reverse process. Additional regularity conditions are confined to Assumption H.4 in Appendix H.2, Case (iii), for the W2\mathcal W_2 bound.

A3. Anisotropic parameters. Thank you for catching this error. You are correct: the anisotropic case requires η>1\eta>1. We will correct it. Since v=(1/p−1/2)+v=(1/p-1/2)_+, v/a†<1v/a^\dagger<1 restricts pp when p<2p<2.

A4. Linear component and saturation. Standard diffusion models learn the full score, as analyzed by Oko et al. (2023). Here −2CCt−1x-2CC_t^{-1}x depends on unknown C0C_0, so treating it as known adds information. Saturation comes from balancing truncation and estimation errors, including linear-branch entropy; covariance weighting and moment bounds also contribute. Whether it is intrinsic or improvable remains open.

A5. Covariance mismatch. The shared eigenvalue decay order of C0C_0 and CC is a structural assumption. The proof uses both comparison directions to bound the score and data moments. Different decay orders make the comparison bound in one direction grow with the retained dimension, potentially affecting the rates.

A6. Finite-dimensional comparison. Even granting applicability of Oko et al. (2023, Theorem 5.4) to the projected distribution and ignoring dimension-dependent constants, minimizing the sum of n−(s+1−ε)/(2s+d)n^{-(s+1-\varepsilon)/(2s+d)} for fixed ss and truncation error d1/2−1/r0d^{1/2-1/r_0} gives d≍log⁡n/log⁡log⁡nd\asymp\log n/\log\log n. The resulting E[W1]\mathbb E[\mathcal W_1] bound is only of logarithmic order:

O ⁣((log⁡n/log⁡log⁡n)−(1/r0−1/2)).O\!\left((\log n/\log\log n)^{-(1/r_0-1/2)}\right).

We exploit the increasing coordinatewise smoothness encoded by γ\gamma-smoothness to obtain polynomial convergence to the original infinite-dimensional distribution while increasing the retained dimension. This does not follow by simply combining Oko et al.'s bound with truncation error.

The revised construction reduces κ\kappa from 99 to 11 for mixed smoothness and from log⁡23+8a†/a1\log_2 3+8a^\dagger/a_1 to 1+δ1+\delta for anisotropic smoothness, for any fixed δ>0\delta>0. In the paper's formal anisotropic comparison for Option 2 (dd fixed, p≥2p\ge2, a†=s/da^\dagger=s/d, r0↓0r_0\downarrow0), using the sharper finite-dimensional basis count, the E[W1]\mathbb E[\mathcal W_1] exponent improves from s/(2s+dlog⁡23+8)s/(2s+d\log_2 3+8) to s/(2s+d)s/(2s+d), closer to Oko et al.'s (s+1−ε)/(2s+d)(s+1-\varepsilon)/(2s+d).

A7. Squared distance. We agree that E[W1]\mathbb E[\mathcal W_1] is clearer for comparison. Theorem 5.1 reports the E[W12]\mathbb E[\mathcal W_1^2] bound obtained in our proof; Jensen's inequality gives an E[W1]\mathbb E[\mathcal W_1] bound with half the exponent. Section 5 explicitly makes this conversion. However, Weakness 3's claim that neither Option can converge faster than n−1/4n^{-1/4} is incorrect: Option 2's E[W1]\mathbb E[\mathcal W_1] saturation exponent is (2−r0)/(2(2+r0))(2-r_0)/(2(2+r_0)), approaching 1/21/2 as r0↓0r_0\downarrow0.

We also note: For Weakness 5, the original mixed-smoothness count used a1<a2a_1<a_2. The revision allows equality via G≲Γμ−12Γ/a1G\lesssim\Gamma^{\mu-1}2^{\Gamma/a_1}, where μ\mu counts coordinates with ai=a1a_i=a_1 (Appendix B.1, p. 22). Propagating this factor through the network and covering bounds changes only logarithmic rate factors.

For Weakness 1, Oko et al. (2023, Assumption 2.4) consider compactly supported densities, whereas our marginals have tails comparable to those of the Gaussian reference. Neither distribution class contains the other; our assumptions are not simply a stronger version of theirs.

AuthorsOpenReview ↗

Follow-up on the discussion

Thank you again for your detailed review. As the discussion draws to a close, we would appreciate hearing whether the clarifications and sharper bound in our response address your main concerns. If any point would benefit from further explanation, we would be happy to elaborate.

  • We confirmed that Theorems 4.4 and 5.1 hold without the Lipschitz condition on C∇ρ0C\nabla\rho_0 in Assumption 4.2. We will also correct the statement of the anisotropic case to require η>1\eta>1.

  • The revised construction yields a sharper convergence bound and reduces the entropy factor κ\kappa from 9 to 1 in the mixed-smoothness case. In the formal fixed-dimensional comparison in A6 of our response, the convergence exponent for E[W1]\mathbb{E}[W_1] under Option 2 improves to s/(2s+d)s/(2s+d), matching the exponent derived from the total variation bound of Oko et al. (2023). This improvement addresses your question about tightness by showing that some of the loss in the original rate came from our construction.

Reviewer_bynAOpenReview ↗

Response to the authors

Thank you for the detailed response and clarifications. I now have a positive assessment of the paper and will raise my score.

Thank you very much for your positive feedback and for reconsidering your assessment. We sincerely appreciate your time and thoughtful evaluation of our work.

Reviewer_1e7tOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper studies the approximation and statistical estimation of diffusion models in infinite-dimensional spaces. Under the assumption that the target distribution has a smooth density ratio with respect to a Gaussian reference measure, the authors derive neural network approximation bounds for the score function and finite-sample error bounds in Wasserstein distance. The convergence rates depend on the effective smoothness and covariance eigenvalue decay, but are independent of the input dimension.

Critical strengths
  • The paper addresses an important theoretical problem by studying the approximation error of score functions using ReLU neural networks and the sample complexity of diffusion models in infinite-dimensional spaces.
  • The paper establishes dimension-independent approximation bounds under assumptions on the smoothness of the Gaussian-relative density ratio and covariance eigenvalue decay. It also provides explicit network architectures and size bounds for achieving the approximation guarantees.
Critical weaknesses
  • The paper provides limited discussion of the practical implications of its theoretical results and assumptions. Please see the questions below.
Questions for authors
  • Approximation guarantees and practical training. Is Theorem 4.4 an existence result showing that a ReLU network of the specified size can achieve the given approximation bound? If so, are there any guarantees that practical training methods, such as SGD, can find a network with comparable approximation error?

  • Practical relevance of the assumptions. How should we interpret the assumptions in practice, especially the requirement that the density ratio with respect to a Gaussian reference measure is bounded between 1/21/2 and 3/23/2? Does this assumption exclude or significantly restrict distributions with well-separated modes? How realistic is this assumption for high-dimensional data distributions, such as natural images?

  • Advantages of infinite-dimensional analysis. Given the discussion of low-dimensional structure and VAE latent spaces, could the authors clarify when the infinite-dimensional analysis provides advantages over a finite-dimensional one?

3 comments

Thank you for your careful reading and thoughtful questions about the scope and practical relevance of our results. We address each point below.

A1. Approximation guarantees and practical training

Yes. Theorem 4.4 is a constructive existence result showing that a ReLU neural network of the specified size can achieve the stated approximation error.

Our study concerns neural network approximation and statistical estimation. Specifically, Theorem 4.4 evaluates the representational capacity of the network class, while Theorem 5.1 provides finite-sample error guarantees for the generated distribution when the score is an empirical risk minimizer of the denoising score-matching objective. This framework of analyzing approximation capacity and the statistical performance of empirical risk minimizers is shared by Schmidt-Hieber (2020), Okumoto and Suzuki (2022), and the diffusion-model analyses of Oko et al. (2023) and Chen et al. (2023).

Whether practical optimization methods such as SGD find networks satisfying these guarantees requires a separate optimization analysis. Such an analysis is beyond the scope of this work, and we do not provide a guarantee that SGD achieves comparable approximation error.

A2. Practical relevance of the assumptions

The specific constants 1/21/2 and 3/23/2 are not essential. The condition can be replaced by ∥ρ0−1∥∞≤c\|\rho_0-1\|_\infty\leq c for any fixed 0<c<10<c<1. What matters is that the density ratio has a positive lower bound and a finite upper bound, allowing us to control its reciprocal in the score. With corresponding adjustments to the constants, the same analysis applies without changing the convergence rates.

Conditions of this type also appear in prior work. The upper and lower density bounds in Oko et al. (2023, Assumption 2.4) are equivalent to upper and lower bounds on the density ratio relative to the uniform distribution on the support. Fu et al. (2024, Assumption 3.3) likewise consider densities given by a Gaussian factor multiplied by a smooth function bounded above and away from zero. Our work studies the same type of structure.

This condition does not rule out multimodality itself. Fixed upper and lower bounds constrain the depth of valleys between modes, but smaller positive lower bounds can be allowed, with their effect reflected in the constants of the error bounds.

For natural images, the latent representation in which diffusion is performed is important. For example, Vahdat et al. (2021) pretrain a VAE with a Gaussian prior to bring the latent distribution close to Gaussian, then use a diffusion model to learn the remaining mismatch. Through a nonlinear decoder, a latent distribution close to Gaussian is compatible with a complex image distribution. Our structure of a Gaussian reference measure with a smooth density ratio is a natural theoretical model for this approach to modeling in latent space. It is also consistent with the forward diffusion, under which the relative density ρt\rho_t approaches 11 in L2(μCt)L^2(\mu_{C_t}).

A3. Advantages of infinite-dimensional analysis

Low-dimensional structure in our work means that a small number of coordinates are dominant; it does not mean that dependence on all remaining coordinates disappears. Our assumptions model distributions in which higher-index coordinates still carry information, while their contributions and complex variations progressively diminish. The discussion of VAE latent representations likewise motivates this effective low-dimensionality.

The advantage of infinite-dimensional analysis is that it exploits this dominant low-dimensional structure while assessing the full distribution over all coordinates. Although the estimator uses finitely many coordinates, Theorem 4.4 also accounts for the effect of the omitted coordinates, and Theorem 5.1 provides a Wasserstein error guarantee relative to the original infinite-dimensional distribution.

We hope this clarifies how the two ideas fit together: low-dimensional structure enables dimension-independent learning rates, while infinite-dimensional analysis controls the error over all coordinates. This advantage is particularly relevant to functional data and to settings where the number of retained coordinates increases with resolution or the desired accuracy.

Thank you for the response. Most of my questions have been addressed.

The relaxation to 1−c≤ρ0≤1+c1-c\leq\rho_0\leq1+c is helpful, but the dependence of the error bounds on cc remains unclear, particularly as cc approaches 1. The discussion of VAE latent spaces is also useful, although it remains unclear whether practical latent distributions satisfy these assumptions.

Overall, I will maintain my score of 3.

Thank you for taking the time to read our response and for letting us know which questions remain. We are glad that the response addressed most of your questions.

Your point about cc is important. However, we can see that cc does not affect the result crucially. Indeed, for each fixed c<1c<1, the convergence exponent stays the same. Only the constants hidden in the order symbol are affected as cc approaches 1. It affects mainly the corresponding approximation error in a linear order, and thus even we take the ratio such as ∇ρt/ρt\nabla \rho_t/\rho_t, the hidden constant scales in a second order of max⁡1+c,1/(1−c)\max\\{1+c,1/(1-c)\\} which is relatively mild.

We also agree with your observation about VAE latent spaces. They motivate studying a density ratio relative to a Gaussian, but whether learned latent distributions exactly satisfy our uniform bounds remains an empirical question. On the other hand, we believe that the theoretical investigation as presented in our submission gives clearer novel insight on the current simplified setting rather than inducing excessive technical complexity.

Reviewer_4ysYOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper studies statistical guarantees of diffusion models for learning data distributions on infinite-dimensional Hilbert spaces. It represents the data distribution through a density ratio relative to a Gaussian reference measure and assumes coordinate-dependent smoothness and covariance decay. Under these assumptions, it constructs neural score approximations and derives the convergence rate of sampling error in the Wasserstein distance.

Critical strengths
  1. The paper addresses the important problem of characterizing the statistical accuracy of diffusion models for functional data.
  2. It incorporates finite-sample score estimation error into statistical guarantees of the generated distribution.
  3. It extends the existing theory in infinite-dimensional diffusion models from uniform reference measures to Gaussian reference measures.
Critical weaknesses
  1. The analysis assumes progressively increasing smoothness along higher-index coordinates, prescribed covariance decay, and a density ratio uniformly comparable to a Gaussian reference. Concrete examples satisfying these assumptions are needed to establish their relevance to the motivating applications.

  2. Since the main contribution is theoretical, matching lower bounds would substantially strengthen the results. Without them, it is difficult to determine whether the dependence on effective smoothness and covariance decay reflects fundamental statistical limits or analysis artifacts. In particular, the rate in Theorem 5.1 is not shown to be intrinsic to the distribution class.

  3. The analysis builds substantially on existing infinite-dimensional diffusion formulations and smoothness-based neural approximation frameworks. Moreover, the Gaussian CDF transformation itself is standard. Although the score approximation analysis requires additional work, the paper should distinguish the new technical ingredients and conclusions from those inherited from prior results.

Questions for authors
  1. Can you provide real-world data examples to support the imposed assumptions?

  2. What precisely is new about the dimension dependence of the established guarantees? In the context of infinite-dimensional data, one would expect the result not to depend on the ambient dimension, which is infinity.

  3. The second contribution on page 2 claims to eliminate density lower-bound assumptions. However, prior works already removed such assumptions in finite-dimensional diffusion estimation [1]. What additional technical advance is established here? Furthermore, how should this claim be interpreted, given that Assumption 4.2 imposes a uniform positive lower bound on the density ratio relative to the Gaussian reference?

  4. Can you provide matching minimax lower bounds, or identify regimes where the upper bounds in Theorem 5.1 are tight?

[1] Zhang, K., Yin, C. H., Liang, F., & Liu, J. (2024). Minimax optimality of score-based diffusion models: Beyond the density lower bound assumptions. arXiv preprint arXiv:2402.15602.

2 comments

Thank you for your careful review and questions about the assumptions, novelty, and rates. We welcome the opportunity to clarify these points and the limits of our results.

A1. Practical relevance of the assumptions

The true distribution of real-world data is unknown, and finite samples cannot certify that all our assumptions hold. Nevertheless, natural image generation provides concrete motivations for these assumptions. Natural-image wavelet coefficients have larger variance at coarser scales, so the contributions of different coordinates are not uniform (Simoncelli, 1999). Moreover, for CelebA-HQ image generation, Vahdat et al. (2021) apply a diffusion model only to the latent variables at the coarsest scale and use standard Gaussian priors for the remaining variables. This design, which primarily learns deviations from Gaussianity in a subset of components, motivates our γ\gamma-smoothness assumption, which permits more complex variation of the density ratio along dominant coordinates.

The upper and lower bounds on the density ratio are used in our score approximation analysis. The density bounds in Oko et al. (2023, Assumption 2.4) are equivalent to bounds on the density ratio relative to the uniform distribution on the support. More directly, Fu et al. (2024, Assumption 3.3) also consider densities given by a Gaussian factor multiplied by a smooth function with a positive lower bound and a finite upper bound.

A2. Novelty concerning dimension dependence

Even in an infinite-dimensional ambient space, fixing the number dd of coordinates used for estimation leaves a truncation error, while increasing dd requires controlling the estimation error. For example, in the finite-dimensional bound of Oko et al. (2023, Theorem 5.4), the convergence exponent tends to zero as dd increases with smoothness fixed. Thus, that bound does not imply polynomial convergence in infinite dimensions. Our contribution is to bound the expected squared W1\mathcal W_1 distance between the generated distribution and the original infinite-dimensional distribution by O(n−α)O(n^{-\alpha}), up to logarithmic factors (Theorem 5.1). The exponent α>0\alpha>0 is independent of the ambient dimension and reflects the effective smoothness of the density ratio and the decay of the covariance eigenvalues.

A3. Density lower bounds and the distinction from prior work

Our analysis does not require a uniformly positive lower bound on the Lebesgue density itself; it does impose a lower bound on the density ratio relative to a Gaussian reference measure. Consequently, the corresponding finite-dimensional marginal densities may tend to zero in the tails.

The guarantees of Zhang et al. (2024) concern a kernel-based score estimator, and analogous score estimation guarantees for neural networks are identified as future work. In neural network score approximation, a denominator approaching zero amplifies approximation errors in the numerator and denominator, making the quotient difficult to approximate. We express the score in terms of the density ratio relative to a Gaussian reference measure and use its positive lower bound to control the approximation of ∂xiρt/ρt\partial_{x_i}\rho_t/\rho_t. This yields neural network approximation and estimation guarantees without imposing a uniformly positive lower bound on the density itself.

A4. Tightness of the rates and comparison with prior work

We have not established a matching minimax lower bound for Theorem 5.1 or a regime in which its upper bound is provably tight. Since submission, however, we have improved the approximation network construction and the basis-counting bounds, yielding sharper convergence rates in the revised manuscript. This improvement shows that part of the loss in the original rates arose from the approximation construction and complexity bounds.

To illustrate the scale of the rates, consider a formal comparison in the anisotropic setting with p≥2p\geq2, a1=⋯=ad=sa_1=\cdots=a_d=s, and infinite smoothness in the remaining coordinates. Setting a†=s/da^\dagger=s/d, taking r0↓0r_0\downarrow0, and using the sharper basis count for fixed dd, the convergence exponent for E[W1]\mathbb E[\mathcal W_1] under Option 2 improves from s/(2s+dlog⁡23+8)s/(2s+d\log_2 3+8) in the original submission to s/(2s+d)s/(2s+d). This matches the exponent of the W1\mathcal W_1 bound derived from the TV bound in Oko et al. (2023, Theorem 5.1).

There remains a gap to their sharper exponent (s+1−ε)/(2s+d)(s+1-\varepsilon)/(2s+d) in Theorem 5.4, for any fixed ε>0\varepsilon>0. Our analysis bounds Wasserstein error via KL and TV, leaving room for improvement in this step. Whether the covariance-dependent saturation terms arising in infinite dimensions are intrinsic also remains to be established through lower bounds.

AuthorsOpenReview ↗

Follow-up of the discussion

Thank you again for your time and thoughtful review of our submission.

As the discussion period is drawing to a close soon, we wanted to gently follow up on our rebuttal posted earlier (including the revised manuscript with improved convergence rates mentioned in A4). We would greatly appreciate it if you could let us know whether our responses have addressed your questions and concerns, or if there are any remaining points we could further clarify before the deadline.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.