- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
The paper studies score-based diffusion models on a separable Hilbert space (identified with ). Because there is no Lebesgue measure, the score is defined through the Radon–Nikodym derivative with respect to a Gaussian reference measure, giving . The initial density ratio is assumed to lie in a -smooth function class (Okumoto & Suzuki, 2022) transported from to by a coordinatewise Gaussian-CDF change of variables, with smoothness increasing along coordinates (mixed or anisotropic). The main results are (i) Theorem 4.4, a ReLU-network approximation bound for the first coordinates of the score with error plus a truncation term , and (ii) Theorem 5.1, bounds on for an empirical-risk-minimizing score network under two loss weightings (Option 1: identity; Option 2: ). The rates are polynomial in with exponents that depend on the effective smoothness , the covariance decay , and a construction-dependent constant , and they saturate at (Option 1) and (Option 2). The paper is purely theoretical; there are no experiments.
Critical strengths
- Relevant and timely question. Explaining why diffusion models scale to ultra-high-dimensional data is important, and a framework that avoids the ambient dimension entirely is a sensible way to attack it. Infinite-dimensional (functional) diffusion models exist in practice, and statistical theory for them is indeed lacking.
- Clean reformulation. Working with the density ratio relative to a Gaussian reference, the OU-semigroup representation of (Lemma A.6), and the Gaussian-CDF transform that pulls the trigonometric theory over to is a natural and well-executed technical device.
- End-to-end argument. The paper goes from function-class assumptions to an explicit network class, approximation bounds, a covering-number/Bernstein oracle inequality (Proposition H.3), and a Wasserstein error decomposition (Lemma 5.2 / H.2) with Girsanov, early stopping, truncation and terminal projection all accounted for. The authors are candid about where the rates lose relative to finite-dimensional results.
- Honest discussion. The limitations paragraph correctly identifies the entropy factor , the KL→TV→ conversion, and the missing lower bound as the sources of looseness.
Critical weaknesses
Theory
-
Assumption 4.2 is a strong density lower bound in disguise, which undercuts the headline claim. The abstract and contribution list emphasize "eliminating the need for restrictive density lower-bound assumptions." But Assumption 4.2 requires , i.e. everywhere. This is a two-sided, uniform bound on the density ratio and it is used throughout (positivity of the denominator in , Gaussian domination in Lemma H.1, the exponential-moment bounds, the envelope ). Substantively, it says the data distribution is a mild bounded perturbation of a fixed Gaussian: it excludes any with compact support, with heavier or lighter tails than , with low-dimensional support, or with multimodal structure where the ratio departs from 1 by more than 50%. This is far more restrictive than the finite-dimensional lower-bound assumptions the paper contrasts itself with (e.g. Oko et al. 2023 assume a density bounded below on a compact support). The claim should be reworded, and the paper should discuss which realistic data models satisfy Assumption 4.2.
-
The covariance assumptions tie the data to the noise process. Assumption 2.2 requires and to be bounded and both sequences to decay as . Combined with Assumption 4.2, this means the second-order structure of is essentially dictated by the noise covariance chosen by the practitioner (or vice versa). In the latent-diffusion / Fourier-coefficient motivation given after Assumption 4.1, the decay of the data coefficients is a property of the data, not something that can be matched to without knowing it. How robust are the results to a mismatch, e.g. but with ?
-
The rates are slow and saturate in a way that makes "dimension-independent" somewhat misleading. The paper reports bounds on . Taking a square root, the rate is at best (Option 1) or (Option 2), i.e. never faster than , regardless of smoothness. The saturation comes from the trade-off between the truncation term and the -linear entropy term , so the "ambient dimension" has been replaced by a free truncation dimension that must be grown with at a rate governed by . The paper's own finite-dimensional sanity check (Section 5) shows the exponent is worse than the TV-based of Oko et al. even before accounting for the conversion. Without a lower bound, it is unclear whether any of this is intrinsic. For a paper whose main message is "polynomial, dimension-independent rates," the fact that smoothness cannot push the rate past in deserves a more prominent, quantitative discussion.
-
The Wasserstein guarantee is obtained by a lossy KL→TV→ route plus terminal projection. Lemma H.1 bounds via Pinsker on a ball of radius . This is why the bound is for rather than and why the exponent is halved. Finite-dimensional work (Oko et al. 2023, Thm 5.4) obtains directly from the score error. The paper acknowledges this, but it means the "dimension-independent" rates are not comparable to existing finite-dimensional rates in a like-for-like way. The results in Appendix H.2 either need an extra regularity assumption (Assumption H.4, which includes an a.s. Lipschitz condition on the fitted estimator, condition (26), that is not implied by the ERM definition) or go through the same projection/Pinsker route.
-
Anisotropic case: the stated parameter range is inconsistent. Assumption 4.3 allows any , but for anisotropic smoothness with is zero unless , in which case the condition cannot hold and the exponents in Theorems 4.4 and 5.1 degenerate. Appendix B.1 also writes "under " while Assumption 4.3 allows . Please state the admissible range precisely for each smoothness type.
-
Parts of Assumption 4.2 appear to be unused. The -Lipschitz condition on is stated in Assumption 4.2, but I could not locate where it enters the proofs of Theorem 4.4 or Theorem 5.1 (the only Lipschitz regularity used in the estimation argument is the separate Assumption H.4 for Case (iii)). If it is not needed, it should be removed; if it is needed, the step should be pointed to.
-
The hypothesis class depends on unknown quantities. The clipping thresholds require knowledge of from Assumption 4.2, and , , , are tuned using , , and (equation (20), Appendix H). The estimator is therefore not adaptive, and the paper does not discuss how to choose these in practice or what happens under misspecification. The Option 1 vs Option 2 comparison also relies on the crude conversion , which is where Option 1 loses; it is not clear this reflects a real difference between the losses rather than a proof artifact.
-
The entropy constant is large and construction-specific. (mixed) comes from an eighth power of in the covering bound (18)–(19). This dominates the exponent unless is very large. The paper presents this as a feature of the construction, but it means that for moderate smoothness the bound is far from the approximation-theoretic limit suggested by Theorem 4.4.
-
No time discretization and no lower bound. The generative guarantee is for the continuous-time reverse SDE with exact simulation of the top coordinates. Pidstrigach et al. (2024) and Wang & Wang (2024), which the paper cites as motivation, already treat discretization in infinite dimensions, so combining with them should be possible. More importantly, the absence of a minimax lower bound for the class defined by Assumptions 2.2, 4.1–4.3 makes it impossible to judge whether the saturation and the loss are real.
Presentation / scope
-
No numerical illustration at all. Even a toy experiment (e.g. a Gaussian-perturbed functional dataset satisfying Assumptions 2.2 and 4.2, comparing Option 1 and Option 2 and varying ) would help readers assess whether the predicted saturation and the Option 1/2 gap are visible, and whether the class in Assumption 4.2 is reasonable.
-
The main text is hard to read on its own. The exponents in Theorem 5.1 involve , , , with the definitions spread across Sections 3–5 and Appendix B; the choice of is only in Appendix H. A short table listing the rates for a few concrete configurations would make the contribution concrete. There are also notational collisions ( for both the operator on and ; for both the Gaussian CDF and the network class).
-
The AI-use statement says generative AI was used "for formulating mathematical claims, providing critical ingredients for proving mathematical claims." Given the 60-page appendix, I spot-checked Lemma A.6, Lemma H.1, Proposition H.3 and the rate derivation in H.1 and found them coherent, but I was not able to verify the full approximation construction (Appendices E–G) line by line.
Scores
- Soundness: 3 (good). The proofs I checked are correct, but the stated assumptions are inconsistent in the anisotropic case and one assumption appears unused.
- Presentation: 2 (fair). The main text is dense and the rates are hard to interpret without the appendix; the "no density lower bound" claim is misleading.
- Contribution: 2 (fair). The framework is natural and the end-to-end analysis is nontrivial, but the restrictive Assumption 4.2, the slow saturating rates, the lack of a lower bound, and the absence of any empirical illustration limit the significance.
- Rating: 2 — marginally below the acceptance threshold (weak reject). I would raise my score if the authors (a) correct the claim about density lower bounds and characterize the class in Assumption 4.2 honestly, (b) fix the anisotropic parameter range, (c) either tighten the conversion or give a lower bound indicating the saturation is intrinsic, and (d) add at least a toy numerical illustration.
- Confidence: 4. I am familiar with the finite-dimensional diffusion estimation literature and with the -smooth function class, and I read the main text and the estimation-error appendix carefully; I did not verify every step of the 40-page approximation construction.
Questions for authors
- Can you give a concrete family of non-Gaussian data distributions on satisfying Assumptions 2.1, 2.2, 4.1 and 4.2 simultaneously? In particular, is there any with non-full support, or with a density ratio exceeding somewhere, for which the results still hold (perhaps with constants depending on and )?
- Where exactly is the -Lipschitz condition on (Assumption 4.2) used?
- For anisotropic smoothness, what is the admissible range of , and does the condition restrict in a way the main text should state?
- Is the saturation at and an artifact of the -linear entropy term ? Could a different parametrization of the linear branch (which is known up to the scalar , since and are assumed known) remove the dependence entirely?
- What happens under covariance mismatch between and (Weakness 2)? Is really needed, or only a one-sided bound?
- Can you comment on how your bound compares with simply projecting the data to the top coordinates and applying a finite-dimensional result (e.g. Oko et al. 2023) with , keeping track of constants? If the finite-dimensional route gives comparable or better exponents under your assumptions, the added value of the infinite-dimensional analysis needs to be articulated more sharply.
- Why state Theorem 5.1 for rather than ? The latter is what the finite-dimensional literature reports, and the Jensen step hides a factor of 2 in the exponent.
4 comments
Follow-up on the discussion
Thank you again for your detailed review. As the discussion draws to a close, we would appreciate hearing whether the clarifications and sharper bound in our response address your main concerns. If any point would benefit from further explanation, we would be happy to elaborate.
-
We confirmed that Theorems 4.4 and 5.1 hold without the Lipschitz condition on in Assumption 4.2. We will also correct the statement of the anisotropic case to require .
-
The revised construction yields a sharper convergence bound and reduces the entropy factor from 9 to 1 in the mixed-smoothness case. In the formal fixed-dimensional comparison in A6 of our response, the convergence exponent for under Option 2 improves to , matching the exponent derived from the total variation bound of Oko et al. (2023). This improvement addresses your question about tightness by showing that some of the loss in the original rate came from our construction.
Response to the authors
Thank you for the detailed response and clarifications. I now have a positive assessment of the paper and will raise my score.
Thank you very much for your positive feedback and for reconsidering your assessment. We sincerely appreciate your time and thoughtful evaluation of our work.
Thank you for your detailed review. Your comments help clarify the assumptions, proof, and rate comparisons. We address them below.
A1. Distribution assumptions. Providing a concrete family modeling real data and satisfying all assumptions is difficult. However, our smoothness assumption is grounded in learning theory: Oko et al. (2023) analyze diffusion estimation over Besov spaces, and Okumoto and Suzuki (2022) establish learnability in the infinite-dimensional γ-smooth extension of mixed/anisotropic smoothness. Building on these structural assumptions, we establish polynomial convergence for infinite-dimensional generation.
The bounds 1/2 and 3/2 are not essential: keeping the other assumptions, any fixed 0<m≤ρ0≤M<∞ is allowed. Smoothing preserves m≤ρt≤M; adjusting clipping ranges and constants preserves convergence exponents. Ratios above 3/2 are allowed, but the positive ratio lower bound excludes distributions without full support. We remove a uniform positive lower bound on the Lebesgue density, not the Gaussian-relative ratio. Fu et al. (2024, Assumption 3.3) similarly use a Gaussian factor times a smooth function bounded above and away from zero.
A2. Lipschitz condition. We will remove the L-Lipschitz condition on C∇ρ0 from Assumption 4.2: Theorems 4.4 and 5.1 and their rates remain valid without it. The ratio and coordinatewise derivative bounds remain; Gaussian smoothing at T>0 provides local regularity for the finite-dimensional reverse process. Additional regularity conditions are confined to Assumption H.4 in Appendix H.2, Case (iii), for the W2 bound.
A3. Anisotropic parameters. Thank you for catching this error. You are correct: the anisotropic case requires η>1. We will correct it. Since v=(1/p−1/2)+, v/a†<1 restricts p when p<2.
A4. Linear component and saturation. Standard diffusion models learn the full score, as analyzed by Oko et al. (2023). Here −2CCt−1x depends on unknown C0, so treating it as known adds information. Saturation comes from balancing truncation and estimation errors, including linear-branch entropy; covariance weighting and moment bounds also contribute. Whether it is intrinsic or improvable remains open.
A5. Covariance mismatch. The shared eigenvalue decay order of C0 and C is a structural assumption. The proof uses both comparison directions to bound the score and data moments. Different decay orders make the comparison bound in one direction grow with the retained dimension, potentially affecting the rates.
A6. Finite-dimensional comparison. Even granting applicability of Oko et al. (2023, Theorem 5.4) to the projected distribution and ignoring dimension-dependent constants, minimizing the sum of n−(s+1−ε)/(2s+d) for fixed s and truncation error d1/2−1/r0 gives d≍logn/loglogn. The resulting E[W1] bound is only of logarithmic order:
O((logn/loglogn)−(1/r0−1/2)).
We exploit the increasing coordinatewise smoothness encoded by γ-smoothness to obtain polynomial convergence to the original infinite-dimensional distribution while increasing the retained dimension. This does not follow by simply combining Oko et al.'s bound with truncation error.
The revised construction reduces κ from 9 to 1 for mixed smoothness and from log23+8a†/a1 to 1+δ for anisotropic smoothness, for any fixed δ>0. In the paper's formal anisotropic comparison for Option 2 (d fixed, p≥2, a†=s/d, r0↓0), using the sharper finite-dimensional basis count, the E[W1] exponent improves from s/(2s+dlog23+8) to s/(2s+d), closer to Oko et al.'s (s+1−ε)/(2s+d).
A7. Squared distance. We agree that E[W1] is clearer for comparison. Theorem 5.1 reports the E[W12] bound obtained in our proof; Jensen's inequality gives an E[W1] bound with half the exponent. Section 5 explicitly makes this conversion. However, Weakness 3's claim that neither Option can converge faster than n−1/4 is incorrect: Option 2's E[W1] saturation exponent is (2−r0)/(2(2+r0)), approaching 1/2 as r0↓0.
We also note: For Weakness 5, the original mixed-smoothness count used a1<a2. The revision allows equality via G≲Γμ−12Γ/a1, where μ counts coordinates with ai=a1 (Appendix B.1, p. 22). Propagating this factor through the network and covering bounds changes only logarithmic rate factors.
For Weakness 1, Oko et al. (2023, Assumption 2.4) consider compactly supported densities, whereas our marginals have tails comparable to those of the Gaussian reference. Neither distribution class contains the other; our assumptions are not simply a stronger version of theirs.