- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3(+1)
Read review
Paper summary
This paper studies the sample complexity of score-based diffusion models for learning distributions supported on low-dimensional smooth manifolds. Under certain regularity conditions, they show that the generated distribution has expected 1-Wasserstein error where denotes the ambient dimension, is the intrinsic dimension, is the sample size, and is the H"older smoothness parameter of the density.
Critical strengths
Their overall proof strategy and theoretical analysis appear sound. Specifically, the separation of score approximation error and statistical estimation error provides a clear framework for controlling the final sampling error. Furthermore, they use different constructions for comparison scores under different noise levels. These constructions are well motivated and yield convergence rates governed by the intrinsic dimension.
Critical weaknesses
-
The bound contains a factor , which may become substantial even when the intrinsic dimension is moderate. In comparison, "Convergence of Diffusion Models Under the Manifold Hypothesis in High-Dimensions" (Azangulov et al. 2025) obtains a Wasserstein bound with only dependence, while "Diffusion Models Are Statistically Optimal for Learning Low-Dimensional Multi-Modal Distributions" (Wu & Cai, 2026) obtains linear dependence on under a union-of-subspaces model. Although these results use different assumptions and estimators, the difference deserves further explanation. Can you identify which steps lead to the larger ambient dependence in your analysis and explain whether this dependence is necessary or could be improved?
-
Novelty compared with existing literature. "Adaptivity of Diffusion Models of Manifold Structures" (Tang & Yang, 2024) and "Convergence of Diffusion Models under the manifold hypothesis in high-dimensions" (Azangulov et al., 2025) already establish intrinsic-dimensional Wasserstein rates without explicitly estimating the manifold. Although the authors discuss these connections, the additional contribution could be made clearer. The main advance appears to be the finite-anchor Gauss–Newton construction and its explicit control of ambient-dimension dependence. Can you explain more precisely what new approximation or statistical guarantee this construction provides beyond the existing results, particularly since the sample-size exponent remains unchanged?
Questions for authors
-
The paper obtains the sample-size exponent , which matches the known benchmark for smooth-density estimation. However, comparisons between upper bounds under different assumptions do not directly establish minimax optimality for the distribution class considered here. Can you provide or cite a matching lower bound under your manifold and H"older-density assumptions, and clarify which geometry and density parameters are held fixed?
-
Please address Weaknesses 1 and 2.
-
Can the assumption that the density is bounded away from zero on the manifold be relaxed? Smooth densities on compact smooth manifolds may vanish at some points, so this assumption excludes otherwise regular distributions. Since the lower bound is used to control the denominators in the score approximation, what are the main obstacles to removing it? Would intrinsic-dimensional convergence guarantees still hold, possibly with a different rate?
4 comments
Response to reviewer tRFk (Part 2)
(Continued; notation as in part 1.)
W2: What new approximation or statistical guarantee does the construction give beyond existing results?
As you note, the main advance is the finite-anchor Gauss–Newton construction with explicit control of the ambient dimension. It gives two approximation results and one statistical guarantee that the cited works do not have.
(a) Approximation guarantees.
(i) Projection network (Theorem F.13). It approximates the nearest-point projection to any accuracy with width and sparsity up to logarithms. This count includes all costs of the construction, and the number of anchors does not depend on . In comparison:
- The Newton networks of Tang–Yang (2024, Lemma C.8) do not track these costs.
- The construction of Cloninger–Klock (2021, Corollary 5) costs .
- The projection network of Zhang et al. (2026, Lemma D.18) approximates a function of variables with smoothness order at least , and its constant may grow exponentially in .
This matters because the projection error enters the low-noise score as , so the accuracy has to increase as . The exponent keeps the score networks within the size bounds of Eqs. (8)–(9).
(ii) Low-noise score network (Lemmas F.27–F.28). The normal residual enters only through finitely many scalar products, so the coefficient functions to approximate depend only on the chart variables and on time. Tang–Yang (2024) expand in polynomials of the ambient variables instead.
(b) Statistical guarantee. The guarantee, a bound for ambient ERM with a fully tracked polynomial dependence on , is new; the argument behind it is standard. It is the interval-wise estimation of Oko et al. (2023) and Tang–Yang (2024), carried out here with explicit ambient costs for classes of intrinsic size that get smaller as the noise grows (Eq. (8)). In comparison, the bound of Tang–Yang (2024) hides constants that may grow like , as Yakovlev–Puchkin (2025, Section 3.1) point out, and the derivation of Zhang et al. (2026) uses only a global bound on the score risk, which does not give the smaller risks that the transfer needs on later intervals. The rate of Azangulov et al. (2024, arXiv v3, Corollary 3.2) is for their structured estimator (see W1 (d)). For their ambient class they bound the score risk linearly in (Theorem 3.3) and state no rate, and their stated model size (number of nonzero weights) has no term in , although the input and output layers are -dimensional (Appendix A.1).
Q1: Is there a matching lower bound for this class, and which parameters are held fixed?
Yes. We have added a matching lower bound, Proposition J.1.
(a) Statement. Over normalized densities in bounded below by , we have , even if is known. The proof uses Assouad's method (cf. Niles-Weed and Berthet (2022); Divol (2022)). So our sample exponent is minimax up to logarithms.
(b) What is held fixed. We fix , , , a compact smooth -manifold without boundary together with its density atlas, and , with , where . The constant depends only on these quantities.
Q3: Can the lower bound on the density be relaxed? What are the obstacles, and would intrinsic rates still hold?
This is a natural question. The present argument does not allow it, and we have added Appendix J.3 to explain why.
(a) Obstacle. Together with the reach condition, positivity gives the lower bound for the normalized low-noise denominator, and it gives local mass bounds at moderate noise. Smoothing keeps , but the lower bound need not be polynomial. For a smooth density with a flat zero, the denominator can be exponentially small in .
(b) Rates without positivity. Chakraborty, Berthet and Bartlett (2026) obtain rates close to , with no gain from smoothness. Keeping both the smoothness gain and intrinsic network sizes without positivity is open, and we see it as a natural next step.
Thank you again for the careful technical reading.
I thank the authors for the detailed response, which resolves most of my concerns. I agree that some dependence on is unavoidable and that the remaining factors are hard to improve for the general function class considered. The explicit treatment of the approximation error and the statistical guarantee clarifies the novelty, and the matching lower bound in Proposition J.1 complements the results well. I also agree that removing the positivity assumption while keeping both the smoothness gain and the intrinsic dimension is difficult. I have raised my scores for soundness and will maintain my overall positive rating.
Thank you for the careful follow-up and continued positive assessment. We appreciate your constructive feedback, which helped us clarify the contribution and strengthen the paper.
Response to reviewer tRFk (Part 1)
Thank you for the positive assessment and the detailed technical questions. In response, we added Table 2, which shows where each power of D comes from, a lower bound (Proposition J.1) and an example on positivity (Appendix J.3). Substantive changes are marked in blue in the updated manuscript. Below, εn=n−(β+1)/(2β+d) is the target rate, pβ,d=max⌈6β⌉+8,⌈5d/4⌉+13 is the power of D in Theorem 2.2, q is the chart smoothness order of Eq. (4), and G is a polynomial in the geometry and density bounds.
The dependence comes from two steps of our upper bound. A linear dependence of the network size on D is necessary. We do not think that the power pβ,d is necessary, and we point to one place where it can be improved.
(a) Which steps. The noise has norm about σtD, so every approximation must hold on a tube of radius σtD∨logn around the manifold, and all bounds are stated in D∨logn. The two entries of pβ,d have different sources (Table 2).
The step to W1 then takes a square root and adds one more factor D∨logn.
(b) What is necessary. An ambient network reads D coordinates and outputs a score in RD, so it needs at least order D nonzero weights (see the example in Appendix A.1), and the entropy bound of the network class grows with D. The early-stopped sampler also has bias Ω(D,εn) on manifolds padded with order D zero coordinates, even with the exact score (Appendix J.2).
(c) What can be improved. The size of the power comes from worst-case control on the tube. For example, the low-noise expansion contains scalar products of the normal residual with chart-derivative vectors. We bound each of them by the Cauchy–Schwarz inequality (proof of Lemma F.27), which costs a factor D∨logn, whereas the projection of Gaussian noise onto a fixed direction has size of order one (a heuristic, since these directions depend on the projection point). The proof of Lemma F.28 also notes that its bound on the low-noise integral errors is conservative. We have not carried out the sharper analysis.
(d) The rates you cite. Their estimators recover the geometry first, so the D-dimensional linear maps are computed from the data and are not learned by a network.
We learn the score directly in the ambient space, and Theorem 2.2 shows that the price is at most polynomial in D.