acceptodds
Under review as a conference paper at ICLR 2027

Diffusion Models Break the Curse of Dimensionality for Mixed-Smooth Besov Densities

Abstract

Diffusion models learn a probability law by adding noise, learning how to denoise, and then running the noising process backward. In high dimensions, classical nonparametric theory predicts a curse: the sample-size exponent worsens with the ambient dimension. We show that this curse can be provably broken when the density has mixed smoothness, meaning that simultaneous high-frequency variation across many coordinates is strongly limited, the structure exploited by sparse grids. On the torus, for densities in a mixed Besov ball, we prove a total-variation minimax rate with the one-dimensional polynomial exponent and only logarithmic dependence on the dimension. We then show that a diffusion estimator using a sparse-grid score approximator attains this rate, up to standard algorithmic polylogarithms, under explicit score regularity and multiplier assumptions; for the multiplier condition holds on the whole class, so the rate is unconditional there. A controlled study reproduces the predicted scaling. Our analysis also reveals a new Wasserstein geometry: transport is governed by the finest coordinate scale, leaving sharp diffusion rates an important open direction.

Primary area:
learning theory
Keywords:
diffusion, mixed-smoothness, curse of dimensionality

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_brevOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

This paper studies distribution learning on Td\mathbb{T}^d with diffusion models under mixed Besov smoothness. The authors provide a proof for the minimax rate of order n−s/(2s+1)(log⁡n)s(d−1)/(2s+1)n^{-s/(2s+1)}(\log n)^{s(d-1)/(2s+1)} with respect to the TV divergence, where the distribution estimator is obtained via a score-based diffusion model. Specifically, sparse-grid/hyperbolic-cross approximation is used to approximate the score function under a multiplier/score-regularity condition that is verified uniformly for s>1/2s>1/2. The authors also developed lower bounds and partial upper-bound results for the Wasserstein distance W1W_1. Furthermore, numerical experiments on synthetic targets are also included to justify the findings.

Critical strengths

This paper gives a sharp minimax rate of learning distributions with mixed smoothness via diffusion models under the TV divergence. Specifically, for the polynomial part in the rate, the sample-complexity exponent becomes s/(2s+1)s/(2s+1), which is independent of dd. This is a useful characterization of how diffusion model can potentially resolve the isotropic curse of dimensionality in high-dimensional density estimation can. Regarding the technical part presented in the paper, proof of the upper bound didn't simply assume that the score belongs to the same function class. Instead, the paper is based on nontrivial assumption on the regularity. It proves an inverse-closure/multiplier result for s>1/2s>1/2, analyzes score matching via techniques from sparse-grids, and combines these different parts with the standard score-to-distribution reduction. The reviewer found this part to be technically solid and substantive.

Critical weaknesses

First, the reviewer thinks that the role of diffusion models in the ``curse-breaking'' claim should probably be framed more carefully. After examining the proof, it seems that the reduction in statistical complexity fundamentally comes from the mixed-smoothness assumption and the hyperbolic-cross representation. In fact, Proposition N.5(ii) shows that a direct hyperbolic-cross density estimator, without diffusion or score estimation, attains the same minimax rate over the entire class for every s>0s>0 under the TV divergence. Therefore, the main statistical phenomenon doesn't seem to be specific to diffusion models. Instead, the contribution is to show that a diffusion model based estimator can inherit the sparse-grid minimax rate under suitable score regularity. Therefore, the reviewer thinks that the title and abstract probably need to be slightly modified in order to reflect such distinction.

Secondly, the reviewer thinks that the empirical evidence only supports the high-dimensional claim partially. Specifically, the controlled study verifies decreasing score error for all tested dimensions and shows an end-to-end TV slope that resembles the theoretical results in d=2d=2. However, the preregistered equal-smoothness isotropic comparison does not consistently favor the mixed target in just slightly higher dimensions. In d=4d=4 and d=6d=6, the isotropic slopes are steeper. The matched-marginal comparison separates the targets, but it also substantially changes density concentration/envelope constants and therefore does not isolate interaction structure alone. Thus, I view the experiments as supportive but not yet strong evidence for the full statistical claim in high dimension.

Questions for authors
  1. Given Proposition N.5(ii), would it be possible for the authors to clarify more explicitly what the diffusion model part contributes statistically beyond the direct density estimator based on the hyperbolic-cross approximation? My reading is that the assumption on mixed Besov smoothness provides the curse-breaking rate, while the main result specific to diffusion models is that score matching/learning based on sparse grids and reverse diffusion can attain the same rate.

  2. How do the constants in Theorems 1 and 3 scale with respect to dimension? The polynomial exponent in the sample size nn is dimension-independent, but the paper allows dimension dependence through logarithmic powers and constants. Quantifying whether these constants are mild or exponential would help us reexamine the ``curse-breaking'' claim.

  3. How should the higher-dimensional results in the preregistered isotropic comparison be interpreted, especially for d=4d=4 or d=6d=6, where the isotropic slopes are steeper than the mixed slopes? Is there a possible comparison that keeps density concentration/envelope constants approximately fixed while changing only the interaction structure?

  4. For the W1W_1 results, should the manuscript be slightly adjusted in a way that the conditional diffusion upper bounds are presented as a roadmap or a central contribution? Since the sharp unconditional diffusion rate remains open, I think the paper would benefit from making this distinction clearer.

3 comments
AuthorsOpenReview ↗

Response to Reviewer brev

Thanks for highlighting the inverse-closure, score-learning and sampling analysis. We agree that the role of diffusion and the scope of the experiments needed clearer wording. Weakness numbers below follow your bullet order.

1. Diffusion's contribution, title and abstract (Weakness 1; Question 1)

Your reading of the diffusion contribution is right. Mixed smoothness supplies the statistical advantage; our contribution is to realize it through learned scores and reverse sampling. For sample size nn, fixed dimension dd and mixed smoothness ss, write

rn=n−s/(2s+1)(log⁡n)s(d−1)/(2s+1).r_n=n^{-s/(2s+1)}(\log n)^{s(d-1)/(2s+1)}.

Direct density estimation attains this total-variation (TV) rate for every s>0s > 0. The original single-noise diffusion bound has an extra log⁡n\sqrt{\log n}; new Theorem Q.2 removes it using repeated noising for s>1/2s > 1/2 on the full bounded positive class. We've rewritten the abstract and discussion accordingly. We propose “Diffusion Models Attain Sparse-Grid Rates under Mixed Smoothness” once title changes are permitted after discussion, as required by the author guidelines.

2. Dimension-dependent constants (Question 2)

On dimension dependence, we cannot claim mild constants in general. Our coarsest tensor space already contains 2d2^d features, and the multiplier constants can also grow with dd. Section 6 now makes the fixed-dimension scope explicit. Appendix S.6 gives an optimistic restricted result: for a known partition into BB independent blocks of at most qq coordinates, fitting and sampling each block separately gives expected TV error at most

C0B,n−s/(2s+1)(log⁡n)s(q−1)/(2s+1).C_0\sqrt B\\,n^{-s/(2s+1)}(\log n)^{s(q-1)/(2s+1)}.

Here s>1/2s > 1/2, qq is fixed, and the global mixed-norm radius and positive density bounds stay fixed as dd grows. Then C0C_0 is independent of d,B,nd,B,n. Unknown blocks and general cross-block dependence remain open.

3. Higher-dimensional slope ordering (Weakness 2; Question 3)

On the original experiments, we retain the steeper isotropic slopes at d=4,6d=4,6 and explicitly acknowledge that they do not validate the high-dimensional minimax claim. A worst-case rate over a density class does not order finite-sample risks for individual targets. Target spectra, the noise window and selected model sizes can affect those curves; we have not established which explains the reversal. Section 7 and Appendix L now say this directly.

4. Matching concentration in the new control (Weakness 2; Question 3)

We also added the requested control in Appendix T. Two four-dimensional targets have pairwise versus four-coordinate density terms, but identical one-dimensional marginals, bounds [0.2,1.8][0.2,1.8], density-value distributions, radial spectra and heat-flow Fisher information. At n=262,144n=262{,}144, over eight paired seeds with matched candidate feature budgets, the geometric mean of the per-seed four-coordinate/pairwise ratios of normalized integrated squared score error is 2.142.14 for hyperbolic ordering and 0.990.99 for Euclidean ordering. Descriptive 95% paired-bootstrap intervals are [2.09,2.18][2.09,2.18] and [0.97,1.01][0.97,1.01], respectively. This supports a representation-dependent effect within this controlled family. It uses single-noise score matching and does not test the repeated-noise refinement, generated-law TV or minimax rates.

5. Conditional Wasserstein results (Question 4)

Finally, we now present the refined Wasserstein-1 results as conditional transfers. The localized score-error, initialization and reverse-marginal density conditions still need verification for the learned sampler; an overall score-risk bound alone does not supply them. The corrected statements are in Appendices D and P. The abstract, Section 6 and conclusion explicitly leave a sharp unconditional diffusion Wasserstein-1 guarantee open.

Reviewer_brevOpenReview ↗

Response

The reviewer would like to thank the authors for the detailed response, which has addressed most of the concerns. Therefore, the reviewer is glad to keep the positive rating of 3.

Thank you for your thoughtful review and for taking the time to consider our response. We are glad that our clarifications addressed most of your concerns, and we appreciate your positive assessment and support.

Reviewer_3nNsOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

They obtains statistical rates for diffusion models in the setting of density estimation (similar to Oko et al. 2023). Instead of the typical Besov smoothness assumption, they employ a mixed Besov smoothness assumption that penalises joint interactions between cooridnates. They develop a framework for density estimation in this setting and provide information theoretic lower bounds for density estimation. They then propose a class of score functions such that when the ERM is minimised, they obtain minimax statistical rates. The method for this is a is a sparse-grid truncation.

Critical strengths
  • Seems to be the first approach at diffusion-based density estimation in the mixed Besov setting. Its a nice conceptual contribution to the density estimation literature given the way it breaks the curse of dimensionality.
  • Total variation analysis comes with a matching lower bound, demonstrating the tightness of the result
  • The paper proposes several interesting open problems that the literature may find interesting (e.g. refining the Wasserstein bound to match lower bounds)
Critical weaknesses
  • Restricted to settings where data is supported everywhere on a torus with a density lower bound which is quite restrictive. Also only Hilbert case is considered, but general Besov space is of interest in the literature. The paper feels unnecessarily limited in scope.
  • Class of score networks is based on sparse grids and not the more realistic (and common in the literature) approach of using NNs. Appendix M.4 now explains how sparse grids can be represented by NNs, but there is no guarantee that a training procedure on NNs would learn them.
  • While the mixed-smooth setting is interesting, there is no convincing argument for why they are a realistic or sensible setting to consider in practice: the examples added in Section 2 are plausible, but the paper’s own real-data check (Appendix O) does not support the assumption.
Questions for authors
  • Is there a good argument for considering the mixed-smoothness setting? Is it known to relate to realistic settings in density estimation/generative modelling?
  • Is there a reason that there are the restrictions listed in Weakness 1 and 2? Would it be easy to add these things to the paper?
1 comment
AuthorsOpenReview ↗

Response to Reviewer 3nNs

Thanks for recognizing the mixed-smoothness diffusion framework and the TV guarantees. We also appreciate your interest in the Wasserstein questions. Your concerns about scope and practical motivation are fair. We've added extensions with proofs in Appendix S and clarified what remains open. Write

rn=n−s/(2s+1)(log⁡n)s(d−1)/(2s+1),r_n=n^{-s/(2s+1)}(\log n)^{s(d-1)/(2s+1)},

where nn is the number of independent observations, dd the fixed dimension, and ss the mixed smoothness. TV below means total variation. Weakness numbers follow your bullet order.

1. Domain, density bounds and Besov indices (Weakness 1; Question 2)

The original restrictions simplify different steps: periodicity avoids boundary terms, the density floor controls division by the density and regression stability, and Hilbert geometry gives orthogonal risk and heat-energy estimates. Several relaxations now fit in the revision:

  • Appendix S.1 covers ordinary nonperiodic [0,1]d[0,1]^d, without boundary matching or a smooth reflected extension of the true density. With fixed positive lower and upper density bounds, boundary-wavelet density estimation followed by diffusion sampling attains TV error O(rn)O(r_n) for every s>0s > 0. This changes the learner; a direct boundary-aware denoising score-matching learner remains future work.
  • Appendix S.2 allows the target density pp to vanish. On the unit torus, train diffusion on q=(p+1)/2q=(p+1)/2, where 11 is the uniform density, and learn the acceptance probability η=p/(p+1)\eta=p/(p+1) from the known mixture labels. Since ηq=p/2\eta q=p/2, a capped rejection step recovers the target at TV rate O(rn)O(r_n) for s>1/2s > 1/2. This uses the repeated-noise learner of Theorem Q.2 and adds scalar regression. Compactness, the density upper bound and the mixed-norm bound remain.
  • Appendix S.3 treats Bρ,τs,mixB^{s,\mathrm{mix}}_{\rho,\tau}, where ρ\rho controls spatial integrability and τ\tau sums mixed-resolution blocks. For ρ≥2\rho\ge2, 0<τ≤20 < \tau\le2 and s>1/2s > 1/2, the same repeated-noise learner retains the O(rn)O(r_n) upper bound. For ρ,τ≥2\rho,\tau\ge2 and s>0s > 0, a density-based diffusion construction attains the sharp TV rate n−s/(2s+1)(log⁡n)(d−1)(1/2−1/[τ(2s+1)])n^{-s/(2s+1)}(\log n)^{(d-1)(1/2-1/[\tau(2s+1)])}, with 1/τ=01/\tau=0 at τ=∞\tau=\infty. These results retain fixed positive density bounds. Sharp score-matching guarantees for all Besov indices need further work.

These extensions concern TV; the refined Wasserstein results still require their separate learning and stability conditions.

2. Neural training (Weakness 2; Question 2)

On neural training, we agree that representability alone is insufficient. Appendix S.4 now gives a restricted training guarantee: prescribe ReLU hidden features and train the linear readout by gradient descent at each sampling time. With explicit feature and optimization tolerances, this preserves O(rn)O(r_n) for s>1/2s > 1/2 on the bounded positive class. It does not analyze freely training all layers of a time-conditioned network.

3. Practical motivation and the real-data diagnostic (Weakness 3; Question 1)

For practical motivation, Appendix S.5 proves membership for dependent interaction models, for example

p(x)=Z^{-1}\exp\left\\{\sum_i g_i(x_i)+\sum_{i < j}g_{ij}(x_i,x_j)\right\\}.

Here xix_i are angular coordinates, ZZ normalizes the density, and gi,gijg_i,g_{ij} are single-coordinate and pairwise log-potentials. Ordinary Sobolev regularity ss and 2s2s, respectively, with controlled total norms and s>1/2s > 1/2, suffices for mixed-smooth membership. This gives a concrete dependent model family; the coordinate system and interaction strength matter.

We also agree that the wind-data diagnostic does not validate the assumption. Appendix O now says this explicitly: recorded directions are quantized and dependent, and the recorded law is discrete. The plots cannot establish mixed smoothness of an unobserved continuous density. We retain these findings as a limitation; the model examples do not turn them into positive empirical evidence.

Reviewer_FsKiOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The manuscript analyzes diffusion-based density estimation under mixed Besov smoothness assumptions. The main contribution is in the sharpened lognlog n factors in the minimax lower bound. The manuscript develops a sparse-grid score estimation and show that its associated diffusion sampler achieves this minimax rate upto poly-log⁡(n)\operatorname{poly-log}(n) factors (I think this is log⁡n\sqrt{\log n} from their Proposition J.3) of this minimax risk. Although the problem is statistically interesting, the contributions seem to be incremental as prior works (such as Ding et al. (2021)) already establish such rates, although without the sharp log⁡n\log n rates.

Critical strengths
  • The connection between mixed-smoothness approximation and diffusion score estimation is interesting.
  • The score regularity analysis, random-design regression bounds, and discretized sampling guarantee require additional work beyond direct density estimation.
Critical weaknesses

Contributions Ding et al. (2021) have already established the error rate (except for log⁡n\log n-sharpness) under mixed smoothness. The improvement is only in the lower bound and the main upper bound (Theorem 3) might not achieve this sharpness in the log⁡n\log n factors. The main contribution in the upper bound seems extending mixed smoothness estimation guarantees to a particular construction. The minimax lower bound proof also closely follows the construction by Ding et al. (2021), whereas the upper bound argument is similar to that by Oko et al. (2023). Though the proof refines both, I would call it incremental.

Assumption 3 The multiplier inequality in Appendix D.1, page 19, lines 981–983, fails in simple cases. When the smooth density is non-constant, applying the inequality to the constant function, the RHS renders zero, whreas, the LHS is strictly positive. Am I missing something?

Related works the authors miss several prominent recent works, comparison against which is necessary to clarify the contributions. Tang and Yang [2] study adaptation to manifold structure, Kwon et al. [3] study factorizable densities, and Chakraborty, Berthet, and Bartlett [4] establish intrinsic-dimension Wasserstein guarantees. Their assumptions and losses differ from those considered here, so these results do not directly subsume the paper. However, a substantive comparison would help clarify what mixed smoothness adds to existing explanations of how diffusion models avoid ambient-dimensional rates.

References

[1] Liang Ding, Lu Zou, Wenjia Wang, Shahin Shahrampour, and Rui Tuo. High-Dimensional Non-Parametric Density Estimation in Mixed Smooth Sobolev Spaces. arXiv:2006.03696, 2021.

[2] Rong Tang and Yun Yang. Adaptivity of Diffusion Models to Manifold Structures. AISTATS, PMLR 238:1648–1656, 2024.

[3] Hyeok Kyu Kwon, Dongha Kim, Ilsang Ohn, and Minwoo Chae. Nonparametric Estimation of a Factorizable Density using Diffusion Models. Journal of Machine Learning Research, 27(22):1–125, 2026.

[4] Saptarshi Chakraborty, Quentin Berthet, and Peter L. Bartlett. Generalization Properties of Score-matching Diffusion Models for Intrinsically Low-dimensional Data. arXiv:2603.03700, 2026.

Questions for authors
  • Can Assumption 3 be corrected while preserving the associated conclusions?
5 comments
AuthorsOpenReview ↗

Response to Reviewer FsKi

Thanks for the careful review and for recognizing the connection to score estimation and the technical work on score regularity, regression and discretized sampling. We've corrected Assumption 3 and clarified the contribution and remaining conditions. Below, nn is the sample size, dd the fixed dimension, and ss the mixed smoothness. Weakness numbers follow your bullet order.

1. Assumption 3 and its consequences (Weakness 2; Question 1)

On Assumption 3, you're right: the original inequality is false. We now include the constant mode and keep the same negative order on both sides. An equivalent Fourier formulation of the corrected norm is

\qquad \widehat f(k)=\int_{\mathbb T^d}f(x)e^{-2\pi i k\cdot x}\\,dx.$$ Here $f$ is a periodic vector field on the unit torus $\mathbb T^d$, $k$ is an integer frequency, and $t > 0$ is the forward noising time. For the heat-evolved density $p_t$ of a target satisfying $0 < a\le p_0\le b$, Lemmas P.1 and P.2 prove $$\\|p_tf\\|_{t,-2}\le Cb\\|f\\|_{t,-2},$$ where $C$ is independent of $t$, $f$ and $n$. This includes $k=0$ and fixes both the missing constant mode and the invalid change of order. The remaining learning condition is explicit: revised Assumption 3 requires localized score-error control (Eq. 26), which ordinary $L^2$ risk does not establish. The added constant-mode contribution is only $O(n^{-1/2})$ for a fixed time horizon. Theorem 6 retains its rate for the signed linearized response. Corollary 2 retains the sampler transfer under exact initialization, Assumption 4's total-variation and score-error controls, a reverse-sampler density bound uniform in $n$, $d > 2$ and $s > 2/d$. The total-variation (TV) main theorem does not use Assumption 3. The corrected statements and proofs are in D.1 and D.2 and P; sharp unconditional Wasserstein-1 convergence remains open. **2. Contribution and logarithmic gap (Weakness 1)** On the logarithmic gap, the original upper bound is $O(r_n\sqrt{\log n})$, where $$r_n=n^{-s/(2s+1)}(\log n)^{s(d-1)/(2s+1)}.$$ New Theorem Q.2 attains $O(r_n)$, including Euler sampling, using $K\asymp\log n$ independent noising draws per observation. Integrated heat-energy control removes the approximation logarithm, while a grouped variance calculation reduces the noising variability. The independent sample size remains $n$. The required reciprocal-density multiplier is verified over the full bounded positive class for $s > 1/2$; at lower smoothness it remains an explicit condition. The improvement applies to this modified learner; the original $K=1$ bound is unchanged. On the contribution, we agree that the polynomial advantage comes from mixed smoothness and was already present in Ding et al. Our lower-bound refinement identifies the full TV logarithm through $L^1$ separation on the hyperbolic shell. We credit Oko et al.'s diffusion framework. The additional work is mixed score regularity, random-design score learning, and certified reverse sampling, now with exact TV logarithms for the modified learner. Sections 1 and 6 and Appendix R make this scope explicit. **3. Related work (Weakness 3)** We've also added substantive comparisons in Appendix R. Tang & Yang exploit lower-dimensional manifold support for Wasserstein guarantees. Kwon et al. allow overlapping low-dimensional density factors and dependence. Chakraborty et al. use covering-based intrinsic dimension and moment conditions, without requiring a manifold. Our gain comes from mixed regularity of a full-dimensional density. It requires neither reduced support dimension nor low-dimensional factorization; Appendix R gives an example ruling out factors on proper coordinate subsets. The different assumptions and losses prevent a general ranking of these methods.
Reviewer_FsKiOpenReview ↗

Response to Authors

I thank the authors for their detailed response. However, my concerns remain regarding the revised guarantees and significance of the work. I think the new arguments regarding my previous objection to the invalidity of Assumption 3 is resolved. However, my main concerns regarding the significance remain. The dimension-independent polynomial exponent comes from the already well-established theory of mixed Besov smoothness (Ding et al. 2021). I find the implications for trained neural networks limited, and the sharper log⁡n\log n improvements remain conditional on additional requirements, as the authors clearly acknowledge. Though the improvement on the lognlog n terms requires added technicality, I do not find the contributions strong enough for ICLR and would retain my score.

Thanks for following up. We’re glad the Assumption 3 issue is resolved. We think the remaining concerns overlook three concrete parts of the revised results.

1. The main sharp TV guarantee has its key conditions verified.

For s>1/2s>1/2, where ss denotes mixed smoothness, Proposition J.1 proves the required regularity condition throughout our stated bounded positive density class. Theorem Q.2 then achieves the exact total-variation (TV) minimax rate, including its logarithmic factor, with explicit learning and sampling controls. The additional conditions for lower smoothness and refined Wasserstein results belong to separate statements. They should not be read as unresolved assumptions behind this main TV result.

2. A known density-estimation rate does not automatically give a diffusion guarantee.

We fully credit Ding et al. for the mixed-smoothness advantage and Oko et al. for the diffusion framework. Our contribution is to show how that advantage survives score learning and reverse sampling. This requires controlling division by the evolving density, regression error and sampling error together. The sharp logarithm additionally requires heat-energy estimates and a careful separation of data variability from artificial noising. These are substantive mathematical steps. Knowing the original density-estimation exponent does not establish them.

3. We also give a new, explicit neural-training guarantee.

To our knowledge, Appendix S.4 gives the first constructive neural-training guarantee achieving this sharp mixed-smooth TV rate under the stated assumptions. We prescribe ReLU hidden features, train their linear readouts with finitely many gradient-descent steps, and prove that the resulting Euler sampler retains the rate. Feature approximation, optimization error and sampling error are all controlled. This shows that the statistical advantage survives an explicit training and generation procedure. Learning all hidden features would broaden the result; it is a further problem, not a prerequisite for this contribution to matter.

This is also the contribution Reviewer 3nNs recognized as a “nice conceptual contribution” and Reviewer brev described as “technically solid and substantive.” We appreciate those assessments and would ask you to reconsider the significance of the full result on that basis.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.