- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper studies distribution learning on with diffusion models under mixed Besov smoothness. The authors provide a proof for the minimax rate of order with respect to the TV divergence, where the distribution estimator is obtained via a score-based diffusion model. Specifically, sparse-grid/hyperbolic-cross approximation is used to approximate the score function under a multiplier/score-regularity condition that is verified uniformly for . The authors also developed lower bounds and partial upper-bound results for the Wasserstein distance . Furthermore, numerical experiments on synthetic targets are also included to justify the findings.
Critical strengths
This paper gives a sharp minimax rate of learning distributions with mixed smoothness via diffusion models under the TV divergence. Specifically, for the polynomial part in the rate, the sample-complexity exponent becomes , which is independent of . This is a useful characterization of how diffusion model can potentially resolve the isotropic curse of dimensionality in high-dimensional density estimation can. Regarding the technical part presented in the paper, proof of the upper bound didn't simply assume that the score belongs to the same function class. Instead, the paper is based on nontrivial assumption on the regularity. It proves an inverse-closure/multiplier result for , analyzes score matching via techniques from sparse-grids, and combines these different parts with the standard score-to-distribution reduction. The reviewer found this part to be technically solid and substantive.
Critical weaknesses
First, the reviewer thinks that the role of diffusion models in the ``curse-breaking'' claim should probably be framed more carefully. After examining the proof, it seems that the reduction in statistical complexity fundamentally comes from the mixed-smoothness assumption and the hyperbolic-cross representation. In fact, Proposition N.5(ii) shows that a direct hyperbolic-cross density estimator, without diffusion or score estimation, attains the same minimax rate over the entire class for every under the TV divergence. Therefore, the main statistical phenomenon doesn't seem to be specific to diffusion models. Instead, the contribution is to show that a diffusion model based estimator can inherit the sparse-grid minimax rate under suitable score regularity. Therefore, the reviewer thinks that the title and abstract probably need to be slightly modified in order to reflect such distinction.
Secondly, the reviewer thinks that the empirical evidence only supports the high-dimensional claim partially. Specifically, the controlled study verifies decreasing score error for all tested dimensions and shows an end-to-end TV slope that resembles the theoretical results in . However, the preregistered equal-smoothness isotropic comparison does not consistently favor the mixed target in just slightly higher dimensions. In and , the isotropic slopes are steeper. The matched-marginal comparison separates the targets, but it also substantially changes density concentration/envelope constants and therefore does not isolate interaction structure alone. Thus, I view the experiments as supportive but not yet strong evidence for the full statistical claim in high dimension.
Questions for authors
-
Given Proposition N.5(ii), would it be possible for the authors to clarify more explicitly what the diffusion model part contributes statistically beyond the direct density estimator based on the hyperbolic-cross approximation? My reading is that the assumption on mixed Besov smoothness provides the curse-breaking rate, while the main result specific to diffusion models is that score matching/learning based on sparse grids and reverse diffusion can attain the same rate.
-
How do the constants in Theorems 1 and 3 scale with respect to dimension? The polynomial exponent in the sample size is dimension-independent, but the paper allows dimension dependence through logarithmic powers and constants. Quantifying whether these constants are mild or exponential would help us reexamine the ``curse-breaking'' claim.
-
How should the higher-dimensional results in the preregistered isotropic comparison be interpreted, especially for or , where the isotropic slopes are steeper than the mixed slopes? Is there a possible comparison that keeps density concentration/envelope constants approximately fixed while changing only the interaction structure?
-
For the results, should the manuscript be slightly adjusted in a way that the conditional diffusion upper bounds are presented as a roadmap or a central contribution? Since the sharp unconditional diffusion rate remains open, I think the paper would benefit from making this distinction clearer.
3 comments
Response
The reviewer would like to thank the authors for the detailed response, which has addressed most of the concerns. Therefore, the reviewer is glad to keep the positive rating of 3.
Thank you for your thoughtful review and for taking the time to consider our response. We are glad that our clarifications addressed most of your concerns, and we appreciate your positive assessment and support.
Response to Reviewer brev
Thanks for highlighting the inverse-closure, score-learning and sampling analysis. We agree that the role of diffusion and the scope of the experiments needed clearer wording. Weakness numbers below follow your bullet order.
1. Diffusion's contribution, title and abstract (Weakness 1; Question 1)
Your reading of the diffusion contribution is right. Mixed smoothness supplies the statistical advantage; our contribution is to realize it through learned scores and reverse sampling. For sample size n, fixed dimension d and mixed smoothness s, write
rn=n−s/(2s+1)(logn)s(d−1)/(2s+1).
Direct density estimation attains this total-variation (TV) rate for every s>0. The original single-noise diffusion bound has an extra logn; new Theorem Q.2 removes it using repeated noising for s>1/2 on the full bounded positive class. We've rewritten the abstract and discussion accordingly. We propose “Diffusion Models Attain Sparse-Grid Rates under Mixed Smoothness” once title changes are permitted after discussion, as required by the author guidelines.
2. Dimension-dependent constants (Question 2)
On dimension dependence, we cannot claim mild constants in general. Our coarsest tensor space already contains 2d features, and the multiplier constants can also grow with d. Section 6 now makes the fixed-dimension scope explicit. Appendix S.6 gives an optimistic restricted result: for a known partition into B independent blocks of at most q coordinates, fitting and sampling each block separately gives expected TV error at most
C0B,n−s/(2s+1)(logn)s(q−1)/(2s+1).
Here s>1/2, q is fixed, and the global mixed-norm radius and positive density bounds stay fixed as d grows. Then C0 is independent of d,B,n. Unknown blocks and general cross-block dependence remain open.
3. Higher-dimensional slope ordering (Weakness 2; Question 3)
On the original experiments, we retain the steeper isotropic slopes at d=4,6 and explicitly acknowledge that they do not validate the high-dimensional minimax claim. A worst-case rate over a density class does not order finite-sample risks for individual targets. Target spectra, the noise window and selected model sizes can affect those curves; we have not established which explains the reversal. Section 7 and Appendix L now say this directly.
4. Matching concentration in the new control (Weakness 2; Question 3)
We also added the requested control in Appendix T. Two four-dimensional targets have pairwise versus four-coordinate density terms, but identical one-dimensional marginals, bounds [0.2,1.8], density-value distributions, radial spectra and heat-flow Fisher information. At n=262,144, over eight paired seeds with matched candidate feature budgets, the geometric mean of the per-seed four-coordinate/pairwise ratios of normalized integrated squared score error is 2.14 for hyperbolic ordering and 0.99 for Euclidean ordering. Descriptive 95% paired-bootstrap intervals are [2.09,2.18] and [0.97,1.01], respectively. This supports a representation-dependent effect within this controlled family. It uses single-noise score matching and does not test the repeated-noise refinement, generated-law TV or minimax rates.
5. Conditional Wasserstein results (Question 4)
Finally, we now present the refined Wasserstein-1 results as conditional transfers. The localized score-error, initialization and reverse-marginal density conditions still need verification for the learned sampler; an overall score-risk bound alone does not supply them. The corrected statements are in Appendices D and P. The abstract, Section 6 and conclusion explicitly leave a sharp unconditional diffusion Wasserstein-1 guarantee open.