- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper provides statistical guarantees for Bregman score matching using mixed ReLU-ReQU neural networks, which extends classical quadratic score matching to more general convex generators. Under Sobolev-smooth score, the authors establish bounds for mean-squared score estimation error and excess Bregman risk, with an additional extension to unbounded scores for quadratic generators. Experiments compare neural architectures and existing score estimators, and investigate how different Bregman generators affect estimation errors.
Critical strengths
- Using mixed ReLU-ReQU networks, the paper extends the analysis of (1) DNN score matching beyond the quadratic loss to general Bregman losses, and (2) quadratic score matching by relaxing the boundedness requirement on the true score function.
- The experiments examine several aspects of the proposed approach, including comparisons with different activation architectures and the effects of Bregman generators on coordinate-specific and magnitude-specific estimation errors. This is valueble for a theory paper.
Critical weaknesses
-
I found the paper difficult to follow. For example, the main mathematical problem of score matching should be formally stated earlier in the paper. The function class studied in the paper is buried at the end of the notation section without sufficient emphasis. The introduction consists largely of verbal descriptions and technical terminology, without clearly presenting the mathematical questions. The intuition and explanations of the theoretical results are briefly stated but not developed in a clear, logical flow that helps readers understand the authors' reasoning. I suggest moving more experimental details to the appendix and devoting more space to explaining the mathematical ideas and helping readers follow the overall argument.
-
Mismatch between theory and experiments. Theorem 1 studies the estimator , where the score itself is represented by a mixed ReLU-ReQU network. However, Section 4.1 actually uses , where is constructed from the network output. For quadratic loss, , while for hypentropy, . While I understand that theory and experiments need not match exactly, I suggest adding experiments comparing different losses with the same score function class.
-
The theoretical results in this paper rely on the ReLU-ReQU network structure, which, together with its approximation error bounds, mainly stands from Yang et al. (2025). I would like the authors to further clarify the necessity and theoretical advantages of using this structure in their score matching set up, as well as the main mathematical difficulties and new contributions beyond Yang's work and other existing DNN score matching results.
Questions for authors
-
In the introduction comparing with Yakovlev et al. (2025), the paper emphasizes "without assuming low intrinsic dimension" as an advantage. However, from my understanding, an important theoretical advantage of DNN estimators over classical nonparametric estimators is their ability to overcome the curse of dimensionality under additional structural assumptions, such as low intrinsic dimension. Can the authors clarify whether their estimator can achieve such convergence rates under similar assumptions? If so, how can these rates be established? If not, what is the theoretical advantage of the proposed DNN estimator considered in this paper?
-
Please refer to the Weaknesses.