- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates noise-unconditioned denoising generation from an optimization perspective. It develops theoretical guarantees concerning manifold convergence, regional coverage, and non-memorization under specific geometric and probabilistic assumptions. The authors provide numerical studies using exact Gaussian-mixture score fields and additional experiments on learned generative models.
Critical strengths
The paper addresses meaningful theoretical questions regarding generative fidelity, diversity, and generalization. In particular, the source–sink analysis and regional coverage guarantees are potentially interesting mathematical contributions.
Critical weaknesses
- Insufficient empirical support for the main theoretical explanations.
My primary concern is the disconnect between the theoretical analysis and the behavior of learned generative models. The central results rely on assumptions about exact Gaussian-mixture score fields, and their numerical validation is largely restricted to synthetic distributions.
However, the paper aims to explain the generative behavior of neural denoisers. The experiments involving learned models mainly demonstrate generation performance and downstream benefits, without directly validating the key theoretical assumptions or predicted mechanisms.
Since the primary contribution is theoretical explanation rather than a new generative method, this missing connection substantially weakens the paper's main contribution.
- Limited conceptual novelty and restricted applicability of the theoretical results.
The connections between score-based generation, KDE, gradient-based mode seeking, and mean-shift dynamics have been extensively studied. Related interpretations have also appeared in earlier diffusion literature, including On the Trajectory Regularity of ODE-based Diffusion Sampling (ICML 2024).
Although the paper establishes additional theoretical results, these rely on restrictive assumptions. Regional coverage requires specific geometric, initialization, and stopping conditions, while the non-memorization guarantee is established for a special affine construction. Their broader explanatory significance remains insufficiently demonstrated.
Questions for authors
Q1. Can the authors directly validate the key theoretical assumptions and predicted mechanisms on learned neural score fields, even using low-dimensional synthetic datasets with known geometry?
Q2. Can the authors provide concrete numerical evaluations showing that the theoretical regional coverage bounds remain nontrivial under the experimental settings, including the relevant approximation and initialization errors?
5 comments
Response to Reviewer QQhg – Part II
Q1. Can the authors directly validate the key theoretical assumptions and predicted mechanisms on learned neural score fields, even using low-dimensional synthetic datasets with known geometry?
We test the predicted mechanisms directly on learned noise-unconditioned fields in known geometry. On the sphere, the exact targets satisfy Assumption A.10, and ordinary DSM raises observed tube entry from to as the learned score approaches the exact target. On the circle, we verify localization and contraction for the exact score and establish off-tube repulsion directly at the origin. After auxiliary training, uniform checks on the three frozen MLPs verify inward attraction, bounded tangential drift, and objective ascent before first entry, as well as a repelling off-tube equilibrium within the verified domain. The finite-scale DSM target gives both experiments a common reference for measuring neural approximation error.
For each reference measure , that exact target is
p_{\mathrm{MN}}\propto\sum_i\sigma_i^2p_{\sigma_i}^Q,$$ where $p_{\sigma_i}^Q$ is $Q$ convolved with Gaussian noise. This is the target of $\sigma^2$-weighted noise-unconditioned DSM. The geometry also lets us track the two effects of approximation error $e_\theta=s_\theta-b$. Its outward normal component can weaken attraction, and its tangential component can move trajectories across region boundaries. The sphere experiment tests whether ordinary DSM recovers the predicted behavior from a finite sample. We draw 512 points on the unit sphere in $\mathbb R^3$ with density proportional to $e^{u_3}$, where the third coordinate $u_3$ biases the sample toward the north pole. We append nine zeros to embed the cloud in $\mathbb R^{12}$ and use its empirical measure as $Q$. Each MLP has three width-128 SiLU hidden layers and trains with 24 log-spaced levels in $[0.05,2]$. The inputs are the corrupted state $x$ and $\log(1+\\|x\\|)$. The exact sphere targets satisfy Assumption A.10 with $\alpha_d=0.5$ and $R_d=3 < \alpha_d d=6$. With $r_0=0.2$, a uniform bound on the finite-scale distance moment gives $\operatorname{tr}(C_x)/\beta(x)\ge10.27 > 6$ at every off-tube critical point. The component-score covariance at these points is supported in the three-dimensional subspace containing the centres, so $r_{\mathrm{eff}}(C_x)\le3$. **For each of three independent cloud/training seeds, the exact and learned sphere flows share the training cloud, the 24-level noise grid, and 512 initial states.** The starts come from the central 99.9% radial shell of $\mathcal N(0,2^2I_{12})$. Both flows use a positive exact-mixture precision normalization, which preserves their continuous flow paths. On independent noisy queries we measure relative score RMSE $\sqrt{\mathbb E\\|s_\theta-b\\|^2/\mathbb E\\|b\\|^2}$. Tube entry means reaching distance $0.03$ from the sphere. Regional TV compares projected first-entry frequencies with twelve equal-probability spherical bands and counts failures as a separate label. We report mean $\pm$ sample SD across seeds. | DSM updates | Relative score RMSE $\downarrow$ | Tube entry (%) $\uparrow$ | Regional TV $\downarrow$ | | --- | ---: | ---: | ---: | | 1k | $0.684\pm0.007$ | $7.9\pm3.1$ | $0.92\pm0.03$ | | 5k | $0.358\pm0.004$ | $38.3\pm2.8$ | $0.62\pm0.03$ | | 20k | $0.256\pm0.005$ | $99.9\pm0.2$ | $0.07\pm0.03$ | The drop in score error is accompanied by smaller sampled outward normal errors, consistent with improved tube entry. By 20k updates, the regional TV is close to the exact-flow value of $0.08\pm0.03$.Response to Reviewer QQhg – Part II (continued)
To check the dynamics between sampled points as well, we use the unit circle, where radial motion governs entry and angular motion governs regional retention. We take to be the uniform measure on the circle in and use 64 log-spaced noise levels in .
The circle's exact score satisfies the original localization and contraction conditions. At each scale, the squared chord moment is , with and modified Bessel functions . Averaging over the precision posterior gives as a sufficient constant in Eq. (8) throughout the tube. Equation (9) gives and , so the chosen stopping radius lies above the exact attraction floor .
To test whether the learned score preserves inward motion and small tangential drift up to that stopping distance, we fit three independent unconstrained tanh MLPs with 4,482 parameters and Cartesian inputs and outputs. We follow 5k DSM updates with auxiliary training against the known score and its spatial derivatives to tighten the verification margins. Generation uses only the frozen MLP through , a constant rescaling of time. We retain the DSM-only checkpoints as controls.
For each frozen MLP, we verify the inward and tangential bounds needed for first-entry coverage at every angle in outside the distance- stopping tube. Arithmetic enclosures of the stored weights give
\\|P_TV_\theta\\|=\\|P_Te_\theta\\|/50\le4.36\times10^{-3},$$ Here $d$ is distance to the circle and $P_T$ the tangent projection. The inward bound ensures entry from the verified annulus. It also establishes ascent of the exact target log-density because the exact score is radial and points in the same direction. During this approach, the tangential bound limits the change in angle that can affect regional coverage. The annular bounds describe trajectories approaching the circle. At the origin, the exact circle target's only off-tube critical point, rotational symmetry gives $r_{\mathrm{eff}}(C_0)=2=d$. Assumption A.10 requires $r_{\mathrm{eff}}\le R_d < \alpha_d d\le d$, so **we establish repulsion directly from the exact Hessian, whose eigenvalues are at least $14.43\\,\beta(0)$**. We then check the same instability mechanism on the neural fields. Combining interval checks on the central disk and the annuli gives **exactly one repelling off-tube equilibrium per frozen MLP within $\\|x\\|\le2.5$**. Each lies within $6.4\times10^{-4}$ of the origin, and the eigenvalues of $DV_\theta$ there have real parts at least $4.37$. From 16,384 shared $\mathcal N(0,0.8^2I_2)$ starts, each circle MLP achieves 100% observed tube entry and a TV of about $0.01$ over twelve equal-angle regions. **Q2 uses these same three frozen MLPs and the same distance-$0.05$ first-entry rule.** The verified inward and tangential bounds determine how much initial mass must stay in its starting region up to entry. **Locations in the revised manuscript** - **Q1.** Appendix F.2.1 (Table 7) gives the exact sphere check and the paired ordinary-DSM comparison. Appendix F.5 details the exact and learned circle checks and the auxiliary supervision.
Response to Reviewer QQhg – Part I
We thank the reviewer for the valuable feedback and suggestions. We first clarify our contribution and scope in response to the weaknesses, then present the additional evidence requested in Q1–Q2. We hope that our responses help clarify our work and strengthen your confidence in our paper.
[W2] Building on prior optimization and diffusion analyses (Zhou et al., 2026 [1]; D. Chen et al., 2024 [2]), our contribution is to turn this viewpoint into a quantitative generative framework that explains both convergence and diversity. Convergence alone leaves open how probability is allocated among outputs. Our source–sink analysis links the law of random trajectory points to the positive-noise diffusion marginal pσminQ; controlling subsequent drift then gives explicit regional coverage bounds. Larger reach permits stronger smoothing over more nearby samples while controlling normal bias. In the affine construction, the same endpoints lie on the manifold, remain separated from training samples, and retain regional mass, making this generalization mechanism precise.
[W1] Our original guarantees concern the exact empirical and population target scores. Sec. 2.1 states this scope. Appendix A.2 identifies the empirical score as the exact DSM regression target and decomposes learned-to-population score error into neural approximation error and empirical-to-population discrepancy. It also gives an ascent bound showing that sufficiently accurate local approximation preserves ascent, while explicitly identifying an end-to-end theory connecting finite-data neural training, score approximation, and autonomous trajectories as future work.
Diffusion convergence analyses similarly derive sampling guarantees under score-error assumptions (H. Chen et al., 2023 [3]). Your suggestion adds an important test of whether our conditions and mechanisms hold on learned fields. Q1–Q2 provide these direct checks and quantify the corresponding coverage guarantees, including approximation and initialization errors.
Locations in the revised manuscript