acceptodds
Under review as a conference paper at ICLR 2027

An Optimization Framework for Denoising Generation

Abstract

Noise-unconditioned denoising learns a single score, so generation can be viewed as gradient ascent on a fixed log-density landscape. This optimization interpretation leaves two questions: do trajectories approach meaningful data, and do different initializations retain diverse outcomes? We establish both phenomena for empirical and population multi-scale Gaussian-mixture objectives. Under explicit posterior and landscape conditions, ascent converges almost surely within of an -dimensional reference manifold with reach at least . We derive a source–sink identity for the same fixed field and use it to prove finite-resolution regional coverage. In an affine construction, we further identify a positive-bandwidth regime where even empirical-score optimization can avoid training-sample memorization while preserving diversity at the training resolution. The same fixed objective naturally supports penalty-guided controllable generation. Experiments support our predicted mechanisms, show improved robustness under large-step guidance, and demonstrate downstream gains consistent with finite-data generative generalization.

Primary area:
generative models
Keywords:
denoising generation, optimization, diffusion models, controllable generation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_QQhgOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates noise-unconditioned denoising generation from an optimization perspective. It develops theoretical guarantees concerning manifold convergence, regional coverage, and non-memorization under specific geometric and probabilistic assumptions. The authors provide numerical studies using exact Gaussian-mixture score fields and additional experiments on learned generative models.

Critical strengths

The paper addresses meaningful theoretical questions regarding generative fidelity, diversity, and generalization. In particular, the source–sink analysis and regional coverage guarantees are potentially interesting mathematical contributions.

Critical weaknesses
  1. Insufficient empirical support for the main theoretical explanations.

My primary concern is the disconnect between the theoretical analysis and the behavior of learned generative models. The central results rely on assumptions about exact Gaussian-mixture score fields, and their numerical validation is largely restricted to synthetic distributions.

However, the paper aims to explain the generative behavior of neural denoisers. The experiments involving learned models mainly demonstrate generation performance and downstream benefits, without directly validating the key theoretical assumptions or predicted mechanisms.

Since the primary contribution is theoretical explanation rather than a new generative method, this missing connection substantially weakens the paper's main contribution.

  1. Limited conceptual novelty and restricted applicability of the theoretical results.

The connections between score-based generation, KDE, gradient-based mode seeking, and mean-shift dynamics have been extensively studied. Related interpretations have also appeared in earlier diffusion literature, including On the Trajectory Regularity of ODE-based Diffusion Sampling (ICML 2024).

Although the paper establishes additional theoretical results, these rely on restrictive assumptions. Regional coverage requires specific geometric, initialization, and stopping conditions, while the non-memorization guarantee is established for a special affine construction. Their broader explanatory significance remains insufficiently demonstrated.

Questions for authors

Q1. Can the authors directly validate the key theoretical assumptions and predicted mechanisms on learned neural score fields, even using low-dimensional synthetic datasets with known geometry?

Q2. Can the authors provide concrete numerical evaluations showing that the theoretical regional coverage bounds remain nontrivial under the experimental settings, including the relevant approximation and initialization errors?

5 comments
AuthorsOpenReview ↗

Response to Reviewer QQhg – Part I

We thank the reviewer for the valuable feedback and suggestions. We first clarify our contribution and scope in response to the weaknesses, then present the additional evidence requested in Q1–Q2. We hope that our responses help clarify our work and strengthen your confidence in our paper.


[W2] Building on prior optimization and diffusion analyses (Zhou et al., 2026 [1]; D. Chen et al., 2024 [2]), our contribution is to turn this viewpoint into a quantitative generative framework that explains both convergence and diversity. Convergence alone leaves open how probability is allocated among outputs. Our source–sink analysis links the law of random trajectory points to the positive-noise diffusion marginal pσmin⁡Qp_{\sigma_{\min}}^Q; controlling subsequent drift then gives explicit regional coverage bounds. Larger reach permits stronger smoothing over more nearby samples while controlling normal bias. In the affine construction, the same endpoints lie on the manifold, remain separated from training samples, and retain regional mass, making this generalization mechanism precise.

[W1] Our original guarantees concern the exact empirical and population target scores. Sec. 2.1 states this scope. Appendix A.2 identifies the empirical score as the exact DSM regression target and decomposes learned-to-population score error into neural approximation error and empirical-to-population discrepancy. It also gives an ascent bound showing that sufficiently accurate local approximation preserves ascent, while explicitly identifying an end-to-end theory connecting finite-data neural training, score approximation, and autonomous trajectories as future work.

Diffusion convergence analyses similarly derive sampling guarantees under score-error assumptions (H. Chen et al., 2023 [3]). Your suggestion adds an important test of whether our conditions and mechanisms hold on learned fields. Q1–Q2 provide these direct checks and quantify the corresponding coverage guarantees, including approximation and initialization errors.

Locations in the revised manuscript

  • W1. Sec. 4.1 summarizes the learned-score checks. Appendix A.9 states the sufficient error, ascent, and stopping conditions that connect these checks to the analysis.
  • W2. Sec. 1, Relation to prior work, positions the results relative to mean shift and time-dependent diffusion ODEs. The paragraph after Eq. (D.29) in Appendix D.8 explains the scope of affine non-memorization.
AuthorsOpenReview ↗

Response to Reviewer QQhg – Part II

Q1. Can the authors directly validate the key theoretical assumptions and predicted mechanisms on learned neural score fields, even using low-dimensional synthetic datasets with known geometry?

We test the predicted mechanisms directly on learned noise-unconditioned fields in known geometry. On the sphere, the exact targets satisfy Assumption A.10, and ordinary DSM raises observed tube entry from 7.97.9\\% to 99.999.9\\% as the learned score approaches the exact target. On the circle, we verify localization and contraction for the exact score and establish off-tube repulsion directly at the origin. After auxiliary training, uniform checks on the three frozen MLPs verify inward attraction, bounded tangential drift, and objective ascent before first entry, as well as a repelling off-tube equilibrium within the verified domain. The finite-scale DSM target gives both experiments a common reference for measuring neural approximation error.

For each reference measure QQ, that exact target is

p_{\mathrm{MN}}\propto\sum_i\sigma_i^2p_{\sigma_i}^Q,$$ where $p_{\sigma_i}^Q$ is $Q$ convolved with Gaussian noise. This is the target of $\sigma^2$-weighted noise-unconditioned DSM. The geometry also lets us track the two effects of approximation error $e_\theta=s_\theta-b$. Its outward normal component can weaken attraction, and its tangential component can move trajectories across region boundaries. The sphere experiment tests whether ordinary DSM recovers the predicted behavior from a finite sample. We draw 512 points on the unit sphere in $\mathbb R^3$ with density proportional to $e^{u_3}$, where the third coordinate $u_3$ biases the sample toward the north pole. We append nine zeros to embed the cloud in $\mathbb R^{12}$ and use its empirical measure as $Q$. Each MLP has three width-128 SiLU hidden layers and trains with 24 log-spaced levels in $[0.05,2]$. The inputs are the corrupted state $x$ and $\log(1+\\|x\\|)$. The exact sphere targets satisfy Assumption A.10 with $\alpha_d=0.5$ and $R_d=3 < \alpha_d d=6$. With $r_0=0.2$, a uniform bound on the finite-scale distance moment gives $\operatorname{tr}(C_x)/\beta(x)\ge10.27 > 6$ at every off-tube critical point. The component-score covariance at these points is supported in the three-dimensional subspace containing the centres, so $r_{\mathrm{eff}}(C_x)\le3$. **For each of three independent cloud/training seeds, the exact and learned sphere flows share the training cloud, the 24-level noise grid, and 512 initial states.** The starts come from the central 99.9% radial shell of $\mathcal N(0,2^2I_{12})$. Both flows use a positive exact-mixture precision normalization, which preserves their continuous flow paths. On independent noisy queries we measure relative score RMSE $\sqrt{\mathbb E\\|s_\theta-b\\|^2/\mathbb E\\|b\\|^2}$. Tube entry means reaching distance $0.03$ from the sphere. Regional TV compares projected first-entry frequencies with twelve equal-probability spherical bands and counts failures as a separate label. We report mean $\pm$ sample SD across seeds. | DSM updates | Relative score RMSE $\downarrow$ | Tube entry (%) $\uparrow$ | Regional TV $\downarrow$ | | --- | ---: | ---: | ---: | | 1k | $0.684\pm0.007$ | $7.9\pm3.1$ | $0.92\pm0.03$ | | 5k | $0.358\pm0.004$ | $38.3\pm2.8$ | $0.62\pm0.03$ | | 20k | $0.256\pm0.005$ | $99.9\pm0.2$ | $0.07\pm0.03$ | The drop in score error is accompanied by smaller sampled outward normal errors, consistent with improved tube entry. By 20k updates, the regional TV is close to the exact-flow value of $0.08\pm0.03$.
AuthorsOpenReview ↗

Response to Reviewer QQhg – Part II (continued)

To check the dynamics between sampled points as well, we use the unit circle, where radial motion governs entry and angular motion governs regional retention. We take QQ to be the uniform measure on the circle in R2\mathbb R^2 and use 64 log-spaced noise levels in [0.12,0.18][0.12,0.18].

The circle's exact score satisfies the original localization and contraction conditions. At each scale, the squared chord moment is 2(1−I1(t)/I0(t))≤t−1+t−22(1-I_1(t)/I_0(t))\le t^{-1}+t^{-2}, with t=∣x∣/σi2t=\\|x\\|/\sigma_i^2 and modified Bessel functions I0,I1I_0,I_1. Averaging over the precision posterior gives Cloc=2.86C_{\mathrm{loc}}=2.86 as a sufficient constant in Eq. (8) throughout the r0=0.18r_0=0.18 tube. Equation (9) gives κ0<0.258\kappa_0 < 0.258 and ϵ0<0.021\epsilon_0 < 0.021, so the chosen stopping radius rstop=0.05r_{\mathrm{stop}}=0.05 lies above the exact attraction floor ϵ0\epsilon_0.

To test whether the learned score preserves inward motion and small tangential drift up to that stopping distance, we fit three independent unconstrained 2→64→64→22\to64\to64\to2 tanh MLPs with 4,482 parameters and Cartesian inputs and outputs. We follow 5k DSM updates with auxiliary training against the known score and its spatial derivatives to tighten the verification margins. Generation uses only the frozen MLP through Vθ=sθ/50V_\theta=s_\theta/50, a constant rescaling of time. We retain the DSM-only checkpoints as controls.

For each frozen MLP, we verify the inward and tangential bounds needed for first-entry coverage at every angle in 0.25≤∣x∣≤2.50.25\le\\|x\\|\le2.5 outside the distance-0.050.05 stopping tube. Arithmetic enclosures of the stored weights give

\\|P_TV_\theta\\|=\\|P_Te_\theta\\|/50\le4.36\times10^{-3},$$ Here $d$ is distance to the circle and $P_T$ the tangent projection. The inward bound ensures entry from the verified annulus. It also establishes ascent of the exact target log-density because the exact score is radial and points in the same direction. During this approach, the tangential bound limits the change in angle that can affect regional coverage. The annular bounds describe trajectories approaching the circle. At the origin, the exact circle target's only off-tube critical point, rotational symmetry gives $r_{\mathrm{eff}}(C_0)=2=d$. Assumption A.10 requires $r_{\mathrm{eff}}\le R_d < \alpha_d d\le d$, so **we establish repulsion directly from the exact Hessian, whose eigenvalues are at least $14.43\\,\beta(0)$**. We then check the same instability mechanism on the neural fields. Combining interval checks on the central disk and the annuli gives **exactly one repelling off-tube equilibrium per frozen MLP within $\\|x\\|\le2.5$**. Each lies within $6.4\times10^{-4}$ of the origin, and the eigenvalues of $DV_\theta$ there have real parts at least $4.37$. From 16,384 shared $\mathcal N(0,0.8^2I_2)$ starts, each circle MLP achieves 100% observed tube entry and a TV of about $0.01$ over twelve equal-angle regions. **Q2 uses these same three frozen MLPs and the same distance-$0.05$ first-entry rule.** The verified inward and tangential bounds determine how much initial mass must stay in its starting region up to entry. **Locations in the revised manuscript** - **Q1.** Appendix F.2.1 (Table 7) gives the exact sphere check and the paired ordinary-DSM comparison. Appendix F.5 details the exact and learned circle checks and the auxiliary supervision.
Reviewer_MxdKOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies noise-unconditioned denoising as gradient ascent on a fixed multi-scale density. Under explicit geometric and landscape assumptions, it establishes convergence near a reference manifold and finite-resolution regional coverage for a stopped sampler. It also constructs an affine example in which empirical-score optimization produces diverse endpoints separated from the training samples, and develops a penalty-based formulation for controllable generation. The validation includes exact-score experiments, guided sampling on synthetic data and CIFAR-10, and downstream augmentation under class imbalance.

Critical strengths

The motivation is reasonable. Convergence toward high-density regions alone does not explain whether optimization retains diverse outcomes or simply returns training samples. The manuscript addresses these questions separately through manifold convergence, regional probability coverage, and a constructive non-memorization result. The source–sink analysis is particularly useful because it connects the fixed score field to probability allocation across regions.

Critical weaknesses
  1. The connection between the theoretical guarantees and the evaluated samplers remains incomplete. For E1’s 24 scales, the residual in Eq. (16) is large enough that Theorem 3.1 yields negative lower bounds for the reported vMF bins, even before other error terms are included. Thus, while the observed coverage is useful, the theorem does not quantitatively explain it at this resolution. The guarantees also assume exact score fields, whereas the experiments use learned scores. The paper does not establish how score approximation affects convergence or coverage, nor how the theoretical stopping rule relates to the fixed sampling budgets used in the image experiments. These gaps limit the theory’s ability to explain the implemented generator.

  2. The practical experiments support useful behavior but provide limited evidence for the proposed robustness and non-memorization explanations. Figure 3 shows a visual advantage at 100 steps, and the unguided control in Figure 12 suggests that guidance contributes to degradation. However, the evaluation covers only two sampling budgets under one guidance schedule and lacks quantitative measures of target satisfaction, quality, and diversity. The guidance strategy also differs from the fixed-penalty objective analyzed theoretically. The downstream comparison is meaningful: Table 3 reports a 5.58 percentage-point improvement in balanced accuracy over diffusion augmentation under matched training conditions. However, the non-memorization guarantee concerns a constructed affine lattice, with no direct measurements linking it to the learned image generators. Since variability is assessed across classifier seeds with the real-data subset fixed, the results demonstrate an augmentation benefit but leave its consistency and relationship to non-memorization unresolved.

Questions for authors
  1. For E1, can the authors verify the residual calculation above and provide a configuration in which the coverage bound is positive at the evaluated partition resolution? Please report the contributions of the Poisson residual, initialization error, and geometric terms. This would clarify whether the coverage theorem gives an informative numerical guarantee in a demonstrated setting.

  2. Which convergence and coverage conclusions are intended to apply to the learned samplers? Please clarify what control on score approximation would be sufficient, and whether a practical stopping rule can replace the manifold-distance criterion. A controlled comparison of exact and learned scores would help assess the resulting loss of objective monotonicity and coverage.

  3. Does the guidance advantage persist across several step budgets and guidance strengths when both models receive comparable tuning? Please provide quantitative target success or constraint error together with sample quality and diversity. A direct measurement of state–noise mismatch in the synthetic setting would help assess the proposed explanation.

  4. Do the augmentation gains persist across independently selected training subsets and generator seeds? Please also report duplication and coverage measurements for the generated samples. These results would help determine whether the downstream improvement is accompanied by the non-memorizing behavior discussed in the paper.

10 comments
AuthorsOpenReview ↗

Response to Reviewer MxdK – Part I

We thank the reviewer for the careful assessment and constructive suggestions. We address the first weakness in Q1–Q2 through complete coverage budgets, learned-score analysis, and stopping conditions, and the second in Q3–Q4 through broader guidance comparisons and replicated augmentation, memorization, and coverage measurements. We hope that our responses help clarify our work and strengthen your confidence in our paper.


Q1/W1. For E1, can the authors verify the residual calculation above and provide a configuration in which the coverage bound is positive at the evaluated partition resolution? Please report the contributions of the Poisson residual, initialization error, and geometric terms. This would clarify whether the coverage theorem gives an informative numerical guarantee in a demonstrated setting.

Your residual calculation is correct. The original E1 residual alone exceeds the mass of a region. On the uniform sphere, refining the noise grid and accounting for the full error budget gives a positive bound at the same twelve-region resolution. With 1,024 and 16,384 levels, the population version of Theorem 3.1 (Corollary I.2) guarantees at least 2.732.73\\% and 8.048.04\\% per region, respectively, out of a reference mass of 8.338.33\\%.

The E1 centres lie in R3×09\mathbb R^3\times\\{0\\}^9, so the nine appended Gaussian coordinates give a residual check independent of the tangential vMF distribution. For the original 24-level grid on [0.02,2][0.02,2], the residual integral over ∣x⊥∣≤0.058\\|x_\perp\\|\le0.058 is about −0.168-0.168. Since RNR_N has zero total mass,

ϵPois=12∣RN∣1≥0.168>1/12.\epsilon_{\mathrm{Pois}}=\tfrac12\\|R_N\\|_1\ge0.168 > 1/12.

Thus the bound is vacuous even before initialization and geometric losses enter. As described in Sec. 4.1 and Appendix F.1, the original E1 was designed to test the mechanism in a finite example.

We reduce the losses with finer log-spaced grids and sharper residual and initialization estimates, keeping the finite-mixture weighting rule and inclusive-grid endpoint correction. Both new configurations retain the same twelve-region resolution as E1, with reference mass 1/121/12 per region. The uniform sphere S2⊂R12S^2\subset\mathbb R^{12} lets us evaluate the geometric losses explicitly.

Both configurations use tube radius r0=0.05r_0=0.05. With 1,024 levels, the cutoffs are [10−3,10][10^{-3},10], transition/stop radii are (0.006,0.0015)(0.006,0.0015), and the placement parameter is ζ=0.1\zeta=0.1. With 16,384 levels, these are [10−5,50][10^{-5},50], (8×10−5,5×10−6)(8\times10^{-5},5\times10^{-6}), and ζ=0.001\zeta=0.001. Both pass the localization, contraction, off-tube instability, and placement checks. The initialization errors include the Gaussian masses omitted by conditioning on the verified shells, 0.0010.001 and 10−610^{-6}, respectively.

The geometric terms account for placement into a region and subsequent drift before stopping. Eroding each region by δcov\delta_{\mathrm{cov}} allows for projection of the fine-noise perturbation and tangential drift, leaving at least p\*=min⁡jQ(Uj−δcov)p_\*=\min_j Q(U_j^{-\delta_{\mathrm{cov}}}) reference mass. Here ZtrZ_{\mathrm{tr}} and ZstopZ_{\mathrm{stop}} are the manifold projections at transition and first entry. Including the probability errors gives

\min_j\mathbb P\\{Z_{\mathrm{tr}}\in U_j,\\ Z_{\mathrm{stop}}\in U_j\\} &\ge\frac1{12}-L_{\mathrm{erosion}}-L_{\mathrm{placement}}\\\\ &\quad-\epsilon_{\mathrm{Pois}}-\epsilon_{\mathrm{init}}, \end{aligned}$$ where $L_{\mathrm{erosion}}=1/12-p_\*$ is the mass in the boundary strip of the worst region and $L_{\mathrm{placement}}=\zeta p_\*$ is the placement loss. | Noise levels | Poisson error (%) | Initialization error (%) | Boundary erosion (%) | Placement loss (%) | Regional lower bound (%) | | ---: | ---: | ---: | ---: | ---: | ---: | | 1,024 | 3.517 | 0.305 | 1.051 | 0.729 | 2.73 | | 16,384 | 0.255 | 0.009 | 0.017 | 0.009 | 8.04 | The last column bounds the probability that both the transition and first-entry points lie in a given region, so it also lower-bounds the returned mass in that region. **Locations in the revised manuscript** - **Q1 (W1).** Sec. 4.1 (E1) and Appendix F.1.1 give the 24-level residual and both population-sphere budgets (Table 5).
AuthorsOpenReview ↗

Response to Reviewer MxdK – Part II

Q2/W1. Which convergence and coverage conclusions are intended to apply to the learned samplers? Please clarify what control on score approximation would be sufficient, and whether a practical stopping rule can replace the manifold-distance criterion. A controlled comparison of exact and learned scores would help assess the resulting loss of objective monotonicity and coverage.

For learned scores, we give sufficient conditions for attraction to a stopping tube and coverage of first-entry outputs. The outward error sets a lower limit on the stopping radius and determines the accuracy needed for a distance-based stopping surrogate. Tangential error can move trajectories across region boundaries. We compare learned and exact flows on a sphere, then check the learned dynamics uniformly on a circle. The three circle MLPs guarantee at least 7.187.18\\% first-entry mass in each of twelve regions in continuous time. Writing the learned field as a perturbation of the exact score gives a common way to quantify these effects.

Following Sec. 2.1 and Appendix A.2, write the exact DSM target as b=∇log⁡pMNb=\nabla\log p_{\mathrm{MN}} and the neural score as sθ=b+es_\theta=b+e. With outward unit normal nn, tangent projection PTP_T, and posterior precision β(x)\beta(x), the two trajectory-level controls within the tube are

\qquad \\|P_Te\\|\le\beta\delta_T.$$ The normalized outward and tangential error bounds are $\delta_N$ and $\delta_T$. The normal error raises the exact-field attraction floor $\epsilon_0$ to $$\epsilon_e=\max\\!\left\\{\epsilon_0+\delta_N, \frac{\delta_N}{1-\kappa_0}\right\\},$$ where $\kappa_0 < 1$ is the paper's contraction constant. **Choosing $\epsilon_e < r_{\mathrm{stop}} < r_0$ preserves inward attraction within the analysis tube.** The tangential bound controls how far a trajectory can move along the manifold before stopping. Appendix A.9 adapts the calculation of Proposition B.2 by adding $\delta_T$ inside the projected-speed bound and using the perturbed inward speed to compute boundary erosion. These geometric controls limit drift along individual trajectories. The original source–sink proof also compares the probability mass transported by the whole flow, so a score perturbation adds the divergence of $A_Np_{\mathrm{MN}}e$ to its residual. The resulting probability loss is bounded by $$\epsilon_{\mathrm{flux}}=\frac{A_N}{2} \mathbb E_{p_{\mathrm{MN}}}\left|\operatorname{div}e+b^\top e\right|,$$ with the paper's normalization $A_N$. This extension requires a complete $C^1$ flow, bounded forward trajectories, and an integrable added divergence with zero total mass. Along a learned trajectory, the score error changes the objective derivative through $b^\top e$. A pointwise bound relative to $\\|b\\|$ preserves ascent of the exact objective. $$\begin{gathered} \frac{d}{dt}\log p_{\mathrm{MN}}(x(t)) =\\|b\\|^2+b^\top e\ge(1-\eta)\\|b\\|^2,\\\\ \\|e\\|\le\eta\\|b\\|,\qquad \eta < 1. \end{gathered}$$ The guarantees above concern attraction and coverage up to first entry. Point convergence of a learned field would require further assumptions on its dynamics. The inward margin also tells us how accurate a practical stopping rule must be. **If a continuous distance estimate satisfies $|\widehat r-r|\le\delta_r$, its first downcrossing of $r_{\mathrm{stop}}-\delta_r$ puts the true distance in $[r_{\mathrm{stop}}-2\delta_r,r_{\mathrm{stop}}]$.** To keep this entire interval above the attraction floor, require $r_{\mathrm{stop}}-2\delta_r > \epsilon_e$. Discrete checking also requires a bound on overshoot. The self-correction mechanism of Zhou et al. (2026, Appendix A.7.2) [1] motivates a rule based on the reverse VE-SDE schedule. In their single-centre analysis, the squared residual per coordinate $\\|x_n-\mu\\|^2/d$ increases in conditional expectation when it is below the step's injected variance $\sigma_n^2-\sigma_{n+1}^2$. On a geometric grid this variance shrinks in proportion to $\sigma_n^2$, so the schedule provides a guide to the resolution of late samples. **A practical rule is to choose a positive terminal noise level on validation data and return the SDE output when the schedule reaches it.** Certifying geometric first entry still requires the distance-error and overshoot bounds above. Our fixed-budget image experiments assess performance empirically.
AuthorsOpenReview ↗

Response to Reviewer MxdK – Part II (continued)

The controlled sphere comparison pairs exact and learned flows on the same point cloud, 24-level noise grid, and initial states. Both flows use the same exact precision normalization, which only changes the continuous-time parametrization. We draw 512 unit-sphere points with density proportional to eu3e^{u_3}, where u3u_3 is the third coordinate, embed them in R12\mathbb R^{12}, and train noise-unconditioned MLPs with ordinary DSM over [0.05,2][0.05,2]. We use the distance-0.030.03 stopping tube and failure-inclusive twelve-region TV for both flows. Across three independent cloud/training seeds, training from 1k to 20k updates reduces relative score RMSE from 0.680.68 to 0.260.26. As the score improves, observed entry rises from 7.97.9\\% to 99.999.9\\% and regional TV drops from 0.920.92 to 0.070.07, close to the exact-flow value of 0.080.08.

The sphere comparison checks the effect on sampled trajectories. On the unit circle, the radial exact score lets us verify ascent and bound angular drift throughout the region of interest. We independently fit three unconstrained 2→64→64→22\to64\to64\to2 tanh MLPs with 4,482 parameters and 64 noise levels in [0.12,0.18][0.12,0.18]. Auxiliary training against the exact score and its derivatives after DSM tightens the verification margins. At generation, we run only Vθ=sθ/50V_\theta=s_\theta/50.

Using arithmetic enclosures of the stored weights, we verify for each network at every angle in 0.25≤∣x∣≤2.50.25\le\\|x\\|\le2.5 outside the distance-0.050.05 stopping tube that

\qquad \\|P_TV_\theta\\|\le4.36\times10^{-3},$$ where $d$ is distance to the circle. The learned radial motion has the same sign as the exact score, so **the learned flow increases the exact log-density before entry in this annulus.** The tangential bound limits how far the angle can drift before stopping. **For these same three frozen MLPs and the same distance-$0.05$ first-entry rule, we bound coverage in twelve regions under the full $\mathcal N(0,0.8^2I_2)$ prior.** The bound counts Gaussian starts that must stay in their initial region. | Seed | Continuous lower (%) | Euler lower (%) | Observed minimum (%, RK4) | | --- | ---: | ---: | ---: | | 0 | 7.21 | 7.17 | 7.98 | | 1 | 7.28 | 7.24 | 7.92 | | 2 | 7.18 | 7.14 | 7.94 | Each reference bin has mass $1/12\approx8.33\\%$. The bounds use unrounded network constants for angular drift. At least $94.47\\%$ of the prior lies in the verified annulus. The remaining starts stay in the experiment and contribute zero mass to the lower bound. In 16,384 sampled starts per model, all three achieve 100% observed entry. The Euler bounds account for the additional interpolation defect at $h=10^{-4}$ and use the first intersection of the real-arithmetic piecewise-linear path with the tube. Certified starts enter by time 10. The separate RK4 runs allow time 20 and return the first hit. Bounds are rounded downward and observations to the nearest displayed value. The DSM-only checkpoints show how much coverage we can certify with ordinary training. Seeds 0 and 1 give weaker positive bounds of $0.78\\%$ and $0.80\\%$. Seed 2 reaches the tube in sampled runs and fails the sufficient normal-margin check. Auxiliary training strengthens the verified margins that enter the coverage bound. **Locations in the revised manuscript** - **Q2 (W1).** Appendix A.9 gives the learned-score perturbation, ascent, and stopping conditions. Sec. 4.1, Appendix F.2.1 (Table 7), and Appendix F.5 (Table 13) report the paired sphere tests and the verified circle first-entry results.
Reviewer_Fh3dOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper views noise-unconditioned denoising generation as gradient ascent on one fixed objective: σ2\sigma^2-weighted noise-unconditioned DSM learns the score of a unified multi-scale Gaussian mixture pMNp_{MN} (Zhou et al., 2026). Its contributions are:

  1. Main result: Under posterior-localization and off-tube instability conditions, the exact ascent flow converges almost surely to within ε0=Clocmσmin⁡2/(2τ)\varepsilon_0=C_{\mathrm{loc}}m\sigma_{\min}^2/(2\tau) of an mm-dimensional reference manifold of reach τ\tau, for empirical and population objectives. The rate is essentially sharp on the sphere.

  2. Diversity and non-memorization: A source-sink identity, ∇⋅(ANpMN∇log⁡pMN)=pσmax⁡Q−pσmin⁡Q+RN\nabla\cdot(A_Np_{MN}\nabla\log p_{MN})=p^Q_{\sigma_{\max}}-p^Q_{\sigma_{\min}}+R_N, gives finite-resolution regional coverage for stopped outputs. In an interlaced affine construction with positive bandwidth, exact empirical-score endpoints stay away from training points while preserving cell-level coverage. A penalty-guided extension covers controllable generation.

  3. Validation: Exact-score studies (E1-E4) on spheres and affine grids check retention, off-tube instability, the separation-coverage window and the σmin⁡2/τ\sigma_{\min}^2/\tau scaling. A compiled FID table, qualitative large-step guided sampling on a toy manifold and CIFAR-10, and minority-class augmentation from 100 cats (+5.58 balanced-accuracy points over diffusion) complete the evaluation.

Critical strengths
  1. Unified and careful theory: Fidelity, coverage and non-memorization are treated in one framework with explicit constants and clearly stated assumptions, and the authors are candid about scope. The sphere calculation shows that the σmin⁡2/τ\sigma_{\min}^2/\tau rate cannot be improved in general.

  2. The source-sink identity: With σ2\sigma^2 weights on a log-scale grid, ANΔpMNA_N\Delta p_{MN} telescopes exactly to pσmax⁡Q−pσmin⁡Qp^Q_{\sigma_{\max}}-p^Q_{\sigma_{\min}}. This explains why the standard DSM weighting matters, drives all the diversity results, and is of independent interest.

Critical weaknesses
  1. Incremental novelty and incomplete positioning: The optimization view and the unified density come from Zhou et al. (2026). The update is multi-bandwidth Gaussian mean shift, with classical convergence theory (Comaniciu & Meer, 2002; Carreira-Perpinan, 2007), and the O(σ2)O(\sigma^2) normal bias of smoothed manifold-supported densities is known from ridge estimation (Genovese et al., 2014). Related work on associative memory (Pham et al., 2025), score smoothing (Scarvelis et al., 2025), RED/gradient-step PnP (Romano et al., 2017; Hurault et al., 2022), and PFGM/Beckmann transport is not discussed, so it is hard to isolate what is new.

  2. Experiments do not match the theoretical setting: E3 uses s<1/2s<1/2, outside the regime of Corollary 3.2 (s0≥1/2s_0\ge1/2). E1 uses the benchmark Cloc=1.25C_{\mathrm{loc}}=1.25, while E4 measures ratios up to 2.372.37 under a different schedule. The guidance experiments use a time-varying λk\lambda_k with score-norm rescaling and are only qualitative, Tables 1-2 largely compile other papers' results, and the downstream study reports no memorization diagnostics.

Questions for authors
  1. Theorem 3.1 seems to hold verbatim with QM=Q^MQ_{\mathcal M}=\widehat Q_M, so a memorizing generator would satisfy it. Since the mark in Lemma B.4 lies on the same trajectory, does TV(L(x∞),(x∞)♯pσmin⁡Q)≤ϵPois+ϵinit\mathrm{TV}\big(\mathcal{L}(x_\infty),(x_\infty)_{\sharp}p^Q_{\sigma_{\min}}\big)\le\epsilon_{\mathrm{Pois}}+\epsilon_{\mathrm{init}} hold, making the exact-score generator equivalent in law to diffusion stopped at σmin⁡\sigma_{\min} plus refinement? How does this affect the contrast in Appendix D.8?

  2. In Corollary 3.2 the endpoints appear to be merged modes near the cell centres, an atomic law with roughly M/2mM/2^m atoms. In what sense is this non-memorization beyond avoiding exact training points?

  3. Is there a sufficient condition, e.g. on fill distance relative to σmin⁡\sigma_{\min}, under which the empirical ClocC_{\mathrm{loc}} stays O(1)O(1)? As a suggestion, nearest-neighbour distances of the generated cats and a baseline of diffusion stopped at σmin⁡\sigma_{\min} with a Tweedie readout would strengthen the downstream claim.

6 comments
AuthorsOpenReview ↗

Response to Reviewer Fh3d – Part I

We sincerely thank the reviewer for the careful reading and thoughtful feedback. Given the length and technical detail of the analysis, we especially appreciate the care taken to engage with our results. Below, we clarify the points raised by reviewer and report the additional theory and experiments prompted by your suggestions. We hope that our responses help clarify our work and strengthen your confidence in our paper.


W1. Novelty and positioning.

We agree that Zhou et al. (2026) [1] provide the unified density and optimization interpretation. Their analysis relates memorization to dominance by individual Gaussian components and to the separation of nearby states after one sampling step. Comaniciu and Meer (2002) [2] and Carreira-Perpiñán (2007) [3] analyze mean-shift convergence to KDE stationary points, and Genovese et al. (2014) [4] bound the Hausdorff distance between a density ridge at one smoothing scale and the underlying manifold.

Pham et al. (2025) [5] relate memorization and the emergence of new attractors to associative memory, and Scarvelis et al. (2025) [6] generate novel samples by explicitly smoothing the empirical score. RED and gradient-step PnP use denoisers in optimization objectives for inverse problems (Romano et al., 2017 and Hurault et al., 2022) [7, 8]. We build on the fixed-objective interpretation to give a quantitative account of when optimization works as a generator. Theorem 2.1 bounds the distance of the multiscale ascent endpoints to the manifold by Clocmσmin⁡2/(2τ)C_{\mathrm{loc}}m\sigma_{\min}^2/(2\tau) under the stated conditions. Larger reach permits stronger smoothing while controlling this normal bias.

Probability allocation is the other part of the generative question. Poisson flow constructs a Coulomb field in an augmented space, and Beckmann transport constructs autonomous flows that satisfy a flux constraint (Xu et al., 2022 and Lee et al., 2026) [9, 10]. For the given DSM field, the σ2\sigma^2 weights on the log-spaced grid yield a source–sink balance between the two diffusion cutoff marginals. The residual accounts for the finite grid and any difference between the empirical and reference laws. Together with the initialization error, this bounds the TV distance between the law of a random point on the trajectory and the σmin⁡\sigma_{\min} marginal. Controlling subsequent drift gives explicit regional probability lower bounds for the stopped outputs (Theorem 3.1). In the affine construction, the same on-manifold endpoints preserve regional probabilities and stay separated from the training set by a margin at the training resolution with high probability.

W2. Experimental settings and theoretical guarantees.

We chose the original exact-score settings to test the plausibility of the assumptions and the predicted mechanisms in finite examples (Sec. 4.1). As Appendix F.3 explains, E3 illustrates separation and coverage on a small grid at s<1/2s < 1/2, below the bandwidth floor in Corollary 3.2. For localization, E1's Cloc=1.25C_{\mathrm{loc}}=1.25 is a flat-limit benchmark for its drift calculation, and E4's 2.372.37 is a sampled maximum of the localization ratio under a separate noise schedule (Appendices F.1 and F.4). We agree that the quantitative guarantees should also be checked in settings that meet their sufficient conditions.

Q3 directly verifies the localization conditions on a finite affine grid with s≥1/2s\ge1/2. It compares training-set separation with cutoff plus Tweedie, checks the retained regional mass, and reports the requested nearest-neighbour distances for generated cats. The new exact-score sphere configuration also gives a positive coverage bound of 8.04% per region at E1's twelve-region resolution.

The annealed guidance, with score-norm matching on CIFAR-10, was a practical robustness test alongside the fixed-penalty analysis (Appendix H.4). The new four-disk VE-SDE sweep quantifies this comparison under shared tuning, with higher target success for noise unconditioning at all five matched main-stage budgets. Tables 1–2 now mark reproduced results with R and previously published results with P.

Locations in the revised manuscript

  • W1. Sec. 1, Relation to prior work, positions the contribution relative to mean shift, density ridges, score smoothing, transport, and denoising priors.
  • W2. Sec. 4.1 and Appendices F.1.1, F.3.1, and F.4.1 report the new E1, E3, and E4 checks. Sec. 4.2 labels the provenance of Tables 1–2. Secs. 4.3–4.4 and Appendices H.4.1 and H.5.1 report the guidance and image diagnostics.
AuthorsOpenReview ↗

Response to Reviewer Fh3d – Part II

Q1. Theorem 3.1 seems to hold verbatim with QM=Q^MQ_{\mathcal M}=\widehat Q_M, so a memorizing generator would satisfy it. Since the mark in Lemma B.4 lies on the same trajectory, does \operatorname{TV}\bigl(\mathcal L(x_\infty),(x_\infty)_\\#p^Q_{\sigma_{\min}}\bigr) \le\epsilon_{\mathrm{Pois}}+\epsilon_{\mathrm{init}} hold, making the exact-score generator equivalent in law to diffusion stopped at σmin⁡\sigma_{\min} plus refinement? How does this affect the contrast in Appendix D.8?

Theorem 3.1 can hold with QM=Q^MQ_{\mathcal M}=\widehat Q_M under its stated assumptions. It controls probability allocation at the chosen resolution. Corollary 3.2 adds a training-set separation margin.

Your endpoint implication is also correct. Let TT map an initial state to its limiting endpoint. Since the mark YY lies on the same trajectory, T(Y)=T(X0)T(Y)=T(X_0), and contraction of total variation under TT gives

\le\epsilon_{\mathrm{Pois}}+\epsilon_{\mathrm{init}}.$$ **The comparator runs exact diffusion for $Q_{\mathcal M}$ from its correct terminal noisy marginal to $\sigma_{\min} > 0$, then applies the same noise-unconditioned endpoint map.** The displayed bound compares the two endpoint laws in TV. They agree when the residual and initialization errors vanish. Interestingly, this hybrid retains the cross-scale smoothing discussed by Zhou et al. (2026) [1], which can reduce single-centre dominance. Our analysis quantifies regional coverage and, in the affine setting, separation from training samples. The zero-noise comparison in Appendix D.8 follows the exact empirical diffusion field to its limiting marginal $\widehat Q_M$. The revised appendix gives both comparisons explicitly. > **Q2. In Corollary 3.2 the endpoints appear to be merged modes near the cell centres, an atomic law with roughly $M/2^m$ atoms. In what sense is this non-memorization beyond avoiding exact training points?** **It's true that the endpoint law may be atomic for a fixed finite training set.** In our construction, however, **the claim goes beyond avoiding exact copies: the outputs are new points within the data support, with controlled regional probabilities.** Specifically, $$x_\infty\in\operatorname{conv}(S_M)\subset D_L\subset A,$$ where $D_L$ is the reference support in the affine manifold $A$. With probability at least $1-\epsilon$, the outputs remain at least $h_{\mathrm{cov},M}/4$ from the training set. Equation (22) also bounds the difference between the endpoint mass in any union of output cores and the reference mass of the corresponding cells by $\epsilon$. Appendix D.6 constructs a sequence on a fixed box with finer grids and $\epsilon\to0$ whose endpoint laws converge in $W_1$ to the continuous reference. **The affine assumption matters here because interpolation stays on the manifold.** It isolates the locally flat geometry relevant to large reach, which permits stronger smoothing while controlling normal bias under our localization assumptions. Related observations in trained diffusion models motivate this setting: S. Chen et al. (2024) [11] report local linearity and low-rank Jacobians of posterior-mean predictors, while Ventura et al. (2025) [12] examine score-Jacobian spectra and tangent/normal geometry. These findings motivate our affine construction as a tractable model of local denoising geometry. **Locations in the revised manuscript** - **Q1.** Appendix D.8, *Positive-cutoff diffusion followed by the same flow*, gives the endpoint TV bound in Eq. (D.29) and discusses coverage with an empirical reference. - **Q2.** The paragraph after Eq. (D.29) in Appendix D.8 explains the support and separation of the affine endpoints. Corollary 3.2 and Appendix D.6 give the cell-mass and $W_1$ results.
AuthorsOpenReview ↗

Response to Reviewer Fh3d – Part III

Q3. Is there a sufficient condition, e.g. on fill distance relative to σmin⁡\sigma_{\min}, under which the empirical ClocC_{\mathrm{loc}} stays O(1)O(1)? As a suggestion, nearest-neighbour distances of the generated cats and a baseline of diffusion stopped at σmin⁡\sigma_{\min} with a Tweedie readout would strengthen the downstream claim.

Yes. For a general manifold, a sufficient condition for empirical Cloc=O(1)C_{\mathrm{loc}}=O(1) combines a noise floor that covers the sampling gaps with regular sample mass across the effective smoothing window. The precise condition also controls curvature within the tube, mass at larger radii, and the contribution of the noise schedule. We verify it on an interlaced affine grid and use that grid for your cutoff-plus-Tweedie comparison. Our full generator retains coarse regional coverage there and gives 1818–2525\\% larger mean nearest-training distances. For the cat generators, the mean pixel-space distance to the nearest training image is 10.3310.33 for noise unconditioning and 6.626.62 for diffusion. The line grid makes the connection between sampling gaps and posterior spread explicit.

For intuition, consider an equally weighted line grid with spacing Δ\Delta. The largest nearest-sample distance occurs at a gap midpoint, so hfill=Δ/2h_{\mathrm{fill}}=\Delta/2. At bandwidths that cover this distance, sample counts grow roughly in proportion to the window radius until the window spans the grid. Together with Gaussian decay, this bounds the posterior mean squared distance from the projection by O(σi2)O(\sigma_i^2) at each level σi\sigma_i. The finite grid limits the relative weight of broad kernels, and Lemma E.3 controls the mixture's squared spread relative to the effective noise variance. The following sufficient conditions make this argument quantitative for finite clouds.

On a general mm-dimensional manifold, regular mass in balls plays the role of even spacing. For each projection zz of a point in the tube, require the empirical mass of the closed ball B‾(z,t)\overline B(z,t) to satisfy aztm≤Q^M(B‾(z,t))≤Kmassaztma_z t^m\le\widehat Q_M(\overline B(z,t))\le K_{\mathrm{mass}}a_z t^m for hreg≤t≤Rloch_{\mathrm{reg}}\le t\le R_{\mathrm{loc}}, with the upper bound also holding for larger balls. Here hreg≥hfillh_{\mathrm{reg}}\ge h_{\mathrm{fill}} is the radius at which regular counts begin, RlocR_{\mathrm{loc}} is how far the lower bound holds, az>0a_z > 0 allows spatially varying density, and Kmass=O(1)K_{\mathrm{mass}}=O(1) bounds the mass ratio.

The ratio of tube radius r0r_0 to reach τ\tau controls how curvature distorts the distances in the Gaussian weights. Suppose r0/τ≤1/(4m)r_0/\tau\le1/(4m) and c≥3c\ge3. For a finite geometric grid with log spacing hh, Lemma E.3 requires hc≤1/27h\sqrt c\le1/27, σmax⁡2≥8cσmin⁡2/(c−2)\sigma_{\max}^2\ge 8c\sigma_{\min}^2/(c-2), and r0≤σmax⁡(c−2)/2r_0\le\sigma_{\max}\sqrt{(c-2)/2}. With these conditions it suffices to choose

\qquad R_{\mathrm{loc}}\ge \sqrt{\frac{8C_{\mathrm{scale}}(m+2)}{1+r_0/\tau}}\\, \max\\{\sigma_{\min},r_0/\sqrt c\\},$$ where $C_{\mathrm{scale}}=[1+30(h\sqrt c+e^{-c/2})]\\,4c/(c-2)=O(1)$. The first inequality puts the noise floor above the scale at which counts become regular. The second keeps the effective smoothing window within that range. The lower mass bound supplies enough nearby weight, and the upper bound controls the contribution of distant points. Together these give **$C_{\mathrm{loc}}=O(K_{\mathrm{mass}}C_{\mathrm{scale}})=O(1)$**. **The interlaced-grid experiment checks the general condition with a fixed fill distance and three noise floors.** Our 82-centre grid in $\mathbb R^{51}$ has $m=1$, $c=50$, infinite reach, $a_z=1/82$, $h_{\mathrm{reg}}=h_{\mathrm{fill}}=1/3$, $K_{\mathrm{mass}}=8$, and $R_{\mathrm{loc}}=12$. With $\sigma_{\max}=50$ and $N=32{,}768$, the choices $\sigma_{\min}=0.50,0.55,0.60$ give $h_{\mathrm{fill}}/\sigma_{\min}=0.667,0.606,0.556$, respectively, and $C_{\mathrm{scale}} < 4.30$. For the tube radius $r_0\approx8.79\sigma_{\min}$, **all three floors exceed the required $\sqrt{2}/3\approx0.471$, and $R_{\mathrm{loc}}=12$ exceeds the largest required radius, which is below $7.57$.** A sharper interval calculation using the grid structure gives $C_{\mathrm{loc}} < 2.61$ uniformly over $z\in[-20.5,20.5]$ and normal distance $r\le r_0$. The corresponding sampled maxima are $1.242,1.215,1.204$. **The cutoff-plus-Tweedie comparison tests separation using the same training centres and the same positive noise floor for both methods.** Our full generator starts from $\mathcal N(0,\sigma_{\max}^2I_{51})$. The baseline draws from the exact diffusion marginal at each tested $\sigma_{\min}$ and applies the requested single-scale Tweedie (posterior-mean) readout. Each setting uses three sampling seeds and 1,024 outputs per seed and method.

Comments on the paper

3 comments
AuthorsOpenReview ↗

General Response

Dear Area Chair and Reviewers,

We sincerely thank you for your careful reading and constructive feedback. Your questions helped us clarify the scope of the theory and strengthen the experimental comparisons. Our point-by-point replies span several comments and include experimental details and derivations, so we summarize the main revisions and findings here for quick reference.

Major revisions

  • Regional coverage. We confirmed that the original E1 grid residual made the bound vacuous at the twelve-region resolution. With a finer population-sphere grid and a full accounting of errors, the lower bound reaches 8.04% per region, against 8.33% reference mass (Fh3d, MxdK, and QQhg, Table 5).

  • Learned samplers. We added sufficient conditions on score error and stopping and compared exact and learned flows on the same sphere problems. For three circle MLPs after auxiliary training, we verified per-region first-entry lower bounds of 7.18% in continuous time and 7.14% with Euler (MxdK and QQhg, Appendix A.9 and Tables 7 and 13).

  • Localization and the cutoff baseline. We gave a sufficient condition for empirical localization and verified it on a finite affine grid. At matched noise floors, ascent met the training-set distance margin 26.4–35.2 percentage points more often than the diffusion cutoff plus Tweedie while retaining coarse regional coverage (Fh3d, Appendices E.4 and F.4.1 and Table 8).

  • Guidance robustness. With the same tuning protocol, noise unconditioning achieved higher target success at all five main-stage budgets. We also report geometric error, target diversity, and measurements of state–noise mismatch (MxdK, Table 3 and Appendix H.4.1).

  • Augmentation and training-image proximity. Across three independently drawn 100-cat subsets with multiple generator seeds, noise unconditioning improved balanced accuracy by 5.07–5.64 percentage points over conditioned diffusion. We report nearest-training-image distances, fewer NN-ratio flags, and higher PRDC coverage (Fh3d and MxdK, Appendix H.5.1 and Tables 17–18).

Additional clarifications

We clarified the contribution relative to prior work and the scope of the affine non-memorization result. We also added a total-variation bound comparing direct ascent with positive-cutoff diffusion followed by the same flow. Tables 1–2 now identify reproduced and previously published results (Fh3d and QQhg, Sec. 1 and Appendix D.8).

We hope our responses adequately address the reviewers' concerns. We would be very happy to clarify any remaining questions and look forward to further discussion. Thanks again for your efforts!

Best regards,
The Authors

Dear Reviewer QQhg,

Thank you for the follow-up. We would like to clarify the following points.

The vacuous lower bound on E1’s 24-level grid is consistent with the original analysis. E1 was originally designed as an empirical diagnostic of regional allocation and refinement drift, testing the mechanisms behind our assumptions (Sec. 4.1 and Appendix F.1). The source–sink identity is exact under continuous log-scale mixing. Eq. (19) explicitly accounts for finite-grid error, and the original submission shows that the quadrature term vanishes under grid refinement at fixed cutoffs (Eq. B.36). Increasing the number of noise levels alone addresses this discretization issue, without requiring the "substantial changes to experimental settings" suggested in your follow-up. At E1’s original cutoffs, 1,024 levels bring the quadrature error bound down to 1.27 percentage points (revised Eq. F.3).

The additional parameter changes in the revised experiments align the settings with all the theorem’s conditions as suggested by other reviewers, extending the original assumption diagnostics to a full quantitative evaluation of the coverage bound. Revised Table 5 certifies 2.73% per region at 1,024 levels and 8.04% at 16,384 levels, against a reference mass of 8.33%. Both evaluations apply Corollary I.2 as stated and proved in the original submission.

Ordinary DSM training already provides strong practical evidence for the predicted mechanisms. On the sphere, relative score RMSE against the empirical target fell from 0.68 to 0.26 as training progressed. Observed tube entry increased from 7.9% to 99.9%, and regional TV fell from 0.92 to 0.07, close to 0.08 for the exact flow (Table 7). On the circle, all three DSM-only models achieved 100% observed tube entry over 16,384 trajectories per model. Their minimum empirical regional frequencies were 7.03%–7.67%, against the uniform reference of 8.33%, with regional TV of 0.021–0.029.

Our earlier response emphasized the certified lower bounds, which rely on uniform worst-case drift estimates and can substantially understate observed regional coverage. This may have given an overly pessimistic impression of ordinary DSM’s sampling performance. Two DSM-only circle checkpoints also yield positive certified coverage bounds. Auxiliary supervision on the exact score and its derivatives improves the uniform margins and produces stronger certificates (Appendix F.5). The ordinary-DSM results above already support the practical relevance of the theory.

Moreover, in our original submission, Theorems 2.1 and 3.1 establish manifold convergence and regional coverage for exact empirical scores under stated assumptions. Appendix I establishes the corresponding population guarantees, and the population mixture score is the exact regression target of population noise-unconditioned DSM. Appendix A.2 discusses this connection and identifies end-to-end analysis of neural approximation and trajectories as future work. The additional experiments provide practical evidence for that direction. We therefore disagree that they reconstruct the original theoretical justification or warrant a new round of review.

The attribution concern as stated refers to a separate paper. Smoothing the Score Function to Enhance Generalization in Diffusion Models is prior work by Zhou et al., which Sections 1 and 2.1 explicitly credit for the noise-unconditioned objective and its gradient-ascent interpretation. Section 1 of the revision also discusses connections to mean shift, density ridges, and generative transport. Our contribution establishes when fixed-objective ascent provides manifold fidelity, regional coverage, and diverse endpoints separated from training samples in the affine construction. So we think our literature review is well-established. If your concern applies to a result in our submission, please identify that result and the relevant prior work so that we can address it directly.

We appreciate your review and response. Please let us know if you have any further questions.

The Authors

AuthorsOpenReview ↗

Follow-up for the clarification

Dear Reviewer QQhg,

As the rebuttal period draws to a close, we would appreciate your consideration of our clarification above, as it directly addresses the main concerns underlying your assessment.

In particular, the vacuous bound on E1’s coarse grid is consistent with the discretization error explicitly accounted for in the original theory. Increasing the number of noise levels alone addresses this discretization issue. The additional parameter changes in the revised experiments align the settings with all the theorem’s conditions, allowing us to extend the original assumption diagnostics to a full quantitative evaluation of the coverage bound. Meanwhile, the original submission already establishes results for both exact empirical and population scores and discusses their connection to learned scores through the DSM objective. The requested experiments therefore provide practical evidence for a direction explicitly identified as future work in our original submission and should be viewed as supplementary evidence, rather than a reconstruction of the original submission.

Our earlier response emphasized the certified lower bounds, which rely on uniform worst-case estimates of tangential drift and can substantially understate actual regional coverage. This may have given an overly pessimistic impression of ordinary DSM’s sampling performance. On the circle, all three DSM-only models achieved 100% observed tube entry over 16,384 trajectories per model, with minimum empirical regional frequencies of 7.03%–7.67%, against the uniform reference of 8.33%, and regional TV of 0.021–0.029. These results already provide strong support for the predicted mechanisms under ordinary DSM training.

We believe these clarifications are important for evaluating the original theoretical contribution and the role of the supplementary experiments. We therefore kindly ask you to reconsider your assessment in light of the explanations and evidence provided.

Thank you for your time and consideration. We remain available to address any specific unresolved questions.

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.