- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper studies end to end sampling guarantees for conditional diffusion models when the denoising network is a finite deep ReLU neural network trained by a finite number of full-batch gradient descent steps. Unlike existing analyses that assume an empirical risk minimization oracle or rely on infinite-width/kernel limits, the paper establishes conditional total-variation guarantees for the actual finite neural network after finite training. Under conditional tail and regularity assumptions, the authors derive denoising regularity and approximation bounds from the conditional data distribution, establish population learning guarantees for the limiting kernel predictor, and then transfer these guarantees to the finite trained network.
Critical strengths
- A major strength is that the analysis moves beyond idealized empirical-risk minimization and infinite-width assumptions to the practically relevant setting of a finite deep ReLU network trained for finitely many gradient steps.
- The comparison among neural, finite-kernel, and limiting-kernel trajectories provides a coherent mechanism for transferring population/kernel guarantees to finite neural training. Explicitly accounting for kernel drift, nonlinear errors, and finite-width approximation over a finite training horizon is a useful theoretical framework.
- The paper obtains its guarantees without assuming a Polyak–Łojasiewicz condition or a positive lower bound on the empirical kernel spectrum. This is theoretically appealing because such conditions can be difficult to justify for finite neural networks.
Critical weaknesses
- Although the paper provides explicit dependencies on sample size, width, training duration, and sampling resources, the abstract does not make clear whether these bounds are quantitatively meaningful or potentially vacuous in realistic regimes. The interaction between network width and training horizon appears central to the result, but it is unclear whether the required width remains computationally feasible when the number of gradient steps is large.
- The guarantees rely on explicit conditional tail and regularity assumptions on the data law, as well as assumptions needed for the neural/kernel trajectory comparison. The paper should more clearly characterize which common conditional diffusion settings satisfy these assumptions and how restrictive the resulting smoothness and tail requirements are.
- A substantial part of the argument appears to proceed by obtaining guarantees for a limiting-kernel predictor and then transferring them to the finite network through trajectory comparisons. While this transfer is technically nontrivial, the paper should more clearly quantify what genuinely new statistical or sampling behavior is captured by the finite-network analysis compared with existing neural tangent/kernel-based guarantees. Otherwise, the contribution may be viewed primarily as a technical extension of kernel-regime analysis.
Questions for authors
- How large are the resulting width and sample-size requirements in representative parameter regimes? Can the authors provide a concrete scaling example showing whether the sufficient width remains feasible for a practically relevant training horizon?
- Can the authors give examples of commonly used conditional data distributions that satisfy conditional regularity and tail assumptions, as well as examples that do not?
- What is the precise improvement over existing kernel/NTK-based diffusion guarantees? Which part of the final bound fundamentally requires the finite-training, finite-width trajectory analysis, rather than following directly from an existing limiting-kernel result?
4 comments
Thank you for the rebuttal
Thank you for the rebuttal, which addresses several specific points, but my main concern regarding the precise improvement of Q3 remains unclear. I prefer to maintain my original score.
We thank the reviewer for the reply. Below we state the Q3 improvement more precisely.
(a) What a kernel guarantee controls, and what ours controls. A limiting-kernel (NTK) guarantee bounds the error of a kernel predictor: kernel ridge regression or kernel gradient flow. The sampler, however, runs with the trained network . Our decomposition (eq. 15) separates the two: A kernel analysis bounds only. is the gap between the finite network trained by GD and the kernel predictor at the same horizon. Prop. 5.1 bounds this gap by , where . Without this term, a kernel bound says nothing about the network that is actually trained and sampled from.
(b) Why this term needs a finite-width analysis.
- Growing horizon. Early stopping is the regularizer, so grows with the sample size. The network-to-kernel gap must therefore be controlled over a growing horizon, which forces (eq. 59). This horizon–width trade-off determines , and it is absent from limiting-kernel analyses.
- No eigenvalue lower bound. Existing finite-width diffusion results control the gap through a lower bound on the empirical Gram eigenvalue. For diffusion training, the inputs come from the time-atomic design with unbounded data, so such a bound is an extra assumption on the training design. We instead use the discrete Duhamel identity: gradient descent with does not amplify errors (), so the comparison needs no eigenvalue bound. This follows the regression analysis of Zhou et al. (2026).
- Deep, all-trained network. All layers are trained, including the readout, and the widths may differ (eq. 4). The relevant kernel is therefore the full deep NTK (Prop. E.3), not the two-layer kernel with a fixed readout. The bound is also proved for the time-atomic training law, using a capacity bound that holds for every input law (Prop. E.19).
Comparison with the closest neural diffusion guarantees.
| Han et al. | Strasman et al. (2026) | This work | |
|---|---|---|---|
| Setting | unconditional | unconditional | conditional |
| Network | two-layer, fixed readout | two-layer, first layer trained | deep, all layers incl. readout trained |
| Spectral assumption | empirical Gram eigenvalue lower bound | none (small eigenvalues of handled by spectral truncation) | none |
| Guarantee | weighted score error | sampling | averaged conditional TV after sampling |
Relative to Han et al., the improvement is that no Gram eigenvalue lower bound is needed. Strasman et al. also avoid this bound in the two-layer case; relative to them, the improvement is the extension to deep networks with all layers trained, and to conditional guarantees in TV.
In short, the analysis is in the kernel regime, so the contribution is not a faster rate or feature learning. The guarantee applies to the deep network actually trained by finite GD, without a Gram eigenvalue lower bound, and composes with conditional sampling in TV.
Dear Reviewer f19D,
Thank you for your reply. Our follow-up above separates what kernel-based guarantees control () from what our analysis adds ( for the trained deep network), and compares it with Han et al. and Strasman et al. If any part of Q3 is still unclear, we would be grateful if you could point to it, so we can clarify before the discussion closes.
We thank the reviewer for the detailed summary, and for noting that the analysis moves beyond ERM and infinite-width idealizations and avoids PL and Gram lower-bound conditions.
Weakness 1/Q1: How large are the width and sample requirements? Table 3 (App. I) gives the explicit scales at fixed t0. With s=(D+2)/2 and effective smoothness kˉ=min{k,s}:
Once k≥⌈s⌉, these become N=O~(ε−(4−2/(D+2))), ηR⋆=O~(ε−2) and m=O~(ε−32). The horizon and width exponents in ε then do not depend on D, because the capacity exponent cancels. For example, D=2 gives N=O~(ε−3.5) and m=O~(N64/7).
These widths are far beyond practical sizes, and we do not present them as guidance. Worst-case polynomial widths of this kind are standard in finite-width analyses of gradient training: m=Ω~(n24L12) (Allen-Zhu et al., 2019) and m=Ω~(n8L12) (Zou & Gu, 2019) for training-loss convergence alone; m≳n(6β+12)/(2β+γ), up to depth factors, for generalization in regression (Zhou et al., 2026, Cor. 6); and m≳poly(d)N26+4/(d+3) in the closest diffusion analysis (Han et al., Thm. 3.13). Several of these also require data-separation or Gram eigenvalue conditions. The revised abstract and Section 4 state this explicitly.
Weakness 2/Q2: Which distributions satisfy the assumptions, and how restrictive are they?
The revision moves these examples and non-examples into Section 3.
Weakness 3/Q3: Improvement over kernel-based guarantees. The closest NTK-based diffusion guarantee (Han et al.) is unconditional and two-layer with a fixed readout, assumes a Gram eigenvalue lower bound, and controls a weighted L2 score error rather than TV. A limiting-kernel result bounds only Eker, the error of a predictor that is never computed; the trained network also requires Eopt and Ewidth (Prop. 5.1, App. C.6), which produce the trade-off above. No existing limiting-kernel result applies directly to the time-atomic training law, so the Eker bound is also proved here (general response, item 3). The analysis is in the kernel regime, and we do not claim feature-learning effects.