- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper studies the constant gap between the denoising and ideal score-matching losses. It shows this gap is a Fisher-metric trace, which for Gaussian noise equals the mutual information lost across the SNR range. It estimates the gap from a trained network. On CIFAR-10 the gap is up to 99.86% of the loss, and raw losses misrank models across noise ranges until it is subtracted.
Critical strengths
None. I learn nothing.
Critical weaknesses
First, the studied problem lacks significance. The floor is a model-independent constant with zero gradient, and thus it affects neither training nor sampling. Its only use, correcting cross-configuration loss comparisons, addresses a practice the paper does not show exists. The paper’s own recommendation of common-range evaluation already avoids that issue.
Second, the analysis offers nothing new. The constant follows from the equivalence of explicit and denoising score matching in Vincent (2011). The subsequent results apply the de Bruijn identity, the I–MMSE relation, and the schedule invariance already known from VDM. The Schrödinger-bridge and Fisher-geometry framing is not used by any result.
Third, the experiments yield no useful insight. They show that the constant is large and that subtracting it reorders a constructed cross-range comparison. The resulting excess is validated only against training steps, never against sample quality or likelihood.
Questions for authors
No more questions
Thank you for the review. We have withdrawn the 99.86% floor estimate and the claim that floor subtraction repairs rankings across different noise ranges. Below we address each concern and identify the changes in the revised manuscript.
1. Significance and the zero-gradient objection.
We agree: subtracting a model-independent constant changes neither training gradients nor the sampling dynamics of a fixed model. Ordinary DSM loss already ranks predictors on a common objective. Changing the noise interval or weighting changes both the floor and the ideal loss, so subtraction does not make different objectives comparable.
The revised practical question is how much of one predictor's expected denoising loss remains reducible. For network risk LA(f), Bayes floor F(A), and any reference predictor g using the available inputs,
F(A)≤LA(g),LA(f)−F(A)≥LA(f)−LA(g).
A better reference therefore certifies attainable improvement without estimating the floor accurately. This motivates the paired reference audit, rather than a correction to optimization or model ranking.
Manuscript: Introduction, pp. 1-2; Section 5, Eq. (5.2) and Table 1, pp. 7-9; limitations, p. 10.
2. Novelty, prior identities, and the role of the bridge.
We explicitly credit Vincent's DSM equivalence for the additive constant, de Bruijn for information flow, Guo-Shamai-Verdu for the Gaussian I-MMSE identity, and VDM for weighting invariance. We do not claim these ingredients as new.
The bridge motivates the construction: its first variation gives the generative drift, and the endpoint-tilt family's log-partition Hessian gives the endpoint covariance form. Conditioning the endpoints on the noisy state defines a distinct posterior family. The Markov property splits its Fisher form into past and future blocks; DSM measures the spatial trace of the clean-endpoint block. We distinguish the two families and remove unsupported claims of an isometry or a unique minimal reduction.
Our novelty claim is restricted to this conditional endpoint construction and its connection to DSM target variance. The empirical audit tests whether learned predictors recover that variance. We also credit VDM's discrete decoder and Fourier features; adding those features to the audited networks was not tested.
Manuscript: Sections 2-3, Proposition 3.1 and Theorem 3.2, pp. 2-5; Section 4 and Appendix C; Section 6, scope, p. 9.
3. Experimental evidence and downstream relevance.
We replaced the ranking-reversal interpretation and training-step proxy with paired held-out comparisons. On CIFAR-10 over log-SNR [13,13.8], the 240k-step network has risk 1095.0 +/- 1.9 nats/image. Hard rounding and the soft discrete reference reduce paired risk by 994.5 +/- 2.0 and 1017.9 +/- 1.9 nats/image, respectively. These comparisons measure improvements on an identical objective.
An independent audit assigns 1100.2 nats/image to the learned Jacobian estimate, while the pixel-lattice bound puts the true floor at most at 100.6. The Jacobian identity is exact for the Bayes predictor; a learned predictor can bias it. Agreement with the forward-only Stein expression checks the same plug-in quantity and does not remove this bias. The soft reference uses known pixel support without assuming the true image posterior or diagnosing a unique cause of the network's error.
A frozen class-conditional ImageNet-64 EDM denoiser provides a second dataset and architecture: the soft reference reduces risk by 6.9% on the tested band. Reported uncertainties are Monte Carlo standard errors. The CIFAR checkpoints share one training trajectory; sample quality, likelihood, and model selection by those criteria remain untested. Appendix E now explains the predictors, evaluation protocol, and uncertainty.
Manuscript: Sections 5.1-5.3, Eq. (5.1), Figure 1 and Table 1, pp. 7-9; Appendix E, pp. 25-27; limitations, p. 10. Appendix D.5 now uses the analytic range example only to explain changes in the objective.