- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper provides a novel direct TV bound for diffusion sampling with respect to discretization, using a mutual information quantity. The role of this quantity, as well as several applications of this result, were explored.
Critical strengths
-
Clear exposition: The paper identifies previous bounds on the sampling error with respect to discretization and successfully sets its' contribution apart. Several applications of their theoretical result were also discussed. The role that the mutual information term plays is illustrated very well in the comparison with intrinsic dimension and metric entropy.
-
Novel theoretical contribution: The paper's main result, Theorem 2, appears to be novel and eliminates the crudeness of several previous approaches in direct TV bounds. The surprising aspect to me is that the proof strategy is concise and perhaps clever.
Critical weaknesses
- Broader implications are somewhat limited. Although I acknowledge and support the paper's main theoretical contribution, I think that the broader consequences for diffusion models are limited. The paper touched on this aspect through the comparison between diffusions in latent space or pixel space. As the paper also identified, there can still be discretization advantages of using latent diffusion if the upper bounds are loose. I would like to clarify that this is the reason that I lean towards weak acceptance rather than clear acceptance (while still supporting the paper's contribution generally).
Questions for authors
I have no questions or concerns.
3 comments
3. Removing the resolution restriction: approximate Gaussian mixtures. To make the theory independent of the target resolution, we extend it to every distribution that can be approximated by an isotropic Gaussian mixture, the setting of Li, Cai and Wei (2026) (Theorem H.1; Corollary 8 applies it to a code and its decoder). The discretization term becomes , in any dimension, where is the mutual information between the mixture label and the data; enters only through the approximation term. This rate is better than the best previously known result (Li et al., 2026), whose discretization term is :
- Mutual information instead of the number of components: , and it is much smaller when components overlap or weights are unbalanced.
- Our worst case is the square root of theirs: against .
- Weaker assumptions: only a second moment , instead of a uniform bound on all component means.
- Infinite mixtures: countably many components are allowed, where any bound in is vacuous.
- Any geometric grid with and explicit constants, instead of a specific schedule with unspecified constants.
For more details, see Section H and Table 4.
We hope these results make the practical implications of our analysis concrete, and we are happy to answer any further questions.
Thank you so much for addressing my concerns. I will retain my positive score for acceptance.
We thank the reviewer for the positive assessment. Our main objective is to close the gap between theory and practice: existing discretization bounds scale with a dimension of the data, yet recent pixel-space models with suitable network design, such as SiD2 (Hoogeboom et al., 2025), JiT (Li & He, 2026) and PixNerd (Wang et al., 2026), approach the quality of latent diffusion in a space about 48 times larger. This suggests that sampling cost is governed by the structure of the data, not its dimension. Our bound makes this precise through one quantity, the mutual information IΓ. It recovers and improves the complexities of earlier analyses: the intrinsic dimension for data near a low-dimensional set (Theorem 3(c)), the metric entropy of the support plus half a nat (Corollary 3), and logKmix instead of log2(KmixN) for Gaussian mixtures (Corollary 6, Theorem H.1). It is strictly smaller whenever the data spread their mass unevenly.
We found the reviewer's point insightful: our comparison rests on upper bounds, and latent diffusion could keep a discretization advantage if they were loose. We address this with new experiments on real data and an extension of the theory.
1. New experiments on real data (MNIST and CIFAR-10). Total variation cannot be computed for real images, so we measure the discretization error with two quantities that our theory controls. The first is FID. The second is a coupled upper bound on W22: for data in a bounded set of diameter B, such as images, W22≤B2TV, so our total-variation bound also controls W22.
So the theory is now backed by experiments with trained networks.
2. A measurable complexity. Our bound depends on the data only through the mutual information IΓ, and this quantity can be measured. By the I–MMSE identity, IΓ=21∫0Γmmse(γ)dγ, and the denoising error of any trained model is never below the MMSE, so a pretrained denoiser gives an upper bound on IΓ. For ImageNet 256×256 (d=196,608), using a pretrained pixel-space model (ADM), we obtain IΓ≤13,600 nats at noise variance δ≈0.01 per pixel, about 15 times smaller than the dimension. For δ≥0.215, IΓ is below 2,480 nats, the entropy bound of the LlamaGen VQ-16 tokenizer's code (Section I, Table 6). We should emphasize that this estimation is only upper bound and true value can be much smaller.