- Rating
- 2
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper analyzes iterative consistency model (CM) sampling through a perturbation analysis that bounds the Wasserstein error after repeated noising and denoising steps. The bounds separate contraction of the initialization error from accumulation of the model approximation error. Furthermore, the paper provides stability estimates from target distribution curvature, covering strongly log-concave distributions and a specific class of non-convex potentials. Gaussian examples concretely illustrate possible attainment of the error bounds.
Critical strengths
- Extremely clear and didactic presentation. As someone who does not work directly on this topic, I am certainly impressed by the authors' effort in explaining and interpreting their results, as well as providing the scope of their contributions. For example, the roles of the assumptions for exact-denoiser stability and learned-map stability are thoroughly discussed (Sections 3.1-3.3). The Gaussian examples very clearly illustrate the trade-off between contraction and new approximation error. I also appreciate the careful distinction between improving an upper bound error and improving the actual sampling error. In all, the nice writing makes the contribution very easy to understand and assess (at least at a high level).
Critical weaknesses
-
(Minor) Limited methodological novelty: The paper provides an error-propagation framework for repeated CM sampling. However, in my view, the contribution remains limited because the approach and its' ingredients are mostly standard, as they follow existing CM stability analysis. To be fair, the paper does acknowledge that it builds on the error recursion of Lyu et al, and mentions the two main high-level consequences of this recursion (Introduction, Remark 3.7). It is thus my perspective that the contribution in this paper mostly explores the estimates and conditions for which the recursion of Lyu et al applies. Subjectively, I think this is a bit modest (of course, the value of a paper does not need to depend primarily on the difficulty of its' mathematical argument).
-
(Major) Limited explanatory power of the results relative to the motivating question: As mentioned above, the motivating question of the paper is why additional sampling steps can improve or degrade sampling quality, and how this depends on the noise schedule (Introduction). To that end, the paper seems to take a robustness perspective, and treats the worst-case triangle bounds as a guideline for schedule design based on one-step error budgets and stability factors (Propositions 3.1, 3.9). In principle, this is not an invalid approach by itself, especially if we only have access to these quantities. However, it is my opinion that it is precisely the alignment and cancellation between inherited and new approximation errors that make this question (and the answer to it) interesting. It seems to me that the results the authors provide are still a bit far away from a meaningful first step towards understanding this question. Even in the authors' Gaussian analysis (Section 3.4), the actual Wasserstein distance, thus the tightness of the upper bound, behaves quite differently depending on whether the discrepancy lies in the mean or variance, and whether inherited and newly introduced variance errors superposition or interfere with one another. It is my sense that there can be information regarding the learned denoiser's error structure and its' relationship with the target distribution that is yet to be found. This is a tricky technical challenge for sure, but certainly important.
Moreover, on the practical aspect, the Gaussian example shows that minimizing the certificate upper bound can introduce an extra sampling step that actually increases sampling error (Appendix E.3, Figure 5). This is already acknowledged by the authors in Section 3.1. However, under the stated goal of the paper, I am a bit concerned that there is no no-harm guarantee with respect to sampling quality.
Lastly, as a minor detail, the comparison with Chen et al. is quantitatively correct but is not methodologically convincing. Chen et al. only propagates KL errors and has a lossy step of converting to a Wasserstein bound, but does not require extra assumptions used in this paper (Remark 3.8). Having superior numerical certificates with a more informative setting is not surprising in my view.
I want to clarify that the authors very attentively scope the contribution of this paper, and I have no issue regarding over-claiming. However, as stated I am not confident there is sufficient novelty or significance.
Questions for authors
I do not have questions. I can change my assessment based on the authors' rebuttal.
7 comments
W2: Error cancellation and explanatory power
(Major) Limited explanatory power of the results relative to the motivating question: [...] it is precisely the alignment and cancellation between inherited and new approximation errors that make this question (and the answer to it) interesting. [\ldots] Even in the authors' Gaussian analysis (Section 3.4), the actual Wasserstein distance, thus the tightness of the upper bound, behaves quite differently depending on whether the discrepancy lies in the mean or variance, and whether inherited and newly introduced variance errors superposition or interfere with one another.
We agree that cancellation can affect sampling error and make the bound conservative. However, it is not needed to explain why an additional step can help or harm. Our Gaussian example with errors in the mean only (§3.4) shows this directly: there the bound is exact for every schedule, yet along a decreasing schedule the error first decreases and then increases. The balance between reducing existing error and introducing new error is therefore sufficient to produce this behavior.
The same example shows that a sharper bound requires more information about the errors, including how they align and cancel. Our Gaussian analysis illustrates these effects, and the framework can accommodate them by tracking different sources of error separately.
We also examine cancellation empirically on SD 1.5. We observe partial cancellation at the first step, but little evidence of it at later steps along the three schedules tested. In particular, the steps that increase the error reinforce existing errors rather than cancel them. These observations support the relevance of the mechanism described by our analysis.
Changes in the revision. Section 3.4 (p. 8, ll. 408–415) clarifies what the bound captures and what additional information is needed to account for cancellation. Appendix F reports the empirical findings, and the conclusion identifies this refinement as future work (p. 10, ll. 524–526).
W2: No-harm guarantee for an extra step
Moreover, on the practical aspect, the Gaussian example shows that minimizing the certificate upper bound can introduce an extra sampling step that actually increases sampling error (Appendix E.3, Figure 5). This is already acknowledged by the authors in Section 3.1. However, under the stated goal of the paper, I am a bit concerned that there is no no-harm guarantee with respect to sampling quality.
No upper bound can guarantee that an extra step helps: that depends on the current error, which the sampler does not know (§3.1 notes that its statements compare certificates, not actual errors, and Fig. 5 shows a step that lowers the certificate but raises the error). An upper bound still gives insight into when a step can harm and by how much: a necessary condition for harm and a cap on it. By Proposition 3.9, a step at level takes the error to at most , a weighted average of and the plateau , so
a step can increase the error only if it is already below the plateau of that level, and never beyond it; the same holds with . Fig. 5 is an instance: after the first step the variance errors nearly cancel, the error is far below every plateau, and the harmful step raises it, but not above the plateau. Trained models behave the same way: along an equispaced SD 1.5 schedule, steps at high noise raise the error by 60%, below their plateaus (our reply to Reviewer nkjH, W3).
Changes in the revision. §3.1 (p. 4, ll. 204–211) states this bound on the harm of a step, after the comparison of certificates, and that whether a step harms must be checked empirically.
W1: Novelty relative to Lyu et al.
We thank the reviewer for a thoughtful report, and for recognising the care taken over the scope of the results.
The error recursion is indeed due to Lyu et al., as the introduction and Remark 3.7 state. Our contribution is to establish its ingredients under different assumptions, which can be verified in closed form for classes of targets and estimated for trained models, and most of the work lies there:
As the reviewer notes, the value of a result need not lie in the difficulty of its argument; here it lies in making each ingredient of the recursion explicit, so that for a given target and trained model one can assess empirically whether the assumptions are plausible, as we do for two latent consistency models.