- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper focuses on how perturbations introduced during diffusion sampling propagate through the remaining sampling trajectory and how it affects the final output. The main argument mentioned in the paper is that perturbations with the same path cost can still lead to substantially different output effects depending on when and how they are introduced. The authors propose a theoretical framework that decomposes path-level changes into output and conditional-trajectory changes and derives a first-order propagation analysis for understanding how local perturbations accumulate or cancel. They validate the framework using controlled equal-path-cost perturbations across JiT, ADM, and DiT, showing that output sensitivity varies across sampling stages and spatial frequencies. The paper further applies the propagation analysis to cache-based diffusion acceleration, where it is able to identify sampling windows associated with larger final-image errors.
Critical strengths
-
The paper provides a well motivated theoretical framework for explaining why perturbations with the same path cost can lead to different final output effects. The decomposition between path level and output level changes, together with the propagation analysis, provides a useful way to understand how perturbations can accumulate or cancel throughout the sampling trajectory.
-
The experimental validation is reasonably broad across JiT, ADM, and DiT, and shows that sensitivity to perturbations depends on both the sampling stage and spatial frequency. The cache-based acceleration experiment further shows a practical use of the framework, where the propagation-based predictor is able to identify caching windows associated with larger final-image errors.
Critical weaknesses
-
The propagation analysis is based on a local first-order approximation, while the practical experiments use finite perturbations. Although the analysis captures the general temporal sensitivity patterns, there are substantial differences between the predicted and observed response magnitudes in the DiT experiments. This limits how directly the local analysis can be used to predict the magnitude of finite output changes.
-
The practical application of the proposed propagation analysis is primarily demonstrated through cache-based acceleration, with only eight candidate caching windows per model. While the reported correlations are strong, broader evaluation across other practical perturbations or acceleration settings would provide stronger evidence that the proposed propagation analysis generalizes beyond the caching setup.
Questions for authors
The first-order propagation analysis appears to capture the temporal sensitivity patterns, but the finite perturbation experiments show substantial differences between predicted and observed response magnitudes, particularly for some DiT settings. Could the authors clarify how they expect the proposed analysis to behave as the perturbation magnitude increases, and whether there is a practical regime in which the first-order approximation can reliably predict both the location and magnitude of output sensitivity?
3 comments
Response to Reviewer QSrU (Part II)
W2: Experimental design and generalization beyond the caching setup
We use caching-induced errors to empirically validate the framework under perturbations arising from a practical sampling approximation, complementing the controlled interventions in Section 4. We have revised Section 5 to make this motivation explicit. The eight equal-length windows keep the reuse rule and number of reused predictions fixed while varying when the errors enter the sampling trajectory. This design lets us compare their downstream effects under a consistent caching protocol across four pretrained generators.
Within this setting, we examine both the observed output changes and the contributions that produce them. The strength sweep discussed above assesses finite-displacement accuracy; the contribution analysis in Appendix D.4 separates individual propagated magnitudes from cross-time interactions. We find that the summed cross-time contribution increases the mean squared first-order response in every evaluated model and window. This provides evidence for how errors from a practical sampling approximation propagate and accumulate, alongside the controlled temporal and directional comparisons in Section 4.
For other differentiable interventions, we specify their modified sampling updates, differentiate them to obtain the local defects, and propagate those defects through the reference dynamics and output map. Appendix C.7 makes this extension concrete by deriving the guidance-scale update defect and describing how to incorporate history-dependent feature caching. We leave systematic comparisons across additional caching methods, window layouts, and practical interventions to future work, using this formulation to examine how their different error structures affect propagation and finite-strength accuracy.
Revision locations: Section 5, pp. 9–10; Appendix C.7, p. 25; Appendix D.3, pp. 28–29; Appendix D.4, pp. 29–30.
Thank you to the authors for the detailed responses
About my concern with the first-order approximation, the additional experiments and results in Appendix D.3 help address prediction accuracy under finite perturbation strengths, although magnitude prediction still remains less reliable when it comes to larger strengths. The caching explanations and cross-time analysis strengthen the experimental evidence, but broader validation across other practical perturbations remains limited.
I would maintain my rating
Response to Reviewer QSrU (Part I)
Thank you for recognizing the theoretical framework and its validation across model families. To address your concern about applying the local theory to finite perturbations, we have expanded the analysis of how prediction accuracy changes with perturbation strength. We also clarify the purpose of the caching experiments and how other interventions enter the framework.
W1&Q1: Prediction accuracy as perturbation strength increases
As perturbation strength increases, higher-order terms can lead to differences between the first-order prediction and the observed output change. Our additional strength sweeps examine both the size of these differences and whether the predictions capture how output errors vary across sampling windows. Under the differentiability assumptions of Proposition 2, the output displacement satisfies Yα−Y0=αV+o(α). With bounded second derivatives, the displacement remainder is O(α2) near zero. The size of this remainder depends on the sampling dynamics and the intervention, so we assess prediction accuracy at specified strengths for each model.
In Appendix D.3, we vary the caching interpolation:
oiα=oi(Xiα)+α(oa(Xaα)−oi(Xiα)),
where a is the window's first step, α=0 gives fresh predictions, and α=1 gives direct caching. We pair classes and randomness across strengths and windows. We measure relative displacement error by ∥(Yα−Y0)−αV∥/∥Yα−Y0∥, taking the median over 64 samples and eight windows. This measures errors in both the direction and magnitude of the predicted image displacement. We separately evaluate the ordering of windows using the Spearman correlation between their sample-mean predicted and observed image MSEs. Representative results from Table 6 are:
At the tested strengths of 0.25 for JiT-B, 0.025 for ADM, and 0.01 for DiT, we obtain median relative displacement errors of approximately 11%–16%, together with high correlations between predicted and observed window errors. These settings provide evidence of both relatively accurate displacement predictions and agreement in which sampling windows produce larger errors. At full caching strength, ADM and DiT retain high ranking correlations despite median displacement errors exceeding 100%. Thus, predicting the ordering of window errors can remain accurate even when predicting the image displacement incurs large errors.
This distinction is consistent with the local analysis: the ordering of two response magnitudes is preserved when the sum of their prediction errors is smaller than the gap between the predicted magnitudes. We also find that the error need not decrease monotonically over the sampled strengths: for JiT-B/L, median errors are lower at 0.25 than at 0.1. In Appendix D.3, we additionally compare symmetric finite differences with the local predictions. Under sufficient smoothness, the symmetric difference cancels even-order terms in the output expansion.
The DiT experiment in Section 4.3 examines the complete propagation through latent sampling states and the nonlinear image decoder. For the controlled DCT interventions at ε=0.8, we recover the same peak-response windows for the local predictions and finite responses. The finite/local peak amplitude ratios are 36.8%, 47.7%, and 87.9% for DC, DCT-4, and DCT-31, respectively. These results show how the propagation analysis can capture the temporal locations of peak responses while overestimating their magnitudes at the tested strength.
The exact constructions in Appendix B further show why finite-strength accuracy depends on the structure of the intervention and dynamics. In the additive mean-control family of B.1, the output displacement equals εV at every strength. In the covariance family of B.2, higher-order effects appear as strength increases, and Figure 9 shows convergence to the local predictions as strength decreases. Together, these examples explain why the range of accurate first-order prediction varies across perturbation families.
Revision locations: Section 4.3, pp. 8–9, and Figure 5, p. 8; Appendices B.1–B.2, pp. 18–21, and Figure 9, p. 20; Appendix D.3, pp. 28–29, and Table 6, p. 29.