- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes PEACache, a caching method for Diffusion Transformers. Unlike most prior methods that address caching error at a single level, it introduces a progressive error alignment framework with three levels. At the branch level, it predicts the missing residual change. At the classifier-free guidance (CFG) level, it supervises prediction on the CFG-composed output. At the trajectory level, it schedules cached steps via dynamic programming using an additive error surrogate. Experiments on three backbones show better fidelity than previous methods, except on VBench. However, there are concerns about the lack of theoretical justification and generalizability.
Critical strengths
- It presents a unified caching framework spanning the branch, CFG, and final-generation levels, unlike existing single-level approaches.
- It shows consistently better fidelity than previous methods across three backbones, with large margins under better acceleration.
Critical weaknesses
- Lack of theoretical justification: the method is justified only empirically, without mathematical analysis. In particular, the additive surrogate is validated only on two-step schedules, while the actual schedules cache many steps.
- Limited generalization evaluation: the unseen-prompt experiment reports only the lower-fidelity 12-NFE setting. It is unclear whether generalization holds in the higher-fidelity setting.
- Unreported overhead: the paper does not report the offline calibration cost or the per-step overhead of the eigendecomposition.
Questions for authors
- Does the 94.991 s latency at L424 include the offline calibration process on the small calibration set? If not, can the comparison with calibration-free methods be considered fair?
- In the unseen-prompt experiment, how well does the method generalize in the higher-fidelity setting (24 NFEs)?
2 comments
Dear authors,
Thank you for the detailed rebuttal. The additional results resolve my concern on W2/Q2. For W3/Q1, I accept reporting the online latency separately. However, the reported calibration cost (about 4 days for schedule calibration plus 5 hours for predictor fitting) is quite large. I think it should be compared with the calibration time of other methods that also rely on offline calibration. Regarding W1, the first-order argument gives useful intuition, but it does not guarantee that the ranking of schedules is preserved. Also, comparing only against random schedules is still a weak test. Therefore, I will keep my score of 2.
Best regards,
Thank you for your review and helpful feedback. We address your concerns point by point below.
W1 — Motivation and validation of the additive scheduling surrogate.
The surrogate is motivated by the propagation of small caching perturbations along the sampling trajectory. For a fixed schedule, let ϵt denote the perturbation introduced at cached step t, and let J0←t map a local perturbation to the final latent along the full-computation trajectory. Under locally smooth updates and small accumulated perturbations, a first-order expansion gives
Δx0=t∈S∑J0←tϵt+O(∥ϵ∥2),where ϵ stacks the per-step perturbations. Thus, propagated perturbation vectors approximately superpose. Their norms are generally nonadditive because the vectors can reinforce or cancel one another. The triangle inequality motivates summing their individual magnitudes as a tractable surrogate; the prompt-normalized cost used in Appendix A.4, Eqs. (41)–(42) remains a calibrated empirical score whose ranking quality must be checked. This is consistent with the distinction between true final-latent error and the surrogate in Sec. 3.3.
Our two-step study provides a direct ranking check: Spearman correlation is 0.983, Kendall correlation is 0.908, and pairwise ordering accuracy is 95.4% (Sec. 4.4). For incorrectly ordered pairs, the average true-error gap is 1.35×10−4. To examine the actual 12-NFE operating point, Sec. 4.4, Table 5 reports the final video fidelity of our schedule and 16 randomly generated alternatives, fixing the predictors and evaluating all candidates on the same 192 held-out prompts with 38 cached steps per schedule.
The DP-selected schedule achieves the best final video fidelity among all 17 evaluated schedules on all three metrics. Relative to the strongest alternative, SSIM increases by 0.026, PSNR increases by approximately 1.24 dB, and LPIPS decreases by 0.035. The complete results for the 16 randomly generated alternative schedules are provided in Appendix C, Table 10. Together with the two-step analysis, this comparison supports the practical utility of the surrogate when 38 of the 50 sampling steps are cached.
W2 / Q2 — Generalization at 24 NFEs.
We additionally evaluated the higher-fidelity 24-NFE setting on the same 192 GPT-generated held-out prompts used in Sec. 4.5. The additional results are SSIM 0.923, PSNR 30.326 dB, and LPIPS 0.039. They are close to the 24-NFE VBench-prompt results in Table 2:
The existing 12-NFE held-out results and prompt-generation protocol are reported in Sec. 4.5. The additional 24-NFE results extend this evidence to the higher-fidelity setting, where the held-out and VBench-prompt measurements remain close across all three fidelity metrics.
W3 / Q1 — Offline calibration, online spectral overhead, and fair latency comparisons.
We have added calibration costs and online overhead to Sec. 4.1. For Wan2.1-1.3B, jointly fitting the rank and scaling predictors takes approximately 5 hours, and schedule calibration takes 4 days, on PPU hardware. A single eigendecomposition takes 263 ms.
The reported 94.991 s is online inference latency, including Gram-matrix construction and eigendecomposition, and excludes offline calibration. We consider this a fair comparison of online inference: all methods are timed under the same A100 protocol, and baselines such as ERTACache and DPCache also require offline calibration, whose cost is likewise excluded from their online latency.
The calibration also generalizes beyond the VBench evaluation prompts. Without recalibration, the same predictors and schedule achieve closely matching SSIM, PSNR, and LPIPS on 192 held-out prompts (Sec. 4.5). This supports calibration as a reusable model-level configuration across prompts.