acceptodds
Withdrawn from ICLR 2027

PEACache: Progressive Error Alignment for Diffusion Transformer Caching

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Diffusion Transformers (DiTs) exhibit strong generative capabilities but require repeated and computationally expensive denoiser evaluations. Feature caching reduces this cost by reusing intermediate computations, but inevitably introduces reuse-induced errors. In this work, we propose PEACache, a Progressive Error Alignment framework that controls caching errors at three progressively higher levels. At the branch level, instead of modeling the Transformer residual at a cached step as in most existing methods, we directly model the error caused by residual reuse, i.e., the missing residual change. Through SVD analysis, we show that this error can be more effectively corrected using historical residual changes, motivating a historical residual change subspace with adaptive spectral rank and update magnitude. At the classifier-free guidance (CFG) level, we show that independently optimizing branch-level errors can still lead to larger CFG output errors, since CFG combines conditional and unconditional predictions and its guidance scale amplifies their mismatch. We therefore supervise cache prediction directly on the CFG-composed output. At the final-latent level, we estimate the impact of caching each individual step on the final latent and use their accumulated impact as a scheduling surrogate, which is optimized via dynamic programming under a computation budget. On Wan2.1-1.3B, Open-Sora 1.2, and CogVideoX, our method substantially improves output fidelity at comparable acceleration. On Wan2.1-1.3B, among compared methods achieving over acceleration, Progressive Error Alignment improves the previous best SSIM/PSNR/LPIPS from to .

Primary area:
generative models
Keywords:
Diffusion Transformers, Cache Reuse, Generation-Aware Optimization, Adaptive Computation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_epWFOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes PEACache, a caching method for Diffusion Transformers. Unlike most prior methods that address caching error at a single level, it introduces a progressive error alignment framework with three levels. At the branch level, it predicts the missing residual change. At the classifier-free guidance (CFG) level, it supervises prediction on the CFG-composed output. At the trajectory level, it schedules cached steps via dynamic programming using an additive error surrogate. Experiments on three backbones show better fidelity than previous methods, except on VBench. However, there are concerns about the lack of theoretical justification and generalizability.

Critical strengths
  • It presents a unified caching framework spanning the branch, CFG, and final-generation levels, unlike existing single-level approaches.
  • It shows consistently better fidelity than previous methods across three backbones, with large margins under better acceleration.
Critical weaknesses
  • Lack of theoretical justification: the method is justified only empirically, without mathematical analysis. In particular, the additive surrogate is validated only on two-step schedules, while the actual schedules cache many steps.
  • Limited generalization evaluation: the unseen-prompt experiment reports only the lower-fidelity 12-NFE setting. It is unclear whether generalization holds in the higher-fidelity setting.
  • Unreported overhead: the paper does not report the offline calibration cost or the per-step overhead of the eigendecomposition.
Questions for authors
  • Does the 94.991 s latency at L424 include the offline calibration process on the small calibration set? If not, can the comparison with calibration-free methods be considered fair?
  • In the unseen-prompt experiment, how well does the method generalize in the higher-fidelity setting (24 NFEs)?
2 comments

Thank you for your review and helpful feedback. We address your concerns point by point below.

W1 — Motivation and validation of the additive scheduling surrogate.

The surrogate is motivated by the propagation of small caching perturbations along the sampling trajectory. For a fixed schedule, let ϵt\epsilon_t denote the perturbation introduced at cached step tt, and let J0←tJ_{0\leftarrow t} map a local perturbation to the final latent along the full-computation trajectory. Under locally smooth updates and small accumulated perturbations, a first-order expansion gives

Δx0=∑t∈SJ0←tϵt+O(∥ϵ∥2),\Delta x_0 =\sum_{t\in\mathcal S} J_{0\leftarrow t}\epsilon_t +O(\|\epsilon\|^2),

where ϵ\epsilon stacks the per-step perturbations. Thus, propagated perturbation vectors approximately superpose. Their norms are generally nonadditive because the vectors can reinforce or cancel one another. The triangle inequality motivates summing their individual magnitudes as a tractable surrogate; the prompt-normalized cost used in Appendix A.4, Eqs. (41)–(42) remains a calibrated empirical score whose ranking quality must be checked. This is consistent with the distinction between true final-latent error and the surrogate in Sec. 3.3.

Our two-step study provides a direct ranking check: Spearman correlation is 0.983, Kendall correlation is 0.908, and pairwise ordering accuracy is 95.4% (Sec. 4.4). For incorrectly ordered pairs, the average true-error gap is 1.35×10−41.35\times10^{-4}. To examine the actual 12-NFE operating point, Sec. 4.4, Table 5 reports the final video fidelity of our schedule and 16 randomly generated alternatives, fixing the predictors and evaluating all candidates on the same 192 held-out prompts with 38 cached steps per schedule.

Schedule, 12 NFEsSSIM ↑PSNR (dB) ↑LPIPS ↓
Strongest of 16 alternatives0.70720.7400.226
DP-selected schedule0.73321.9780.191

The DP-selected schedule achieves the best final video fidelity among all 17 evaluated schedules on all three metrics. Relative to the strongest alternative, SSIM increases by 0.026, PSNR increases by approximately 1.24 dB, and LPIPS decreases by 0.035. The complete results for the 16 randomly generated alternative schedules are provided in Appendix C, Table 10. Together with the two-step analysis, this comparison supports the practical utility of the surrogate when 38 of the 50 sampling steps are cached.

W2 / Q2 — Generalization at 24 NFEs.

We additionally evaluated the higher-fidelity 24-NFE setting on the same 192 GPT-generated held-out prompts used in Sec. 4.5. The additional results are SSIM 0.923, PSNR 30.326 dB, and LPIPS 0.039. They are close to the 24-NFE VBench-prompt results in Table 2:

SettingPrompt setSSIM ↑PSNR (dB) ↑LPIPS ↓
12 NFEsVBench0.73021.5430.188
12 NFEsHeld-out0.73321.9780.191
24 NFEsVBench0.92430.5250.038
24 NFEsHeld-out0.92330.3260.039

The existing 12-NFE held-out results and prompt-generation protocol are reported in Sec. 4.5. The additional 24-NFE results extend this evidence to the higher-fidelity setting, where the held-out and VBench-prompt measurements remain close across all three fidelity metrics.

W3 / Q1 — Offline calibration, online spectral overhead, and fair latency comparisons.

We have added calibration costs and online overhead to Sec. 4.1. For Wan2.1-1.3B, jointly fitting the rank and scaling predictors takes approximately 5 hours, and schedule calibration takes 4 days, on PPU hardware. A single eigendecomposition takes 263 ms.

The reported 94.991 s is online inference latency, including Gram-matrix construction and eigendecomposition, and excludes offline calibration. We consider this a fair comparison of online inference: all methods are timed under the same A100 protocol, and baselines such as ERTACache and DPCache also require offline calibration, whose cost is likewise excluded from their online latency.

The calibration also generalizes beyond the VBench evaluation prompts. Without recalibration, the same predictors and schedule achieve closely matching SSIM, PSNR, and LPIPS on 192 held-out prompts (Sec. 4.5). This supports calibration as a reusable model-level configuration across prompts.

Dear authors,

Thank you for the detailed rebuttal. The additional results resolve my concern on W2/Q2. For W3/Q1, I accept reporting the online latency separately. However, the reported calibration cost (about 4 days for schedule calibration plus 5 hours for predictor fitting) is quite large. I think it should be compared with the calibration time of other methods that also rely on offline calibration. Regarding W1, the first-order argument gives useful intuition, but it does not guarantee that the ranking of schedules is preserved. Also, comparing only against random schedules is still a weak test. Therefore, I will keep my score of 2.

Best regards,

Reviewer_azXdOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

PEACache accelerates video diffusion transformers through adaptive low-rank prediction of residual changes, CFG-aligned predictor fitting, and dynamic-programming cache scheduling based on offline estimates of final-latent error. Experiments on Wan2.1-1.3B, Open-Sora 1.2, and CogVideoX demonstrate improved fidelity to full-computation outputs at comparable inference latency.

Critical strengths

The paper demonstrates a consistent improvement in the fidelity–latency trade-off across three video diffusion backbones. At comparable measured inference latency, PEACache achieves better LPIPS, SSIM, and PSNR than the evaluated baselines (Table 2). These results provide meaningful evidence of practical value for accelerating video generation while preserving fidelity to the original models’ outputs.

Critical weaknesses

The incremental methodological contribution is not sufficiently established. The empirical results are promising, but comparisons with the closest methodological alternatives are incomplete, and the ablations do not clearly separate the benefits of the proposed alignment objectives from those of the combined prediction and scheduling design. This leaves uncertainty about which proposed ideas account for the improvements and whether they provide a substantial advance over existing approaches, limiting my confidence in the claimed contribution.

Questions for authors
  1. Could the authors compare PEACache with a closely related global scheduler, such as DPCache [1] or CEM [2], and with SVD-Cache [3], using the same backbone and comparable latency and calibration budgets? These comparisons would clarify the advantage over alternative global scheduling and subspace prediction designs.

  2. With the prediction architecture, cached CFG-gap reuse, and cache schedule held fixed, how do predictors fitted with conditional-branch supervision versus CFG-aligned supervision compare in CFG-output error and final generation fidelity? Table 1 reports an oracle comparison; the corresponding comparison using the fitted predictors would clarify how much of that advantage survives predictor fitting.

  3. What is the benefit of using final-latent error as the scheduling objective? With the predictor, DP algorithm, calibration data, and computation budget held fixed, how does the proposed objective compare with a local objective based on CFG-output error or one-step latent error? Table 3 demonstrates the benefit of replacing the scheduling framework, but does not isolate the contribution of this particular objective.

[1] Bowen Cui et al. “Denoising as Path Planning: Training-Free Acceleration of Diffusion Models with DPCache.” CVPR, 2026. https://arxiv.org/abs/2602.22654

[2] Tong Shao et al. “Plug-and-Play Fidelity Optimization for Diffusion Transformer Acceleration via Cumulative Error Minimization.” ICLR, 2026. https://arxiv.org/abs/2512.23258

[3] Guantao Chen et al. “Forecast the Principal, Stabilize the Residual: Subspace-Aware Feature Caching for Diffusion Transformers.” CVPR, 2026. https://openaccess.thecvf.com/content/CVPR2026/papers/Chen_Forecast_the_Principal_Stabilize_the_Residual_Subspace-Aware_Feature_Caching_for_CVPR_2026_paper.pdf

1 comment

Thank you for your review and helpful feedback. We address your concerns point by point below.

Q1 — Comparison with a global scheduler and SVD-Cache.

SVD-Cache has not released its implementation in the official repository. We therefore calibrated DPCache and evaluated its resulting schedule on Wan2.1-1.3B using the VBench prompt set. Both schedulers use 20 calibration prompts. For our method, we report the TeaCache + Our schedule ablation from Table 3, which retains direct residual reuse.

ScheduleSSIM ↑PSNR (dB) ↑LPIPS ↓
DPCache schedule0.65318.1640.250
Our schedule (TeaCache + Our schedule)0.68921.1460.214

Our schedule achieves better fidelity across all three metrics, demonstrating its advantage over the calibrated DPCache schedule.

Q2 — Conditional-branch versus CFG supervision under fixed inference settings.

We calibrated the predictors with conditional-branch-only and CFG-aligned supervision, keeping the prediction architecture, cached CFG-gap reuse, and 12-NFE schedule fixed. Evaluation on the same 192 held-out prompts shows substantial improvements with CFG-aligned calibration (Sec. 4.3, Table 4):

Calibration targetSSIM ↑PSNR (dB) ↑LPIPS ↓
Conditional branch only0.59818.0220.321
CFG-composed output0.73321.9780.191

For reference, naive residual reuse under the same schedule (TeaCache + Our schedule, Table 3) achieves SSIM/PSNR/LPIPS of 0.689/21.146/0.214 on VBench, whereas the conditional-only results above are measured on the 192 held-out prompts. Although these are different test sets, our generalization experiment (Sec. 4.5) shows closely matching fidelity metrics on both sets for the same calibrated PEACache configuration. The numerically better direct-reuse results therefore provide a suggestive comparison consistent with Sec. 3.2: optimizing an individual branch without accounting for CFG may even underperform direct reuse.

Q3 — Final-latent versus local scheduling objectives.

As discussed in Sec. 3.3, an error introduced early in denoising perturbs the latent passed to subsequent steps and can affect the remaining trajectory. Consequently, one-step errors of the same magnitude at early and late steps can have different impacts on the final video. Since our goal is to preserve final video quality, we measure each caching decision by its impact on the final latent (Appendix A.4). This captures downstream propagation that local CFG-output or one-step latent errors do not measure.

Reviewer_zyBxOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

PEACache accelerates video diffusion transformers by predicting residual changes at cached steps, fitting the predictor against the classifier-free guidance (CFG) output, and selecting cache schedules via dynamic programming over an offline final-latent error surrogate. Experiments on Wan2.1-1.3B, Open-Sora 1.2, and CogVideoX show improved fidelity to full-computation outputs at comparable latency.

Critical strengths

The fidelity gains are substantial across all three backbones. On Wan2.1-1.3B, PEACache achieves a 3.44× speedup and an LPIPS of 0.188, compared with 0.290 for D2Cache at 3.25×. Table 1 also illustrates why optimizing each CFG branch independently can worsen the combined prediction, which directly motivates the proposed objective.

Critical weaknesses
  1. Table 3 shows that changing the schedule accounts for most of the improvement over TeaCache, and the remaining ablation adds residual-change prediction and CFG-aligned fitting together. This leaves the end-to-end contribution of CFG alignment unclear. The oracle comparison in Table 1 establishes the motivation but does not measure the gain from the fitted predictor during generation.
  2. The scheduling surrogate is validated on two cached steps on Wan2.1, while the 12-NFE setting caches 38 of 50 steps. Since the paper acknowledges that cache errors interact, the pairwise result provides limited evidence that the surrogate ranks schedules well at the aggressive budgets where the main gains are reported.
  3. Improved fidelity does not consistently translate into better generation quality. Table 2 shows consistent gains in LPIPS, SSIM, and PSNR against full-computation outputs. However, PEACache scores 78.36 on VBench versus 78.72 for ERTACache on Open-Sora at nearly identical latency (21.32 s vs. 21.13 s). Without uncertainty estimates or a breakdown of VBench dimensions, it is unclear whether the closer match to the reference videos translates into a better quality–latency trade-off.
Questions for authors

At the same 12-NFE schedule, how do the final video metrics change when the same predictor is fitted with branch-wise supervision instead of CFG supervision? At that budget, how does the DP-selected schedule rank against alternative schedules when evaluated using actual final-latent error on held-out prompts?

1 comment

Thank you for your review and helpful feedback. We address your concerns point by point below.

W1 / Q1 — Contribution of CFG-aligned calibration with fitted predictors.

Table 1 provides an oracle comparison that motivates CFG alignment, whereas the original progressive ablation in Table 3 combines residual-change prediction and CFG-aligned calibration. We have added a controlled calibration ablation in Sec. 4.3, Table 4. It retains the same predictor architecture, 12-NFE schedule, and cached CFG-gap reuse, and recalibrates the rank and scaling predictors using conditional-branch-only supervision instead of the CFG-composed target. This provides a branch-supervised control within the deployed prediction architecture, changing the supervision target while holding the inference formulation fixed. Both variants use the same 192 GPT-generated held-out prompts described in Sec. 4.5, with paired 50-step full-computation references.

Calibration targetSSIM ↑PSNR (dB) ↑LPIPS ↓
Conditional branch only0.59818.0220.321
CFG-composed output0.73321.9780.191

The CFG-composed values are the unchanged 12-NFE held-out results reported in Sec. 4.5. With the schedule and inference formulation fixed, CFG-aligned calibration improves all three final-video fidelity metrics. This shows that the benefit of the supervision target persists with fitted predictors, beyond the oracle comparison in Table 1.

W2 / Q2 — Schedule selection with 38 cached steps.

The 12-NFE schedule selected using our additive surrogate achieves the best SSIM, PSNR, and LPIPS among all 17 evaluated schedules (38 cached steps each). All schedules use the same fitted predictors and 192 held-out prompts. Below we compare against the strongest of 16 randomly generated alternatives; Appendix C, Table 10 provides the complete results for all alternatives.

ScheduleSSIM ↑PSNR (dB) ↑LPIPS ↓
Strongest of 16 alternatives0.70720.7400.226
DP-selected schedule0.73321.9780.191

W3 — VBench and the fidelity–quality distinction.

In our view, cache-based acceleration should aim solely to preserve the original model's generation quality by keeping accelerated videos as close as possible to full-computation outputs.

Appendix B, Tables 7–9 provides the complete VBench breakdown. On Open-Sora 1.2, PEACache outperforms ERTACache on 9 of 16 dimensions, although weighted aggregation yields a lower total score (78.36 vs. 78.72).

Some accelerated methods even score above the original model on individual dimensions. On Wan2.1, DiCache scores 69.78 in appearance style versus 68.59 for full computation, and PEACache at 24 NFEs scores 76.08 in human action versus 75.46 (Table 7). Since caching approximates the original model's computation, we do not regard these score increases alone as evidence that the generated videos are better.

We therefore use VBench as a supplementary reference. In our view, the primary criterion for caching is consistency with the original videos, measured by SSIM, PSNR, and LPIPS.

Comments on the paper

1 comment

Dear Reviewers,

As we are approaching the end of reviewer-author discussions, please go through authors' responses and raise questions that you would like authors to reply.

Thanks,

AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.