acceptodds
Under review as a conference paper at ICLR 2027

When to Recompute: Error-Regulated Caching for Diffusion Transformers

Abstract

Diffusion transformers have become the dominant backbone for high-fidelity image and video generation, yet their inference remains prohibitively expensive: each sample requires dozens of full network evaluations under strict temporal dependencies that preclude parallelization. Feature caching alleviates this cost by reusing or predicting intermediate features across denoising steps, but existing methods rely on fixed or heuristic caching schedules that are not explicitly regulated by the actual prediction error, providing limited capability to detect when cached predictions drift from the full-computation trajectory. As the acceleration ratio increases, such accumulated errors can eventually lead to severe quality degradation. We present Error-Regulated Adaptive Caching (ERAC), an acceleration framework that regulates cache reuse through an explicit error budget, adapting to the current sample and timestep while bounding trajectory drift. Our method organizes the denoising trajectory into computation segments that begin with a full evaluation and continue with cached predictions, while a lightweight, time-aware autoregressive estimator monitors the cumulative drift of the cached trajectory and triggers a full computation whenever the estimated error exceeds a prescribed threshold. Because the trigger adapts to individual samples and timesteps, computation is reallocated to the steps that actually require full evaluation, keeping expensive forward passes sparse while maintaining trajectory error within a controllable tolerance. On FLUX.1-dev, our method achieves an ImageReward above 0.9736 at a 6.04× acceleration, reducing the deviation from full evaluation by more than a factor of three compared with the strongest fixed-schedule baseline, while maintaining performance at 7.08× acceleration where such baselines break down. On Wan2.1 1.3B, ERAC achieves the best PSNR, LPIPS, and SSIM among the compared methods at speedups of up to 3.04×.

Primary area:
generative models
Keywords:
Diffusion Transformers, Feature Caching, Inference Acceleration, Error Estimation, Adaptive Computation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_rnnBOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Error-Regulated Adaptive Caching (ERAC), a framework designed to accelerate Diffusion Transformers through adaptive feature caching. The key idea is to dynamically determine when to perform a full model computation based on estimated caching errors, rather than relying on fixed caching schedules. Specifically, the authors introduce a lightweight, time-aware autoregressive error estimator trained on calibration trajectories generated using fixed-schedule caching. The estimator predicts caching errors from latent drift, temporal information, and previous error estimates. When the predicted error exceeds a predefined threshold, a full DiT computation is triggered to refresh the cached reference.

The method is evaluated on FLUX.1-dev for image generation and Wan2.1 1.3B for video generation. Experimental results demonstrate competitive quality-efficiency trade-offs compared with existing acceleration methods, particularly at higher FLOPs-based speedup ratios, using Image Reward and GenEval for images and PSNR, LPIPS, and SSIM for videos.

Critical strengths

Strengths

  1. The paper is well-written, with clear and fluent explanations, making it easy to follow.
  2. The generated images demonstrate promising visual quality.
Critical weaknesses

Weaknesses

  1. The paper lacks sufficient experimental completeness, particularly ablation studies and other necessary analyses that could be included in the appendix (e.g., the analyses mentioned in Weakness 2).
  2. Although the method employs a lightweight error predictor, the paper does not provide sufficient details about the number and types of trajectories used for training. This makes it difficult to assess the computational cost and the method's sensitivity to the amount of calibration data.
  3. Around Line 148, the authors argue that the proxy signals adopted by previous methods are merely correlated with the true error rather than directly representing it. I agree with this observation. However, I believe the proposed method also essentially relies on proxy signals. More importantly, as mentioned around Line 253, the authors train the error predictor using calibration trajectories generated under fixed-schedule caching, whereas inference dynamically determines whether to perform a full DiT computation based on the predicted error. Consequently, the error estimates may not accurately reflect the actual inference conditions, potentially leading to a distribution shift between training and inference. It remains unclear whether the predictor can reliably estimate the true error under such circumstances.
  4. Around Line 275, the authors state that the MLP "shares the same weights across all time steps", but the paper does not sufficiently justify this design choice. Different denoising stages may exhibit different error evolution patterns.
  5. The experiments lack latency measurements. Additionally, VBench scores should be reported for video generation, while additional metrics such as PSNR are recommended for image generation.

Errors

  1. The categorization in Section 2.2, Training-Free Diffusion Acceleration, appears to be incorrect. Δ-DiT should be categorized as a feature caching method rather than a sampling-step reduction method.
  2. Around Line 366, the reported PSNR difference of 0.91 dB does not appear to correspond to any of the baseline comparisons in the experimental results.
Questions for authors

Questions

My main concerns have been detailed in the Weaknesses section. I would appreciate further clarification and explanations from the authors regarding these issues.

Reviewer_ESsCOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper addresses the problem of accelerating diffusion transformers through caching. The authors propose to estimate the accumulated error during denoising and decides when to perform caching.

The method builds on TaylorSeer for feature prediction and introduces a lightweight autoregressive error estimator to guide caching decisions. The authors evaluate the approach on FLUX.1-dev for image generation and Wan2.1 for video generation.

Critical strengths

The paper addresses a relevant problem, as reducing the computational cost of diffusion models remains an important challenge.

The evaluation on both image and video generation is also appreciated.

Critical weaknesses

I have major concerns on novelty and experiments:

  1. Novelty and related work. The paper builds on TaylorSeer and introduces an error-based criterion for deciding when to cache. It seems related to SenCache (CVPR26) that the authors are not citing and comparing to.

  2. Error regulation is not clearly demonstrated. The authors claim that the method controls accumulated error through a prescribed threshold. However, the threshold is applied to a learned estimate rather than the actual error. Without evaluating how often the true error exceeds the threshold, it is difficult to determine whether the method reliably regulates error or simply provides another adaptive caching heuristic.

  3. Limited evidence for the benefit of online adaptation.

The method uses an offline-trained estimator to make sample-specific caching decisions. However, the experiments do not isolate the benefit of these online decisions from the learned timestep-dependent error profile. A fixed schedule derived from the same estimator could potentially achieve similar results.

  1. Limited generalization and calibration analysis.

The experiments cover FLUX.1-dev and Wan2.1 1.3B, but the estimator is calibrated for a particular diffusion pipeline. It remains unclear how sensitive the method is to the calibration data and whether the learned estimator generalizes across different sampling schedules, model architectures, or generation settings.

  1. Experiments and quantitative results.

When looking at the quantitative tables, the numbers are not consistent across other related papers. One reason might be that speed-up is not exactly the same at (10% variance). But it does not justify the change in ranking in the methods... It will be nice to run the same experiments as previous work, at least on one table to validate the method on same setup.

Questions for authors
  1. How does your method differ and compare from SenCache (CVPR 2026)?
  2. How often does the actual accumulated error exceed the prescribed threshold? Can the authors quantify the accuracy of their error estimator?
  3. What is the benefit of online adaptation compared with a fixed schedule derived from the same estimator, at matched computational cost?
  4. How sensitive is ERAC to the calibration data?
  5. Can the authors explain the differences in baseline results and rankings compared with previous papers? Could they reproduce at least one existing benchmark under identical experimental settings?
Reviewer_vPT9OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

ERAC accelerates diffusion transformers through adaptive feature caching. The denoising trajectory is divided into segments that begin with a full forward pass, followed by cached steps using TaylorSeer feature extrapolation. Before each cached step, a small MLP predicts approximation error from latent drift, time since the last refresh, the reference timestep, and the previous error estimate. When the predicted error exceeds a threshold, a full forward pass starts a new segment and resets the reference and accumulated error. The predictor is trained offline on fixed-schedule caching trajectories to estimate relative latent error against full computation and remains frozen during inference.

Experiments evaluate FLUX.1-dev on DrawBench and GenEval and video generation with Wan2.1. On FLUX.1-dev, ERAC achieves comparable ImageReward to SpeCa at greater FLOPs speedups, reporting 6.04× and 7.08× versus 5.70× and 6.34×, respectively. On Wan2.1, it reports better PSNR, LPIPS, and SSIM than TeaCache, TaylorSeer, and SeaCache at approximately 2× and 3× speedups. The paper does not include ablations, wall-clock latency measurements, or an appendix.

Critical strengths

S1. A lightweight refresh policy with interpretable control. The estimator uses four scalar inputs, with reported overhead below 0.1% of a forward pass, and requires no backbone modifications. A single threshold controls the trade-off between quality and computation, offering a practical alternative to manually tuned reuse intervals. Compatibility with feature predictors and efficient attention further supports ease of adoption.

Critical weaknesses

W1. Limited novelty and incomplete positioning. The accumulate-and-refresh framework is established in TeaCache and MagCache, while SpeCa already combines TaylorSeer with per-sample error verification. ERAC’s main distinction therefore appears to be its four-feature MLP. Relevant methods, including MagCache, DiCache, EasyCache, and AdaptiveDiffusion, are not cited. The distinction from proxy-based approaches also needs qualification: ERAC predicts error from drift and timing features, whereas SpeCa directly computes a verification error.

W2. The core error estimator lacks direct validation. No ablations establish the MLP’s benefit over simpler triggers using the same TaylorSeer predictor, such as accumulated change, latent-drift thresholds, or fixed reuse intervals. Oracle comparisons, feature ablations, and threshold sweeps are also missing. Without held-out prediction accuracy, calibration, or missed-trigger statistics, the claim that the estimator accurately predicts accumulated drift remains unsupported.

W3. Unclear baseline provenance and comparability of FLOPs speedups. All baseline results in Table 2 exactly match SeaCache’s Table 4, yet the Reproducibility Statement claims that all baselines were reproduced using their official implementations. This apparent inconsistency requires clarification. Without confirming that ERAC and the baselines use identical generation settings, reference computation, and FLOPs accounting, the reported acceleration may not represent a fair comparison.

Questions for authors

Q1. Ablation of the estimator design. With TaylorSeer fixed and FLOPs matched, please compare the MLP against accumulated-change, latent-drift, and fixed-interval triggers, alongside an oracle using the true error. Please also ablate the four input features to establish which design choices contribute to the gains.

Q2. Baseline provenance and evaluation protocol. Were baseline results taken from SpeCa’s Table 1 and SeaCache’s Table 4? Please clarify their provenance, correct the mislabeled settings, reconcile the Reproducibility Statement, and confirm matched prompts, seeds, generation settings, evaluation code, and FLOPs accounting.

Q3. Practical efficiency. Please report end-to-end latency under identical hardware and generation settings, together with estimator and cache memory overhead. How do per-sample triggers affect batched inference, and how many full computations are required across samples?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.