acceptodds
Under review as a conference paper at ICLR 2027

SoftCache: Adaptive Step Size Caching for Diffusion Transformers

Abstract

Diffusion and flow models generate a sample over iterative steps, each a full pass through a large network, and that pass dominates generation cost. Training-free caching accelerates them by reusing activations from recent evaluations instead of recomputing them. Dynamic caching methods decide when to refresh by accumulating a cheap probe of the network input into a staleness signal and spending a fresh evaluation once it crosses a threshold. They perform hard caching: the signal is a binary switch between refresh and reuse, and every step, fresh or cached, advances by the prescribed step size. We observe that the signal carries much more than this switch. How far it sits below the threshold, its headroom, predicts the error induced by reuse. We propose SoftCache, which performs soft caching: instead of a binary switch, the headroom sets how far each cached step goes. Where the headroom is large, the sampler jumps past several scheduled nodes, and as a refresh nears, its steps shrink back toward the original step. These adaptive jumps may land between the scheduled nodes, taking the sampler off its fixed grid. SoftCache requires no training and adds no per-backbone settings: on FLUX for images and on HunyuanVideo and Wan2.1 for video, at speedups from to , it is ahead of every baseline on all four fidelity metrics.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Diffusion acceleration, feature caching, diffusion transformer acceleration, training-free acceleration

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 4 reviews, above 85% of the others.
Reviewer_CJjrOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper proposes SoftCache, which uses additional information from the staleness signal that dynamic caching methods typically use to decide whether to reuse or refresh the cache. The authors observe that the accumulated signal correlates with the relative error caused by reusing cached residuals. They use the remaining margin below the refresh threshold to choose the sampling step size: a larger margin allows a longer step, which can skip scheduled timesteps and land outside the original grid. At comparable speedups, they report better fidelity to the uncached model’s outputs according to the evaluated metrics.

Critical strengths

The paper presents a clear, simple, and well-motivated idea: using information from the cache refresh signal as a measure of cache reliability to choose larger sampling steps. The framework is well structured, easy to apply, and requires no additional training. It achieves notable fidelity improvements over the evaluated baselines. For example, on HunyuanVideo at 6.05× speedup, it achieves 22.27 dB PSNR, 0.739 SSIM, 0.267 LPIPS, and 0.141 DreamSim. Compared with SeaCache at 6.11× speedup, this represents 1.89 dB higher PSNR and 0.082 lower LPIPS. The experiments are comprehensive, covering FLUX.1-dev for image generation and HunyuanVideo and Wan2.1-T2V-1.3B for video generation, with SoftCache achieving the best fidelity among the compared methods at similarly high speedups.

Critical weaknesses
  1. The benefit of headroom at 7× is not entirely clear (Table C.4.1). LPIPS and DreamSim show only small improvements, which are difficult to assess without uncertainty estimates, while PSNR worsens.
  2. It would be helpful to clarify when jumps happen and how big they actually are for different thresholds τ\tau. The tables do not clearly show this behavior during generation. Figure 1 provides a schematic illustration, but does not show the actual jump locations and sizes observed during sampling.
Questions for authors

see weaknesses

1 comment

We thank the reviewer for the constructive feedback.

Headroom benefit (W1): The reviewer is right to point out that at 7× the headroom does not improve the average scores. However, the headroom does matter at lower speeds, e.g. at 3× removing it costs 0.8 dB PSNR and 0.029 LPIPS (Tab. C.4.1). Our main contribution involves the headroom together with longer jumps, as further discussed in the response to W1 of reviewer jLNv.

Jump locations (W2): We added Sec. B.2 to the revised version. This subsection includes Tab. B.2.1 with the start time, length and headroom of the jumps at each speed of the main tables (Tab. 1-2) on all backbones, along with the τ\tau values that are used for each speed.

It shows that a run takes three to seven jumps across the whole trajectory, with almost half the jumps with a size bigger than six scheduled steps. Moreover, it shows that the headroom is soft rather than binary, so jump lengths vary with it instead of switching between a regular step and the cap.

Reviewer_jLNvOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper accelerates diffusion transformers with training-free caching. Dynamic caches (TeaCache, SeaCache) recompute the transformer stack when an accumulated staleness signal AiA_i exceeds a threshold τ\tau and otherwise reuse the cached residual, always stepping along the fixed grid. SoftCache instead uses the headroom Hi=1−Ai/τH_i = 1 - A_i/\tau to lengthen cached steps up to J=8J=8 scheduled steps, extrapolates the residual linearly to the off-grid landing, and re-spaces the remaining nodes. With 50 Euler steps on FLUX, HunyuanVideo and Wan2.1, it outperforms SeaCache and DPCache (plus TeaCache and TaylorSeer on FLUX) on PSNR, SSIM, LPIPS and DreamSim against the uncached sample at matched speedups of 3.5×–6.9×.

Critical strengths
  1. Consistent gains under a careful protocol. On three backbones and three matched speed bands, SoftCache beats every baseline, including SeaCache and DPCache, on all four fidelity metrics (e.g., +1.12 dB PSNR over SeaCache at ≈6.7–6.9× on FLUX; +0.54 to +2.78 dB on HunyuanVideo). Full speed sweeps, uninterpolated main tables, efficiency columns, per-prompt plots and a held-out test set make the matched-speed claim credible.
  2. Simple and training-free, with no extra network cost. Letting the existing staleness signal set the step length and leave the fixed grid costs one extra stored tensor and no additional stack evaluations, and it transfers to TeaCache's probe (Tab. C.2.1). Being orthogonal to the refresh rule, it can be layered onto most dynamic caches.
Critical weaknesses
  1. The headroom is not isolated as the source of the gains, although it is the paper's central claim. Fig. 2a shows that AiA_i correlates with the reuse error at scheduled nodes, not that HiH_i predicts the error of the long, extrapolated, off-grid step SoftCache actually takes. In Tab. C.4.1, setting Hi=1H_i=1 (always jumping the full JΔiJ\Delta_i) stays within 0.008 LPIPS of SoftCache at 5× and 7× and scores higher PSNR; the headroom helps clearly only at ≈3×, and jumps fixed in advance near the end of the trajectory trail by only 0.25 dB at τ=0.8\tau=0.8 (Tab. C.4.3). With β=1\beta=1, the PSNR margin over SeaCache drops to +0.04 dB at 6.89× (Tab. C.2.1), so the high-speed advantage comes largely from damping to β=0.5\beta=0.5 (+1.65 dB at 7×, Tab. C.4.2). The gains at ≥5× may therefore stem mainly from grid coarsening, tail re-spacing and extrapolation tuning, which would reduce the conceptual contribution to an engineered combination of SeaCache, step skipping and linear forecasting (Q1–2).
  2. Robustness to the sampler configuration is untested, although SoftCache rewrites it. All results use 50-step first-order Euler with one seed (Wan2.1's native UniPC is replaced by Euler), and all constants (J=8J=8 scheduled steps, guards t<0.1t<0.1 and ρi>0.18\rho_i>0.18, the β\beta switch at 4×) were fixed on 50 FLUX prompts. At N=20N=20–30 a J=8J=8 jump spans 1.7–2.5× more of the trajectory, and jumps have no error estimate or rejection, unlike adaptive ODE solvers. Since practical value depends on plugging into common solvers and step counts, this evidence is too narrow (Q3).
Questions for authors

Clarification

  1. Per speed band, what are the distributions of HiH_i and Δˉi/Δi\bar\Delta_i/\Delta_i at jumping nodes, and does HiH_i predict the measured local error of each jump?
  2. At matched speed, how does SoftCache compare with SeaCache run on a fixed non-uniform grid matching SoftCache's average node placement (dense opening kept), with and without extrapolation?
  3. How does SoftCache perform at N=20N=20–30 and with Wan2.1's native UniPC, and how should JJ and the guards scale with NN?
  4. β\beta switches at 4× achieved speedup, which depends on backbone and GPU. How is it set without first measuring speed, and is this not a per-backbone setting?
  5. Please report NFE per method and band: at equal TFLOPs, DPCache is about 15–21% slower in wall-clock (Tabs. B.1.1, B.1.3), so wall-clock matching partly reflects implementation overhead.
1 comment

We thank the reviewer for the constructive feedback.

Headroom gains (W1): We agree that the headroom alone does not explain all of the gains, and as noted by the reviewer, it matters most at low speed (Tab. C.4.1). Our main contribution involves the headroom, together with jumps that are longer than one scheduled step (J>1J>1), which take the sampler off the fixed grid and make the grid "soft". At 7×, jumps are a key component, e.g. showing a difference of 0.219 LPIPS with the same β\beta (Tab. C.4.1), so damping β\beta alone does not explain the gain. Following the reviewer’s remarks, since Fig. 2a measures error only at scheduled nodes, the revision now measures the jumps themselves (Tab. C.1.1, see Q1) and compares against a fixed grid (Tab. C.4.4, see Q2).

Jumps distribution (Q1): The revised version now contains App. B.2 and Tab. B.2.1, listing the start, length and headroom of every jump at each speed of the main tables, on all backbones. A run takes three to seven jumps across the whole trajectory, with headroom that varies continuously rather than switching between two values (see the response to reviewer CJjr, W2).

Moreover, to test whether HiH_i predicts each jump’s error, the revised version contains Tab. C.1.1, showing the error against the GT (uncached sampling over the same interval). At 5× and 7×, jumps with more headroom are more accurate (Spearman −0.58-0.58 and −0.74-0.74), so the sampler jumps further where the error is low. At 3×, all jumps have similar headroom, so it mainly acts as a brake that keeps jumps below the cap.

Fixed grid (Q2): On DPG-50 (FLUX), we froze SoftCache's most common schedule into one grid for every prompt and ran SeaCache on it at the same speed, with and without extrapolation (residual motion in the paper, Eq. 10). The revised version shows the results in App. C.4 and Tab. C.4.4. Without extrapolation, the fixed grid is behind SoftCache on LPIPS and DreamSim at every speed. With extrapolation, it is SoftCache with its jumps chosen in advance, tied at 2.95× and 5.44×, with a small gap at 6.90× (0.015 LPIPS, 0.009 DreamSim). Most of the gain therefore comes from the grid the headroom produces, but that grid is known only after running SoftCache.

Sampler and step count (W2, Q3): The choice of using 50 Euler sampling steps on all backbones was to keep the comparison unified. Following the reviewer's remark, we also tested fewer steps, and Wan2.1's native UniPC scheduler, as explained below:

  • Fewer steps (W2): App. C.5 and Tab. C.5.1 in the revised version compare our method with SeaCache, both using N=20 and N=30 steps, across different speedups. The cap remains at J=8J=8 on any step count, since it only caps a jump and HiH_i sets its actual length. The one change is the probe guard, which measures the change over one step and so becomes 0.18⋅50/N0.18\cdot50/N (App. A.3). With this, SoftCache leads SeaCache on LPIPS and DreamSim at every compared speed at both 30 and 20 steps, and only loses by 0.3 dB PSNR without statistical significance at τ=0.8\tau=0.8 with 20 steps.
  • UniPC (Q3): On Wan2.1's native 50-step UniPC, with no retuning, SoftCache beats fewer UniPC steps on all four metrics from 2.50× to 6.07×, and beats SeaCache on SSIM, LPIPS and DreamSim, with PSNR tied at 3.82× and 6.07×.

β\beta switch (Q4): 4× speedup indeed depends on measured speed. The rule for β\beta needs no timing, as it can be stated on the threshold τ\tau alone, with β=1\beta=1 for τ≤0.5\tau\le0.5 and β=0.5\beta=0.5 otherwise. τ\tau sets the number of evaluations, so it is GPU-agnostic. The same cut applies on all three backbones, so it is not a per-backbone setting. Following this remark, the revised version states the rule this way (Sec. 5.4).

NFE (Q5): NFE is added to Tab. B.1.1-3 in the revised version. Our speedup closely follows the NFE (e.g. 13 NFEs out of 50 give 3.79× on FLUX), and our method still wins on all 4 metrics in every band, also when it uses more NFEs than another method.

Reviewer_VyGtOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes SoftCache, which achieves joint optimization of caching and sampling. The overall motivation, methodology, and experimental design are sound. The effectiveness of the proposed method is validated through experiments on FLUX, HunyuanVideo, and Wan2.1.

Critical strengths
  1. The paper is well-written and well-organized.
  2. The methodology is robust, effectively implementing joint optimization of caching and sampling.
  3. The experiments are comprehensive, covering both image and video generation, with ablation studies that clearly demonstrate the contribution of each component.
Critical weaknesses
  1. Hyperparameters vary across different models, necessitating significant tuning.
  2. There is a trade-off between step-size control and error, which largely relies on empirical hyperparameter settings.
Questions for authors
  1. See the Weaknesses.
  2. The primary concern is that the hyperparameters require an extensive search process and are not adaptive.
1 comment

We thank the reviewer for the constructive feedback.

Hyperparameters concerns (W1, Q2): Throughout the paper, our hyperparameters were chosen on DPG-50 (50 DPG-Bench prompts, one seed) on FLUX, and used by that rule for all backbones, including video, without any additional tuning.

The search covers JJ and β\beta, one at a time. Using a close value of JJ gives similar results, as seen in Tab. C.4.2. For example, J=12J=12 instead of J=8J=8 changes LPIPS by at most 0.021 and PSNR by at most 0.83 dB.

Step size control and error tradeoff (W2): The step size and error tradeoff is handled dynamically during sampling by the headroom HiH_i, which reflects the relative reuse error (Fig. 2a), so SoftCache takes longer steps where the error is low and vice versa.

The two parameters that change, only with the speed and not per backbone, are τ\tau, which controls the achieved speed, as in previous dynamic caches (TeaCache, SeaCache), and β\beta, which we found works better when damped at higher speeds (Tab. C.4.2). Following reviewer jLNv's remark, β\beta now follows τ\tau directly (Sec. 5.4): β=1\beta=1 for τ≤0.5\tau\le0.5 and β=0.5\beta=0.5 otherwise, without affecting any result throughout the paper.

Reviewer_KA9LOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes SoftCache, a training-free caching framework that adaptively adjusts diffusion sampling step sizes using cache staleness signals. It combines headroom-based step-size control, residual extrapolation, and dynamic schedule re-spacing. Experiments on FLUX, HunyuanVideo, and Wan2.1 demonstrate improved fidelity over existing caching methods at comparable speedups.

Critical strengths
  1. Systematic and coherent design. The method integrates adaptive step sizing, residual prediction, and schedule re-spacing into a unified framework. The overall design is intuitive and computationally lightweight.

  2. Solid experimental results. The evaluation covers multiple image and video diffusion models, with consistent improvements over competitive baselines. The ablation studies and qualitative comparisons are also relatively comprehensive.

Critical weaknesses
  1. Substantial overlap with prior work. The residual extrapolation in Eq. (10) is highly similar to the second-order delta correction proposed in D2Cache (CVPR 2026). Both methods use the difference between two historical Transformer residuals to correct the current cached residual. The main difference is the scaling coefficient: SoftCache uses relative timestep distances, while D2Cache employs timestep-embedding-based adaptive scaling. Despite the nearly identical core formulation, D2Cache is neither cited nor discussed. This raises concerns about the novelty of one of the paper's main components.
Questions for authors

How does the residual extrapolation in Eq. (10) differ from the second-order delta correction in D2Cache (CVPR 2026)? Beyond the scaling strategy, what is the novel contribution of this component?

Overall, I find the framework well-designed and the experimental results convincing. My main reservation concerns the novelty and positioning of the residual extrapolation component. I currently lean toward weak rejection, but the paper is close to the acceptance boundary, and I would be open to revising my rating if this concern is convincingly addressed during the discussion.

2 comments

We thank the reviewer for the constructive feedback.

Regarding the overlap with D2Cache: Eq. 10 does have the same form as their delta term. We do not claim the extrapolation as our main contribution, as it was used in previous methods like TaylorSeer. Our main contribution is adaptive longer jumps, set using the headroom.

Following the reviewer’s remarks, we have both addressed D2Cache in the related works (Sec. 2.3) and in App. B.1, and compared it using a backbone that was used in the paper, WAN2.1. Tab. B.1.3 now shows that our method beats D2Cache on all 4 metrics in all evaluated speed bands (setup details in App. A.1).

Moreover, to check whether their extrapolation scaling will improve our method, we swapped their scale into our sampler on FLUX, and got small PSNR differences on all evaluated speeds (−0.04, −0.02, +0.25 dB). This means that the scaling coefficient is not what drives our gains.

Thank you for the clear clarification and for incorporating the comparisons and discussions with D2Cache in the revision. All my concerns have been well addressed, and I have raised my rating accordingly.

Comments on the paper

1 comment

We thank all reviewers for their careful reading and detailed comments. We are glad the reviewers found the method simple and lightweight, with no training and no extra network cost (KA9L, jLNv, CJjr), the paper clear and well organized (VyGt), and the experiments comprehensive across image and video models (all reviewers), with consistent gains over strong caching baselines (KA9L, jLNv, CJjr).

Below we first address the concerns raised by several reviewers, and then list the changes in the revised version. Each point is discussed in full in the individual responses.

Shared concerns

What drives the gains (jLNv, CJjr). The headroom matters most at lower speeds, and the jumps matter most at higher speeds (Tab. C.4.1). At 5× and 7×, jumps with more headroom are more accurate (Tab. C.1.1), so the sampler jumps further where the error is low.

Hyperparameters (VyGt, jLNv). All hyperparameters were chosen on the FLUX DPG-50 set and kept fixed on every backbone, including video, without any additional tuning. Only τ\tau and β\beta change, with the speed and not per backbone. β\beta now follows τ\tau directly (β=1\beta=1 for τ≤0.5\tau\le0.5 and β=0.5\beta=0.5 otherwise), without affecting any result.

How the jumps behave (jLNv, CJjr). A run takes three to seven jumps across the whole trajectory, and the headroom varies continuously rather than switching between two values.

Main revisions in the uploaded PDF (all changes are marked in red)

  • D2Cache (KA9L). Discussed in the related work (Sec. 2.3) and in App. B.1, and compared on Wan2.1 (Tab. B.1.3).
  • Where the jumps land (jLNv, CJjr). App. B.2 and Tab. B.2.1.
  • Jump error against uncached sampling (jLNv). Tab. C.1.1.
  • SeaCache on a fixed grid (jLNv). App. C.4 and Tab. C.4.4.
  • Fewer sampling steps, N=20 and N=30 (jLNv). App. C.5 and Tab. C.5.1.
  • NFE (jLNv). Added to Tabs. B.1.1–B.1.3.
  • β\beta rule stated on τ\tau (VyGt, jLNv). Sec. 5.4.

Our detailed response to each reviewer is posted separately, as a reply under the review.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.