- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper revisits timestep-level caching for diffusion Transformers and proposes a two-component framework: a quality-driven cache scheduler and a lightweight neural compensator. The paper first analyses residual caching through a first-order Taylor expansion and argues that, because the variation of the input hidden states is small relative to the output feature scale, residual caching is approximately equivalent to direct output caching. On that reading, timestep-level caching becomes a numerical integration problem whose quality is jointly determined by where the enlarged steps are placed and how the resulting truncation errors are rectified.
Critical strengths
-
Strong empirical results from a genuinely lightweight framework.
-
The ablations are broad and separate the two components reasonably well.
-
The method is validated on both image and video modalities.
Critical weaknesses
-
The paper treats residual caching as approximately equivalent to direct output caching and uses this equivalence to establish a numerical integration interpretation. However, a small variation in the input hidden states relative to the output feature magnitude does not imply that it is negligible relative to the actual caching error. The approximate error vectors of the two caching formulations differ by , so this term should be assessed relative to the actual error terms, rather than only against the absolute output feature magnitude. Table S1 provides some empirical support, but does not sufficiently establish the approximation’s scope of validity across models, timesteps, and cache intervals.
-
The paper searches for a reusable cache schedule using very few calibration samples. However, similar temporal feature-variation profiles across prompts do not imply consistent rankings of cache positions in terms of their impact on final perceptual quality. The calibration-set-size ablation does not sufficiently report variability across independently sampled calibration sets. Consequently, the schedule’s robustness to prompt content, initial noise, and calibration-sample selection remains unclear.
-
Some conclusions about scheduling hyperparameters rely on small evaluation sets and modest LPIPS differences, without reporting variability due to calibration-set resampling or generation randomness. It therefore remains unclear whether the recommended configurations consistently outperform the alternatives.
-
The complete framework requires offline search and training of a neural compensator, whereas the main experimental baselines require no gradient-based training. Although the scheduler-only ablation helps distinguish the contributions of the two components, comparisons with learning-based caching methods are still missing. The authors should include learning-based baselines under comparable implementation conditions and report offline optimization costs alongside matched inference latency to establish the framework’s advantages over existing learning-based caching strategies. If trainable methods perform poorly, the authors should analyze why training-based approaches have developed relatively slowly in diffusion caching.
-
The authors are encouraged to include a focused analysis of GOC[1] and HiCache[2] in the related work section. Discussing these representative studies would help trace the development of diffusion feature caching and clarify the technical positioning of this submission.
[1]. Qiu, Junxiang, et al. "Accelerating diffusion transformer via gradient-optimized cache." 2025 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE, 2025.
[2]. Feng, Liang, et al. "HiCache: A plug-in scaled-Hermite upgrade for Taylor-style cache-then-forecast diffusion acceleration." International Conference on Learning Representations. Vol. 2026. 2026.
Questions for authors
See the Weaknesses.
3 comments
Rebuttal from authors for Weakness2 and Weakness3
For W2
We have tested calibration robustness through repeated end-to-end searches with resampled calibration prompts and initial noise. Across the tested N=3–8 range, adding calibration samples does not consistently improve fidelity; N=3 provides competitive average fidelity at lower search cost, consistent with our original results reported in Appendix ("Calibration Set Size (N)" subsection), now E.1 of our newest revised manuscript.
On FLUX.1-dev, all searches use 512×512 resolution, 30 total sampling steps, 10 evaluations, guidance 3.5, k=4 and the adjacency constraint.
We examine two sources of variation: five independently sampled nested calibration sets with search seed 0, and five search-noise seeds (0–4) with one fixed nested calibration set. Entries are mean ± standard deviation across complete searches. Both panels evaluated on the same 50 prompts and matching uncached references.
| Calibration size N | Five calibration sets: LPIPS ↓ | Five noise seeds: LPIPS ↓ |
|---|---|---|
| 3 | 0.16919 ± 0.04663 | 0.16693 ± 0.02984 |
| 4 | 0.17884 ± 0.03121 | 0.18514 ± 0.03572 |
| 5 | 0.18173 ± 0.03559 | 0.17041 ± 0.03103 |
| 6 | 0.15633 ± 0.01257 | 0.17876 ± 0.03200 |
| 7 | 0.17667 ± 0.02961 | 0.16458 ± 0.03705 |
| 8 | 0.18606 ± 0.05539 | 0.16262 ± 0.00628 |
The calibration-draw schedules also have a broader evaluation on 128 prompts × two seeds. In that panel, N=3 has lower mean LPIPS than N=8 in four of the five paired calibration draws.
In the five-search-seed panel, the N=3-minus-N=8 mean difference is +0.00431 LPIPS. N=3 uses 6,555 logical full forwards versus 17,480 for N=8, 2.67x speedup. The measured seed standard deviations are 0.02984 and 0.00628, respectively. The simultaneous 95% paired interval for the endpoint difference is [-0.04395, +0.05257]. These measurements quantify the fidelity–calibration-cost trade-off and its variability.
For W3
We have added repeated ablations with paired calibration resampling and search-noise variation, separating the effects of and the adjacent-step constraint. We sample 10 independent 3-prompt calibration sets, assigning a different search-noise seed to each set. Within each draw, all settings share the same prompts and seed. The primary experiment crosses with constraint on/off. All searches use FLUX.1-dev, 512×512 resolution, 10/30 model evaluations and CFG 3.5. Both panels evaluated on the same 50 prompts and matching uncached references. Entries are mean ± standard deviation across ten complete searches.
| Search budget | With constraint: LPIPS ↓ | Without constraint: LPIPS ↓ | Paired draws favoring constraint |
|---|---|---|---|
| 3 | 0.14042 ± 0.01388 | 0.14955 ± 0.02367 | 7/10 |
| 4 | 0.14888 ± 0.02202 | 0.15387 ± 0.03504 | 8/10 |
| 8 | 0.14879 ± 0.02190 | 0.15852 ± 0.04459 | 8/10 |
Ablation of : hold the constraint enabled and vary . Here is the maximum number of cache positions selected per greedy round. Comparing the constrained column, gives lower LPIPS than in 8/10 paired draws, with a mean difference () of -0.00846 and lower search-to-search dispersion. Logical search work is 8,406, 6,555 and 4,065 full-model forwards for , respectively. uses 22.0% fewer calculation than , quantifying the quality–search-cost trade-off at the same inference budget. Together with our original ablation in Appendix E.2, these results support moderate values of as practical choices, with favoring fidelity and reducing offline search cost. We also use in 30 full steps experiments in our main paper.
Ablation of constraint: Each row compares paired searches with and without the adjacency constraint while holding the per-round search budget fixed. The constraint reduces mean LPIPS by 6.1%, 3.2%, and 6.1% at , respectively. It also reduces the between-search standard deviation at all three budgets, indicating more stable fidelity across repeated searches. Among the ten paired draws, the constrained variant achieves lower LPIPS in 7/10, 8/10, and 8/10 cases, respectively. These results consistently support the constraint’s benefits for both average fidelity and search robustness.
We have added this results to our newest revision (Appendix E.2, E.3).
Rebuttal from authors for Weakness4 and Weakness5
For W4
We compare the author-published ReCache FLUX policies with TaylorSeer orders 1 and 2 at 512×512 resolution, 100 MSCOCO prompts and the native 50-step flow-matching Euler grid. ReCache uses 13 full evaluations. Ours method is evaluated at 13 and 15, with the budget shown explicitly. All methods use seed 0, guidance 3.5 and a BF16 backbone, and are compared against the same uncached 50-step references.
| Method | Full50 | PSNR ↑ | SSIM ↑ | LPIPS ↓ | Gen. time | Speedup |
|---|---|---|---|---|---|---|
| ReCache + TaylorSeer O1 | 13 / 50 | 24.664 | 0.84488 | 0.13725 | 3.895 s | 2.52× |
| ReCache + TaylorSeer O2 | 13 / 50 | 24.275 | 0.83928 | 0.14633 | 4.041 s | 2.43× |
| Ours (G2-top5-13) | 13 / 50 | 25.042 | 0.84362 | 0.14534 | 2.578 s | 3.81× |
| Ours (G2-top5-13++) | 13 / 50 | 26.142 | 0.87176 | 0.11880 | 2.608 s | 3.77× |
| Ours (G2-top5-15) | 15 / 50 | 26.970 | 0.87218 | 0.11722 | 2.991 s | 3.29× |
| Ours (G2-top5-15++) | 15 / 50 | 27.787 | 0.89442 | 0.09612 | 2.999 s | 3.28× |
For scheduler, our greedy-search selected schedules.
-
13/50:
[0, 1, 2, 3, 4, 5, 7, 10, 15, 21, 30, 39, 46] -
15/50:
[0, 1, 2, 3, 4, 5, 6, 7, 8, 10, 15, 21, 25, 37, 47]
The ++ variants additionally use our new-trained neural compensator. We retain the compensator architecture from the main paper and use 120 training prompts for preparation in the native 50-step setting. At the matched 13/50 budget, our complete framework improves all 3 metrics over both ReCache variants.
The latency difference is consistent with the implementations: TaylorSeer maintains per-layer derivative estimates and forecasts attention, feed-forward, and projection outputs across multiple Transformer blocks, while retaining surrounding per-step network operations. Our cached steps operate directly on the cached model output with a single lightweight compensator call.
Offline preparation costs:
| Method | Teacher-trajectory collection | greedy search | compensator train |
|---|---|---|---|
| G2-top5-13++ | 26.75 min | 51.37 min | 25.82 min |
| G2-top5-15++ | reused | 47.60 min | reused |
The 13/50 config requires 1.73 hours of total preparation on a single RTX 4090 48GB GPU. 15/50 config requires an additional 0.79 hours, reusing both the teacher data and the trained compensator. (Comparision: ReCache official paper reports 22 and 23 H100 GPU-hours for training its budget-conditioned FLUX policies with TaylorSeer O1 and O2, respectively). We have added this results to Appendix C.5 in our revised manuscript.
For W5
Thank you for suggesting these studies! We have expanded Appendix A to clarify their relationship to our compensator and scheduler.
Forecasting and correction share the goal of approximating the temporal evolution omitted by cache reuse. A feature forecast can be expressed as a cached feature plus an estimated temporal increment. GOC estimates this increment through a weighted first-order feature difference, while HiCache stabilizes polynomial forecasting through a scaled Hermite basis. Our compensator learns the corresponding increment for timestep-level predictions through teacher supervision, conditioned on historical predictions and timesteps. It can therefore be understood as learned residual forecasting, with a nonlinear conditional mapping providing an alternative to prescribed temporal extrapolation rules.
Our scheduler addresses the complementary question of where fresh computation is most valuable. The effect of a local approximation error on final quality depends on its position and propagation through subsequent denoising steps. We therefore search for cache schedules under a fixed budget by scoring final-output LPIPS. GOC’s inflection-aware gate determines whether to apply correction at an already skipped step; our scheduler determines which timesteps receive full evaluations. The schedule thus sets the refresh anchors and prediction horizons, while the compensator estimates the missing evolution between those refreshes.
Rebuttal from authors for Weakness1
For W1
We agree that the relevant scale is the actual caching error. The revised Appendices C.1–C.3 quantify the discrepancy on this scale, establish when the approximation applies, and show closely matched local errors and global final fidelity in more tested settings.
The δH in your comment is exactly the quantity denoted by δek in the revised paper; we use the latter notation below. For a shared refresh anchor a, the same current input and conditioning, and local errors measured against the same fresh output, C.1 establishes
\delta e_k&:=e_k^{\mathrm{res}}-e_k^{\mathrm{out}}=H_k-H_a,\\ \qquad r_k:=\frac{\|\delta e_k\|}{\|e_k^{\mathrm{out}}\|}, \qquad \left|\frac{\|e_k^{\mathrm{res}}\|}{\|e_k^{\mathrm{out}}\|}-1\right|\le r_k. \end{aligned}$$ The relative bound holds for $\|e_k^{\mathrm{out}}\|>0$, and the identity is exact in real arithmetic for nonlinear, time-dependent networks. **Thus, $r_k\ll1$ means that switching between output and residual caching changes the prediction by an amount much smaller than the error already introduced by caching.** C.2 measures this quantity on full-teacher trajectories covering 200 FLUX prompts and 30 OpenSora prompts, and on actual cached inputs through shadow evaluations. The probes cover early, middle and late denoising stages; shadow evaluations compute fresh outputs without changing the sampling trajectory. Table S1 reports: | Model | Full teacher: $r_k$ (%) | Cached-input shadow: $r_k$ (%) | | --- | --- | --- | | FLUX.1-dev | 0.188 / 0.404 | 0.169 / 0.420 | | OpenSora-1.2 | 2.902 / 5.382 | 3.431 / 6.010 | Entries are median / 95th percentile; normalization by the residual-cache error gives nearly identical values. Across teacher lags 1/2/4/8, the median discrepancy ranges from 0.146% to 0.296% for FLUX and 2.833% to 3.021% for OpenSora (Table S2 in our revised manuscript). OpenSora's discrepancy is a few percent of the existing caching error. At a probe with $r_k=0.03$, the bound above places the residual-cache error norm within $[0.97,1.03]$ times the output-cache error norm. The observed norm changes are much smaller because the drift is nearly orthogonal to the existing error. With $\phi_k$ denoting the angle between $\delta e_k$ and $e_k^{\mathrm{out}}$, C.2 gives $$\frac{\|e_k^{\mathrm{res}}\|^2}{\|e_k^{\mathrm{out}}\|^2} =1+2r_k\cos\phi_k+r_k^2.$$ The median $|\cos\phi_k|$ is 0.019 / 0.015 on teacher / cached inputs. Directly measured changes in error norm have medians of **0.038% / 0.063%** and 95th percentiles of **0.245% / 0.392%**. The median angles between the two error vectors are **1.663° / 1.965°**. These measurements show closely matched error magnitudes and directions in the evaluated probes. C.1 also carries this discrepancy explicitly into the integration analysis. Exact Euler grouping holds when the complete velocity after the head, guidance and parameterization conversion is frozen. With the current-time head retained, the residual-cache interval bound contains $$\sum_{k=a}^{b-1}|h_k|\bigl(L_{G,k}\|\delta e_k\|+D_k^{\mathrm{res}}\bigr)$$ in addition to the $O(\Delta^2)$ integration term, under the stated regularity assumptions. Here $h_k$ and $\Delta$ are the solver-time increment and interval span. $L_{G,k}$ bounds the sensitivity of the remaining map to its hidden input, and $D_k^{\mathrm{res}}$ measures its drift with the current state and time; these terms are evaluated along the residual-cache trajectory. The numerical-integration interpretation therefore accounts for boundary-input drift and head drift separately. C.3 evaluates the downstream effect over complete generation trajectories. Under shared schedules, Table S3 reports the following mean LPIPS against matching uncached references: | Model and shared schedule | Output cache | Residual cache | | --- | --- | --- | | FLUX / ERTACache, 10/30 | 0.2418 | 0.2420 | | OpenSora / Ours, 10/30 | 0.1469 | 0.1482 | | PixArt / Uniform, 10/30 | 0.2713 | 0.2667 | | Wan2.1 / BudCache, 15/50 | 0.1685 | 0.1629 | **Residual reuse is therefore supported as an approximation to output reuse when $r_k\ll1$, as observed in the reported probes. Appendix C.1 quantifies how their difference enters the numerical integration error, while Appendix C.3 further supports the practical validity of this approximation across a range of mainstream diffusion models.**