acceptodds
Under review as a conference paper at ICLR 2027

ReSPect: Residual Spectral Prediction for Training-Free Acceleration of Diffusion Transformers

Abstract

Diffusion transformers underlie current image and video generation models, but incur substantial sampling costs due to iterative denoising. Training-free caching accelerates sampling through two coupled decisions: which steps to evaluate, and what to substitute for the skipped features. Many existing schedules gate on raw distances between adjacent features, a drift signal with poor signal-to-noise ratio, where stochastic variation masks meaningful drift, while ignoring how the noise level itself evolves along the sampling trajectory. Many existing reconstructors, in turn, rely on local approximation, copying the most recently computed features or extrapolating from a short local history. This approach causes approximation errors to increase with the skip length and sample quality to degrade at high speedups. We propose Residual Spectral Prediction for Efficient Caching of Transformers (ReSPect), a training-free cache that jointly addresses these limitations through spectrally aligned reuse decisions and residual trajectory prediction. Scheduling decisions are based on a spectrally aligned representation obtained from a timestep-dependent filter derived from the linear minimum mean square error (linear-MMSE) denoising formulation, which preserves content-relevant components while suppressing noise. Skipped residuals, rather than hidden states, are forecast by a weighted sum of ridge-regularized Chebyshev regression over the accumulated residual history, which captures the global trajectory, and a first-order discrete Taylor correction, which captures local dynamics. Analysis shows the forecast error admits a skip-length-independent bound, whereas the corresponding bounds for copy-reuse and Taylor extrapolation grow with skip length. Across extensive experiments on FLUX.1-dev, Wan2.1-1.3B, and HunyuanVideo, ReSPect achieves speedups of , , and over full denoiser evaluation while achieving lower output error than existing caching baselines at matched or lower compute.

Primary area:
generative models
Keywords:
Generative models, diffusion transformers, training-free inference acceleration, feature caching

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_RNgBOpenReview ↗
Rating
1
Confidence
4(+1)
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes ReSPect, a training-free caching method for diffusion transformers. Skip decisions are made by a gate that accumulates the drift of a timestep-dependent Wiener-filtered block input. On skipped steps, the transformer residual, rather than the hidden state, is forecast by blending a ridge-regularized Chebyshev fit with a first-order Taylor extrapolation. Experiments on FLUX.1-dev, Wan2.1-1.3B, and HunyuanVideo show higher PSNR/SSIM and lower LPIPS than TeaCache and TaylorSeer.

Critical strengths

The method achieves high reference similarity on video models.

Critical weaknesses

[W1] Limited Novelty

The proposed method is essentially a combination of two prior works. The “when to skip” component (Spectral-Aligned Gate) is highly similar to the spectrally aligned schedule of SeaCache (CVPR 2026). The “what to fill” component (Residual Forecaster) is highly similar to the Chebyshev prediction of Spectrum (CVPR 2026). I therefore find that the paper offers little to no methodological contribution.

[W2] Limited experimental evaluation

a) Although a substantial part of the method is drawn from SeaCache and Spectrum, Tables 1 and 2 include no comparison with either work.

b) The evaluation relies solely on similarity metrics (PSNR, SSIM, LPIPS). The prior works compared in this paper, TeaCache (CVPR 2025) and TaylorSeer (ICCV 2025), also report image and video quality metrics such as ImageReward or VBench scores, but these are missing here. Following at least the evaluation protocol of prior work is necessary to demonstrate the robustness of the results.

[W3] Presentation

Overall, the figures have poor readability, which makes the paper difficult to follow. In Figure 2, the text is too small, and many symbols appear before they are defined, which is confusing. Figures 3–5 make inefficient use of space. Tables 1 and 2 also do not explain their symbols (δ, S).

Questions for authors

See weaknesses.

3 comments
AuthorsOpenReview ↗

Response to Reviewer RNgB

We thank the reviewer for the detailed review and for noting ReSPect's high reference similarity on video. All changes are in the revised paper and supplementary (blue).

[W1] Limited novelty. ReSPect builds on SeaCache and Spectrum, and the revised paper states this plainly: the spectral gate is SeaCache's, unmodified, and the Chebyshev–Taylor forecaster and its bounds are Spectrum's (both CVPR 2026, within the four-month contemporaneous window of the ICLR 2027 AC guidelines); polynomial feature forecasting is an active line of work (TaylorSeer, ICCV 2025; HiCache, ICLR 2026; SADA, ICML 2025, at the solver level). Like most methods, ReSPect is built from known parts, and its contribution lies in how the parts are composed: the cache design. We measured each of its choices (FLUX.1-dev):

  1. Forecast the cached residual instead of copying it. Residual caches (TeaCache, SeaCache) reuse the last block residual; ReSPect forecasts it from all evaluated steps, as a drop-in: +0.80 dB [+0.65, +0.95] and +0.43 dB [+0.20, +0.66] over copy-reuse on identical steps at 18 / 13 NFE.
  2. Forecast on content-adaptive steps. With the same forecaster, the gate's prompt-dependent steps beat Spectrum's schedule by +0.66 dB at 18 NFE, a fixed schedule by +1.2 dB at 27 NFE and uniform spacing by +10.2 dB.
  3. Link the two decisions. Under a Lipschitz assumption the gate bounds the copy-reuse error of every skipped step, and the forecast reduces its systematic part: 43% lower integrated latent error, on all 16 prompts (Sec. 3.2, App. J).

Placement matters more than the forecaster: a Taylor forecaster on a fixed interval (TaylorSeer) reaches 19.68 dB at 18 NFE, a first-order Taylor forecast of the block residual on our gate's steps 26.5 dB. The composition beats both parents at matched compute: on FLUX.1-dev 26.14 dB at 18 NFE vs. 25.34 (SeaCache) and 25.48 (Spectrum); on video +7.9 / +4.5 dB over Spectrum on Wan2.1 and +4.8 dB on HunyuanVideo (δ=0.27), and +0.89 / +0.64 dB over SeaCache on HunyuanVideo. The revised paper rebuilds its contributions around these findings (Secs. 1–3).

[W2a] SeaCache and Spectrum. Both are now compared, every baseline re-run from its released code in one harness (same prompts, seeds, uncached reference and GPU; no published number reused). FLUX.1-dev, DrawBench-200, 1024², 50 steps:

MethodNFEPSNR↑SSIM↑LPIPS↓Speedup
Spectrum (α=0.3)1825.480.8850.1012.56×
SeaCache (δ=0.37)18.125.340.8790.1132.62×
ReSPect (δ=0.37)18.126.140.8950.0922.52×
Spectrum (α=1.0)1323.720.8470.1423.48×
SeaCache (δ=0.6)1321.260.8000.2133.55×
ReSPect (δ=0.6)1321.690.8230.1753.40×

At 18 NFE ReSPect is best on all three metrics, and it is ahead of SeaCache at both budgets. At 13 NFE the gate skips one of FLUX.1-dev's first five, structure-forming steps; forcing them raises ReSPect to 23.18 dB. The revised paper compares both on every model and budget: Tab. 1 (with TeaCache, TaylorSeer, SADA and fewer steps), Tab. 2 (SeaCache's published video rows) and App. L of the revised supplementary (both on video in one harness, 944 prompts, matched NFE; margins in W1).

[W2b] Quality metrics. The revised paper reports the quality metrics used by TeaCache and TaylorSeer for every model and budget, SeaCache and Spectrum included (App. M of the revised supplementary): ImageReward and CLIP score on FLUX.1-dev, and all 16 VBench dimensions (944 prompts, official code) on Wan2.1 and HunyuanVideo. ReSPect's ImageReward is 1.008 / 0.988 at 18 / 13 NFE (uncached 0.999) and its CLIP score 27.76 / 27.68 (uncached 27.70), both within the paired 95% CI of the uncached model. On VBench, ReSPect stays within 0.2 points of the uncached model's Total at every budget, and at the smaller budget it scores above SeaCache, Spectrum and TeaCache on both models.

[W3] Presentation. We revised each item. (i) Fig. 2 is moved into Sec. 3.2, after the text that introduces its notation, enlarged to full width, and its caption defines every symbol in the paper's notation; (ii) Figs. 3 and 4 now share one row, and Fig. 4 is one quality–compute plot with every re-measured FLUX.1-dev method on a common NFE axis; (iii) the captions of Tabs. 1–2 define δ, N (TaylorSeer's interval, previously S) and α, both tables have an NFE column, and the source of every row is stated. A redrawn Fig. 2 with larger text, and qualitative figures with fewer methods at larger size and the prompt in the caption, follow in the camera-ready version.

We hope the added comparisons and the revision address the reviewer's concerns and are glad to clarify anything further.

I thank the authors for the extensive revision. However, my main concern, the limited novelty (W1), remains, and the new results confirm it. I therefore maintain my rating.

The revised paper now states that the gate is SeaCache’s, unmodified, and that the forecaster is Spectrum’s. The remaining distinctions do not explain the gains:

  • Residual target: no gain over the hidden state (App. K). The authors themselves “do not attribute the gain to the choice of target.”
  • Forecaster: on Spectrum’s schedule, it reproduces Spectrum exactly (25.48 vs. 25.48 dB). On ReSPect’s own steps, Taylor-only outperforms the proposed blend.
  • Code: util_seacache.py is identical to SeaCache’s file, and the forecaster is described as a “faithful reimplementation” of Spectrum’s, with Spectrum’s default settings.

The contribution is therefore placing Spectrum’s predictor inside SeaCache’s cache, which I do not consider sufficient for ICLR.

We thank the reviewer for reading the revision and for the follow-up. We agree with the facts listed: the gate is SeaCache's, unmodified; the forecaster reimplements Spectrum's, with its released settings; and the residual target alone brings no gain over the hidden state. The revised paper states all three, and we do not claim them.

What we claim is the cache design: residual caches such as SeaCache copy the cached block residual on skipped steps, whereas ReSPect forecasts it from all evaluated steps, on the content-adaptive steps selected by the gate. The revision isolates both parts:

  • Forecast instead of copy (same gate, identical steps): +0.80 dB [+0.65, +0.95] and +0.43 dB [+0.20, +0.66] on FLUX.1-dev (paired 95% CIs), and +0.89 / +0.64 dB on HunyuanVideo (Tab. 3, App. L).
  • Content-adaptive instead of fixed steps: with the same forecaster at 18 NFE on FLUX.1-dev, Spectrum's schedule gives 25.48 dB, reproducing Spectrum as it should, and the gate's steps give 26.14 dB (App. K). Against Spectrum itself on video, at matched NFE: +7.9 / +4.5 dB on Wan2.1 and +4.8 dB on HunyuanVideo (δ=0.27) (App. L).

The two observations in the comment support this reading. That the forecaster reproduces Spectrum on Spectrum's schedule shows that the forecaster is unchanged and that the difference comes from the steps it runs on. That Taylor-only outperforms the blend shows that the gain comes from where and when the residual is forecast, not from the particular forecaster: a first-order Taylor forecast reaches 26.5 dB inside our cache, while TaylorSeer reaches 19.68 dB at the same 18 NFE (Tabs. 1 and 3). The settings in which copy-reuse or Spectrum remains ahead are listed in the Limitations.

Reviewer_DLUjOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes ReSPect, a training-free method for accelerating diffusion transformers in image and video generation. It uses a timestep-dependent spectral criterion to decide when to skip computation and combines Chebyshev regression with first-order Taylor extrapolation to approximate transformer residuals at skipped steps.

Critical strengths

S1: The method demonstrates strong empirical speed fidelity tradeoffs. The similarity metrics (LPIPS/SSIM/PSNR) is very competitive on selected video models.

Critical weaknesses

W1: Missing theoretical connection between the criterion and the approximation. The paper does not establish why steps selected for skipping by the proposed criterion can be accurately approximated by the proposed predictor. Does the criterion guarantees a bounded approximation error at those steps? What is the connection between the two part of the method, as a whole? The two methods seems separated, and the effect of criterion on generation is not yet sufficiently ablated.

W2: Missing comparison with a relevant baseline. The paper addresses two problems in diffusion acceleration: (1) when to skip, using a criterion based on the spectral response, and (2) how to skip, using a Chebyshev-based approximation. However, this joint consideration of skipping decisions and approximation method was already explored in SADA (ICML 2025). Moreover, SADA proposes an approximation method using Lagrange interpolation for multistep global prediction, which is similar in spirit to this paper’s motivation and method of using chebyshev. The author should directly discuss and compare with the previous work. Meanwhile, most related work discussed by authors in the paper (SeaCache, Spectrum) is also not compared.

Questions for authors

Q1 Please implement and report with Flux1 and Flux2. replace the proposed adaptive criterion with a fixed skipping schedule. For 50 sampling steps, keep the first 10 and last 5 steps as exact evaluations. In the intermediate steps, skip two out of every three evaluations and use the proposed Chebyshev–Taylor predictor to approximate the skipped steps. What is the resulting generation fidelity and speedup?

Q2 Why approximate residuals rather than hidden states? Given that the current-step input is known, these two formulations appear equivalent in terms of the mean and variance of the prediction error. What advantage does residual prediction provide?

Q3 The teaser figure is visually compelling, but it does not provide enough information. For example, the author should (may be in a separate plot) compare the unmodified baseline output and ReSPect output.

2 comments
AuthorsOpenReview ↗

Response to Reviewer DLUj

W1: Connection between criterion and predictor. Bound. Let Δ̃ₛ be the filtered input drift the gate accumulates. Within a skip run after the evaluated step k, the gate guarantees Δ̃ₖ₊₁ + … + Δ̃ₜ < δ. If the block residual is L-Lipschitz in the filtered input, then ‖r(t) − r(k)‖₁ ≤ L · Σₛ Δ̃ₛ ‖Ĩₛ₋₁‖₁ < L δ · maxₛ ‖Ĩₛ‖₁ (sum over s = k+1, …, t; maximum over k ≤ s < t): the gate bounds the copy-reuse error of every skipped step, uniformly in the skip length. Empirical check (16 FLUX.1-dev prompts, 509 skipped steps): the accumulated drift predicts the copy-reuse error (Spearman 0.77). Copy-reuse substitutes residuals from an earlier noise level, so its errors are systematic and add up over the trajectory, while the forecast's largely cancel: its latent error integrated over the whole trajectory is 43% lower, on all 16 prompts. Effect of the criterion (same predictor, FLUX.1-dev, PSNR):

Schedule18 NFE27 NFE
Uniform spacing15.9418.00
Spectrum's index-based schedule25.4825.78
Fixed schedule of Q1n/a29.34
Spectral gate (ReSPect)26.1430.57

n/a: the schedule of Q1 always has 27 evaluations. (Schedule comparison: App. K, summarized in Sec. 4.4.) The revised paper adds the bound and this check to Sec. 3.2 (proof: App. J).

W2: SADA, SeaCache, Spectrum. Both decide when to skip and what to substitute, at different levels: SADA from previous model outputs, substituting at the solver level (Lagrange polynomials on a fixed sub-grid); ReSPect from the current block input before the network runs, forecasting the block residual at the feature level from all evaluated steps. Same harness, SADA from its released code:

Method (FLUX.1-dev)NFEPSNR↑SSIM↑LPIPS↓Speedup
SADA (faster configuration)21.525.670.8920.0992.27×
ReSPect (δ=0.37)18.126.140.8950.0922.52×
ReSPect (δ=0.3)20.827.640.9160.0712.23×
SADA (demo configuration)24.029.410.9350.0542.04×
ReSPect (δ=0.24)23.829.020.9320.0571.95×
ReSPect (δ=0.23)25.029.780.9400.0481.87×

At SADA's faster budget ReSPect is +2.0 dB higher with fewer evaluations, and even our 18-NFE setting beats it on all three metrics while being faster. At the demo budget the two are on par. SADA's code does not cover the video models. The revised paper discusses SADA in Sec. 2 and compares it in Tab. 1 and App. L. SeaCache and Spectrum are compared in Tab. 1 (FLUX.1-dev) and App. L (video).

Q1: Fixed schedule. Steps 0–9, every third step from 10 to 43, and 45–49 evaluated (10+12+5=27 of 50), Chebyshev–Taylor predictor on skipped steps, DrawBench-200:

ModelScheduleNFEPSNR↑SSIM↑LPIPS↓Speedup
FLUX.1-devFixed schedule of Q12729.340.9370.0511.76×
Spectral gate (δ=0.2)27.030.570.9470.0421.74×
FLUX.2-devFixed schedule of Q12731.580.9650.0261.81×
Spectral gate (δ=0.264)27.033.540.9730.0191.78×

At the same 27 NFE the content-adaptive gate improves ReSPect by +1.23 dB on FLUX.1-dev and +1.96 dB [+1.56, +2.38] on FLUX.2-dev (better on 78% of prompts).

Q2: Residuals vs. hidden states. Given the known input the two are closely related; residual prediction is ahead or on par in all eight image settings we ran (+0.00 to +0.11 dB; seven on FLUX.1-dev, one on FLUX.2-dev). It is also the better-posed choice: it removes the large, exactly known input from the regression, which shrinks the regression target and the constants of the ridge bound (Sec. 3.5, App. D), and it plugs into existing residual caches (TeaCache, SeaCache) with no interface change. The main gain comes from forecasting the residual instead of copying it: +0.80 / +0.43 dB on FLUX.1-dev on identical steps, +0.93 dB (blend) / +2.30 dB (local term) on Wan2.1 (App. G, H), and +0.89 / +0.64 dB on HunyuanVideo at the same gate and NFE. Sec. 3.5 now states this.

Q3: Teaser. The comparison with the unmodified model is in Figs. 3 and 5 (same prompt and seed, PSNR under each panel) and App. I (Figs. A5–A10: eleven more FLUX.1-dev prompts, ReSPect closest to the base on 19 of 20, and full video strips). The revised caption points to them.

Thank you for the thorough rebuttal and the additional experiments. My concerns have been substantially resolved.

The method demonstrates solid performance on LPIPS, SSIM, and PSNR across different models and modalities, and I consider it a valuable contribution to training-free acceleration.

I would respectfully disagree with Reviewer RNgB on one point. For training-free diffusion acceleration, I believe faithfulness metrics (LPIPS/SSIM/PSNR) should carry more weight than distributional or aesthetic metrics. This is because the goal of training-free acceleration is to reproduce the original model’s output at lower cost. Faithfulness metrics measure such deviation directly.

I therefore lean towards acceptance.

  • One note on the SADA: I could not find the “faster configuration” (2.27×) in the SADA paper or its repository, could the author state how it was obtained? Since baselines are best compared at their released settings, I would suggest treating the demo configuration (2.04×), where the two methods are on par, as the primary comparison in the revised paper.

  • One minor presentation suggestion: please use vector graphics for the main figures and increase the font size.

Reviewer_n4S3OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

ReSPect combines spectral gating with a mixture of Chebyshev regression and first-order Taylor extrapolation to predict skipped residuals. The main additions over prior work are the residual prediction target and the mixed predictor. The current evidence does not establish their independent benefit, and I recommend rejection.

Critical strengths

The method requires no additional training and covers both image and video generation. The supplement includes useful comparisons against SeaCache and ablations of the predictor components.

Critical weaknesses
  • The Chebyshev component is not justified by the ablations. In Appendix H, pure Taylor improves over copy-reuse by 2.30 dB on Wan, while the proposed blend improves by only 0.93 dB—a loss of 1.37 dB from adding Chebyshev prediction. On FLUX, the gains are nearly identical: 0.50 versus 0.49 dB. Pure Chebyshev performs worse still. These results do not support the proposed blend. A direct Spectrum comparison and a residual-versus-full-state control are also still missing.

  • The claimed analytic parameter selection remains unsupported. The implementation fixes the blend weight at 0.5 rather than computing the derived optimum. Appendix E does not justify why four observations form a valid warm-up threshold for five coefficients, or why interpolation accuracy at observed points implies better prediction at skipped points. Appendix C also does not establish the growth condition needed to keep K/(σ_min²+λ) controlled as observations accumulate; a non-decreasing σ_min alone is insufficient.

  • The claimed fixed-schedule controls are not established. Appendix G argues that an unchanged input-based gate makes the schedule independent of the reconstructor. However, reconstructed outputs affect subsequent sampling states and therefore future gate inputs. Equal thresholds or NFE do not guarantee identical evaluation steps. The component comparisons need explicitly shared evaluation positions to isolate reconstruction quality.

  • The main-table baseline provenance remains unclear. At the reported precision, the quality metrics for 17 baseline configurations match the published SeaCache tables. Yet §4.1 states that all quality and latency measurements were made on a B200, while Figure 4 refers to published baseline points. The supplement does not resolve which results were reused or remeasured, including the change from 944 to 946 video prompts. This needs clarification before the comparisons can be assessed.

Questions for authors

See weakness for details please.

2 comments
AuthorsOpenReview ↗

Response to Reviewer n4S3

Chebyshev component, Spectrum, residual vs. full state. We re-ran every reconstructor on identical evaluation steps (each prompt replays the steps ReSPect evaluated), FLUX.1-dev, DrawBench-200, PSNR:

Reconstructor (same steps)18.1 NFE13.0 NFE
Copy-reuse (SeaCache)25.3421.26
Taylor only (w=0)26.5321.86
w=0.2526.5521.80
ReSPect (w=0.5)26.1421.69
Chebyshev only (w=1)24.6621.24

We agree that the gain comes from forecasting the residual trajectory and is carried mainly by the local term, as App. H also shows on Wan2.1. The Chebyshev term adds two things: it helps at the most aggressive budget (FLUX.1-dev at 13 NFE with the first five steps evaluated, the w=0.5 blend is the best arm, +0.35 dB over Taylor only), and its degree M gives a capacity setting that a first-order forecast lacks (degree sweep: M=2→4 adds +0.55 dB). Because the best weight depends on model and budget and choosing it per setting would be test-time tuning, we fixed w=0.5 and M=4 in advance for every model. The revised paper reports the sweep in the main text (Tab. 3, with Wan2.1; App. K), presents w as a fixed default (Sec. 3.6) and states that a Taylor-dominant weight is preferable where it can be selected.

Spectrum, at matched NFE in the same harness: ReSPect is better on FLUX.1-dev at 18 NFE (26.14 vs. 25.48 dB, LPIPS 0.092 vs. 0.101) and on video by +7.86 dB [+7.68, +8.03] and +4.46 dB on Wan2.1 and +4.80 dB on HunyuanVideo (δ=0.27), on 92–100% of the 944 prompts. On Spectrum's own schedule our predictor matches or exceeds Spectrum's (25.48 vs. 25.48; 23.75 vs. 23.72 dB). Residual vs. full state: on identical steps the two targets are within 0.11 dB on the image models (26.14 vs. 26.13 dB, CI [−0.04, +0.05]; FLUX.2-dev 31.58 vs. 31.47 dB), and the hidden state is 0.26–0.30 dB ahead on Wan2.1 (Tab. 3); see our response to Reviewer DLUj, Q2.

Analytic parameter selection. We agree with all three points. Blend weight: w⋆ cannot be evaluated at inference time, so w=0.5 is a fixed default; the revised paper removes the claim that the analysis avoids a hyperparameter search and reports the sweep. Warm-up: with four observations and five coefficients the system is under-determined; the ridge penalty makes the fit well defined. The revised paper presents the warm-up as a design choice with its measured sensitivity (App. H: longer warm-ups do not hurt) and removes the interpolation argument. Ridge factor: we computed K/(σmin⁡2+λ)K/(\sigma_{\min}^2+\lambda) at every forecast on FLUX.1-dev, Wan2.1 and HunyuanVideo; it stays near its ridge limit K/λ (median 80–180, at most 206), so the bound is controlled by λ and the finite number of evaluations (K ≤ T=50), not by growth of σmin⁡\sigma_{\min}. The revised paper states the lemma in this finite-sample form (Sec. 3.4, App. N).

Fixed-schedule controls. The reviewer is right that equal thresholds do not guarantee equal steps. Measured: SeaCache and ReSPect at the same δ evaluate exactly the same steps on 97% (18 NFE) and 93% (13 NFE) of the prompts. The reconstructor comparisons on identical steps (Tab. 3, App. K) replay the recorded per-prompt steps of one reference run, so only the reconstruction differs; copy-reuse on the replayed steps gives SeaCache's PSNR (25.34 / 21.26 dB). The revised supplementary replaces the argument in App. G by this protocol (App. K).

Baseline provenance. We apologize for the unclear statement. In the submitted Tabs. 1–2 the latencies were measured by us on one B200, while the baselines' quality metrics were taken from SeaCache's published tables (also in Fig. 4). We have now re-measured every FLUX.1-dev row in one harness (same diffusers version, reference, prompts and seeds; no published number reused; Tab. 1 of the revised paper); our ReSPect values reproduce the submitted ones (26.14 vs. 26.24 and 21.69 vs. 21.63 dB). The revised Tab. 2 states the source of each video row. Prompt count: VBench's standard list has 946 entries, of which 944 are unique (two duplicates); the revised paper uses and states 944.

Dear Reviwer,

Since the other two reviewers have different opinions on the novelty and performance of this work and you also mentioned similar questions of the paper, it would be critical to hear your comments of the response. Thanks.

AC

Comments on the paper

3 comments
AuthorsOpenReview ↗

General Response to All Reviewers

We thank the reviewers for their careful and constructive reviews. All three asked, in different ways, for direct and controlled evidence. We re-ran the evaluation in a single harness and added every requested comparison and control.

What we added.

  • Re-measured baselines. On FLUX.1-dev every method (fewer steps, TeaCache, TaylorSeer, SADA, SeaCache, Spectrum, ReSPect) is run from its released code with the same model version, prompts, seeds, uncached reference and GPU; no published number is reused.
  • New baselines at matched NFE: SeaCache and Spectrum (both CVPR 2026, June 2026, within the four-month window that the ICLR 2027 Area Chair guidelines use for contemporaneous work), and SADA (ICML 2025).
  • Controlled ablations: every reconstructor on explicitly shared evaluation steps (copy-reuse, Taylor, Chebyshev, blends, residual vs. hidden state), and the same predictor on different schedules (our gate, uniform, Spectrum's, the fixed schedule requested by Reviewer DLUj).
  • New metrics and model: ImageReward and CLIP score, all 16 VBench dimensions on Wan2.1 and HunyuanVideo, paired 95% confidence intervals, and FLUX.2-dev for Reviewer DLUj's fixed-schedule question.

Key findings.

  • Better than both parent methods. At 18 NFE on FLUX.1-dev, ReSPect is the best method on PSNR, SSIM and LPIPS (26.14 dB vs. 25.34 for SeaCache and 25.48 for Spectrum), and it outperforms SeaCache at every FLUX.1-dev budget. On video, at matched NFE, it exceeds Spectrum by +7.9 and +4.5 dB on Wan2.1 and by +4.8 dB on HunyuanVideo (δ=0.27), and on HunyuanVideo it exceeds SeaCache at both budgets (+0.89 and +0.64 dB, same gate and NFE).
  • Forecasting beats copying. On identical evaluation steps, forecasting the residual improves on SeaCache's copy-reuse by +0.80 dB [+0.65, +0.95] and +0.43 dB [+0.20, +0.66] on FLUX.1-dev; with the local forecaster it improves in every controlled setting in which we ran it (up to +2.2 dB).
  • The gate matters. With the same predictor, the content-adaptive gate beats uniform spacing by 10 dB and Reviewer DLUj's fixed schedule by 1.2 dB on FLUX.1-dev and 2.0 dB on FLUX.2-dev at equal NFE.
  • Ahead of SADA: +2.0 dB at SADA's faster budget with fewer evaluations, and on par at its demo budget.
  • Quality preserved. ImageReward and CLIP score stay within the paired 95% CI of the uncached model at both budgets. On VBench (all 16 dimensions, 944 prompts, both video models) ReSPect stays within 0.2 points of the uncached model's VBench Total at every budget; all VBench tables are in App. M of the revised supplementary.

Revision. We have uploaded a revised paper and supplementary material (Apps. A–N) in which everything added or changed is shown in blue; the main text stays within 10 pages. They contain the re-measured FLUX.1-dev table with SeaCache, Spectrum and SADA (Tab. 1), SeaCache's published video rows with the source of every row (Tab. 2), the one-harness video comparison with SeaCache and Spectrum (App. L), the identical-step ablations (Tab. 3, App. K), the quality metrics (App. M), the gate–predictor analysis (Sec. 3.2, App. J; Reviewer DLUj), corrected statements on the blend weight, warm-up and ridge bound (Secs. 3.4–3.6, App. N; Reviewer n4S3) and an explicit statement of what ReSPect adopts from SeaCache and Spectrum (Reviewer RNgB). A separate comment lists every change.

Dear AC and Reviewers,

ReSPect is, in substance, SeaCache’s gate combined with Spectrum’s predictor. Neither the revision nor the general comment shows a methodological contribution beyond combining these two methods.

1. The method is a combination of two existing methods, with no gain from the additions. The revised paper itself states that the gate is SeaCache’s, unmodified, and that the forecaster is Spectrum’s. The supplementary code confirms this. util_seacache.py is identical to SeaCache’s released file. The forecaster is described in its own docstring as a “faithful reimplementation” of Spectrum’s code, and it uses Spectrum’s recommended settings. The only substantive change is the residual target, and the authors’ own ablation shows it brings no gain (App. K).

2. Contemporaneity does not create novelty. The contemporaneous rule exempts authors from comparing against recent work. It does not make a combination of that work novel. Note also that the ICLR 2027 guidelines are inconsistent on this point: the Reviewer Guidelines use two months, while the AC Guidelines use four. In any case, the rule excuses authors who could not reasonably have known about such work. Here, the authors built their method directly on both papers, so contemporaneity is irrelevant to the question of novelty.

3. The attribution in the original submission was inadequate. The original main text derived equations (6)–(9) without attributing them to SeaCache. The fact that the formula is “identical to SeaCache’s” appeared only in a supplementary figure caption. The revision corrects this, but only after the reviews raised it.

I also note that the authors posted their first response less than 24 hours before the end of the discussion period. This left reviewers little time to examine an extensive revision and new supplementary material.

I do not consider the paper suitable for ICLR.

Dear AC and reviewers,

We thank Reviewer RNgB for the summary and respond briefly to each point.

  1. Combination of two methods. The revised paper states that the gate is SeaCache's and the forecaster Spectrum's, and the supplementary code makes this easy to check. Methods in this line of work are built from shared parts: TaylorSeer, HiCache and Spectrum all forecast cached features with a polynomial and differ mainly in the basis (Taylor, Hermite, Chebyshev) and in how it is fitted, while SADA applies Lagrange interpolation at the solver level. Each is a contribution because its design choice was shown to matter, and we ask that ReSPect be assessed by the same standard. The choice we claim is not the residual target, which we agree brings no gain, but forecasting the cached residual instead of copying it, on the gate's content-adaptive steps. Its effect is measured on identical steps and at matched NFE: +0.80 / +0.43 dB over SeaCache's copy-reuse on FLUX.1-dev and +0.89 / +0.64 dB on HunyuanVideo; +0.66 dB over Spectrum's schedule with the same forecaster, and +7.9 / +4.5 dB on Wan2.1 and +4.8 dB on HunyuanVideo (δ=0.27) over Spectrum. Details are in our reply to the reviewer's comment.
  2. Contemporaneity. We agree that contemporaneity does not create novelty. We mentioned it only because the comparison with SeaCache and Spectrum was missing from the submission; the revision adds it, with both run from their released code in one harness.
  3. Attribution in the submission. We agree that the original main text should have attributed the gate equations (Eqs. 6–9) to SeaCache. The revision does so in Sec. 3.2 and states which parts are adopted in the abstract and introduction.

Timing. We apologize for the late response. Re-running every baseline in one harness, adding SeaCache, Spectrum and SADA, and evaluating the video models on 944 prompts with all 16 VBench dimensions took most of the discussion period.

We leave the judgment of whether this contribution suffices for ICLR to the AC and reviewers, and thank all reviewers for their time.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.