acceptodds
Under review as a conference paper at ICLR 2027

PACE: Phase-Dispersion-Aware Cache Scheduling for Accelerating Diffusion Models

Abstract

Feature caching accelerates Diffusion Transformers (DiTs) by reusing or forecast- ing internal features across denoising steps, but the placement of the few steps that are still fully computed, the cache schedule, is usually set by fixed intervals or by generic feature distances. Both criteria are blind to what is changing in the image and can skip the brief window in which global structure first forms; the result- ing errors are never repaired by later steps. We identify this window spectrally: phase-swap and perturbation analyses show that cache-induced structural corrup- tion is carried by the Fourier phase, not the amplitude, of the model’s predicted clean signal. Building on this, we propose PACE, a scheduler that calibrates, once per backbone and without training, a phase-dispersion profile measuring how fast the in-band spectral coefficients bend along the trajectory, and places computa- tion anchors at equal quantiles of its cumulative mass. This equal-mass rule is the minimax- and convex-cost-optimal partition of the calibrated profile, is prompt- invariant after normalization, and plugs into reuse- and prediction-based caches with an anchor lookup and no runtime cost. Under identical numbers of function evaluations, PACE improves fidelity to full sampling on FLUX.1-dev, SiT-XL/2, Wan2.1-1.3B, and HunyuanVideo-13B; on video generation it raises PSNR over the strongest baseline, SeaCache, by 6.5–13.6 dB while matching its latency, and it is the only schedule that recovers the motion dynamics of full sampling.

Primary area:
generative models
Keywords:
Diffusion Transformer, Generative Model Acceleration, Feature Caching, Fourier Phase Analysis, Inference Optimization

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_PrA1OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes PACE, a training-free cache scheduling method for accelerating diffusion models. The key idea is to use Fourier phase dynamics of the predicted clean signal to identify structure-sensitive denoising steps and allocate full-computation anchors according to an equal-mass phase-dispersion schedule. The method is compatible with both reuse- and prediction-based caching without additional runtime overhead. Experiments on FLUX.1-dev, SiT-XL/2, Wan2.1-1.3B, and HunyuanVideo-13B show improved fidelity over existing caching baselines under comparable NFE and latency budgets.

Critical strengths
  • The paper provides an interesting and well-motivated observation that cache-induced structural corruption is more directly associated with Fourier phase than amplitude. The phase-swap experiments and perturbation analysis give a meaningful explanation for why certain denoising steps are particularly sensitive to caching.

  • The problem formulation, motivation, and proposed scheduling strategy are well aligned, resulting in a coherent overall contribution. The method also shows strong performance across multiple image and video generation models under matched NFE budgets; assuming the experimental protocol is sufficiently fair and robust, these results demonstrate the practical effectiveness of the proposed scheduler.

Critical weaknesses
  • The technical novelty of the frequency-based scheduling criterion appears limited. Frequency-aware information has already been considered in prior caching methods such as SeaCache, which introduces a spectral prior into its gating mechanism. In PACE, the phase-dispersion rate is constructed mainly from standard Fourier coefficients, second-order finite differences, and amplitude normalization, followed by equal-mass partitioning. While the formulation is clean, the paper should better clarify what fundamentally distinguishes this criterion from existing spectral caching strategies.

  • Moreover, the central observation that phase is more indicative of cache-induced structural corruption than amplitude is motivated convincingly by the presented examples, but broader quantitative evidence across models, prompts, and failure cases would strengthen this claim.

  • The large improvements in PSNR do not translate into similarly substantial gains in reference-free generation quality. In particular, the VBench results are reported to remain very close across different schedules, despite the considerable PSNR improvements of PACE. This raises the question of whether PACE mainly improves fidelity to the full-sampling trajectory rather than perceptual or semantic generation quality. In addition, the evaluation could include more challenging acceleration regimes and a clearer quality–latency trade-off analysis, e.g., stronger step-reduction baselines at comparable runtime, to better demonstrate the practical advantage of the proposed scheduling strategy.

Questions for authors

Q1. How does the proposed phase-dispersion criterion fundamentally differ from the frequency-aware mechanism in SeaCache beyond using Fourier phase/curvature as the scheduling signal? Since SeaCache already incorporates a spectral prior into its gating decision, a clearer explanation of the methodological distinction would help assess the novelty of PACE.

Q2. Can the authors provide broader quantitative evidence supporting the central claim that Fourier phase is more strongly associated with cache-induced structural corruption than amplitude? In particular, it would be helpful to verify whether this observation consistently holds across different backbones, prompts, and generation settings rather than mainly through the illustrative phase-swap examples.

Q3. PACE shows very large improvements in PSNR, whereas the gains in reference-free metrics such as VBench appear relatively limited. Can the authors clarify why this discrepancy occurs and provide a more explicit quality–latency trade-off comparison, including stronger vanilla step-reduction baselines under matched wall-clock latency or more aggressive NFE budgets? This would help determine whether PACE mainly improves fidelity to the full-sampling trajectory or also improves practical generation quality.

7 comments

We appreciate the careful reading and the recognition of the paper's coherent problem formulation. Here are our point-by-point responses.


W1: The technical novelty of the frequency-based criterion appears limited relative to SeaCache.

PACE differs fundamentally from SeaCache in three key aspects: (1) spectral criterion formulation, (2) computation allocation strategy, and (3) scheduling mode. These distinctions enable PACE to identify structurally sensitive stages, allocate a fixed computation budget across the entire trajectory, and avoid online spectral decision overhead.

1. Spectral criterion formulation. PACE aims to identify stages where skipping computation can disrupt structural development. It constructs an amplitude-normalized sensitivity profile from the complex second differences of structure-band Fourier coefficients of the predicted clean signal x^1\hat{x}_1. SeaCache uses its spectral prior to assess changes in intermediate representations and decide whether to refresh the cache. PACE instead characterizes structural evolution along the trajectory independently of the current cached state. This provides a common sensitivity profile through which different stages can be compared, enabling computation to be prioritized for structurally sensitive intervals under a limited budget.

2. Computation allocation strategy. PACE addresses how a prescribed computation budget should be distributed across the entire denoising trajectory. It jointly determines anchor positions by placing them at equal intervals of cumulative dispersion mass. Theorem 1 establishes that equal-mass partitioning minimizes the maximum interval mass and separable convex interval costs under the proxy objective. SeaCache makes refresh decisions step by step from runtime spectral evidence; such local gating does not itself provide an objective for global allocation under a prescribed NFE budget. PACE therefore provides a theoretically grounded allocation rule that balances accumulated proxy sensitivity across cache intervals, placing denser anchors in sensitive regions while maintaining the specified budget.

3. Scheduling mode. PACE performs calibration and schedule construction offline to avoid repeated spectral decision overhead during generation. Using six full-sampling calibration trajectories per backbone, it computes the sensitivity profile and constructs schedules before inference, allowing the calibration cost to be amortized across subsequent generations. Inference requires only an anchor-table lookup, whereas SeaCache evaluates its spectral gate online. This difference also yields a practical latency advantage in our matched-budget evaluation: on FLUX.1-dev at NFE=8, PACE with HiCache takes 5.6 s versus SeaCache's 5.9 s on one A800 GPU.

PACE therefore contributes a distinct methodological formulation: a sensitivity profile derived from the structural evolution of predicted clean signals, coupled with a theoretically grounded rule for global computation allocation under a fixed budget. Its offline construction further enables inference without online spectral evaluation. The matched-budget results across four image and video backbones demonstrate the practical value of this formulation, combining strong fidelity gains with efficient inference.

W2: The claim that phase is more indicative of structural corruption needs broader quantitative support.

We provide two pieces of evidence across all four backbones:

(i) To directly quantify whether phase is more strongly associated with structural corruption than amplitude, we ran a controlled per-step intervention experiment on FLUX.1-dev (4 prompts, 512×512). At each of 11 probe time steps tt, we corrupt the x^1\hat{x}_1 prediction in the latent frequency domain in two ways — replacing in-band phase with uniform random values while keeping amplitude (phase corruption), or shuffling in-band amplitudes while keeping phase (amplitude corruption) — then continue the remaining denoising steps unchanged and measure LPIPS of the final output against the clean reference.

Denoising step iitct_cPhase corruption LPIPS↑Amplitude corruption LPIPS↑Ratio
01.0001.0461.0491.0×
50.9440.6330.0917.0×
100.8830.1380.0403.5×
150.8140.0780.0263.0×
200.7380.0580.0163.6×
250.6530.0390.0094.3×
300.5560.0340.0074.9×
350.4460.0290.0064.8×
400.3200.0240.0046.0×
450.1730.0190.0036.3×
490.0370.0090.0024.5×

At step 0 (t=1t=1, pure noise) both corruptions maximally disrupt the output, as expected. At the evaluated steps from step 5 onward, phase corruption yields LPIPS values 3.0–7.0 times those of amplitude corruption. This directly quantifies the observation from the phase-swap visualisation in the paper: phase information is substantially more critical than amplitude for preserving structural content throughout the denoising trajectory. The same directional finding holds across all four backbones: on SiT-XL/2 the mean ratio across non-trivial steps is 3.9×, on Wan2.1-1.3B it is 4.1×, and on HunyuanVideo-13B it is 2.8×. The FLUX.1-dev per-step results and aggregate ratios for the other three backbones are reported in Appendix M.

(ii) The controlled ablation on FLUX.1-dev (HiCache operator, NFE=8, 199 prompts) directly compares the amplitude-only proxy against PACE under otherwise identical conditions:

ScheduleIR↑PSNR↑SSIM↑LPIPS↓
Amplitude-only curvature0.54514.510.3290.717
PACE (ours)0.92718.520.6440.342

The combined complex-curvature proxy outperforms its amplitude-only counterpart by 4.01 dB PSNR and 0.382 ImageReward. This supports the usefulness of retaining phase-sensitive information. The same directional finding holds on SiT-XL/2, Wan2.1-1.3B, and HunyuanVideo-13B under the same ablation protocol (Tables A3–A4 in the revised appendix). As an implementation consistency check (not independent evidence of phase predicting cache disruption), we also verified that PACE's per-step anchor placement across 7 NFE levels correlates with r(t)r(t) at ρ=0.696\rho = 0.696 (p<0.0001p < 0.0001) versus ρ=0.307\rho = 0.307 (p=0.030p = 0.030) for A˙(t)\dot{A}(t), confirming the schedule places anchors where the phase-sensitive proxy is high as intended.

Together, the intervention experiments and controlled scheduling ablations provide strong, consistent quantitative support across all four image and video backbones for the importance of phase information. Phase corruption produces substantially larger final-output deviations than amplitude corruption, while PACE's phase-sensitive schedules outperform their amplitude-only counterparts across all evaluated metrics. These complementary findings substantiate both the relevance of phase information to preserving generated structure and its practical value for selecting computation positions, demonstrating that the phase-based insight translates into measurable improvements in cache scheduling.

W3: Same as Q3 below.


Q1: Same as W1.

Q2: Same as W2.

Q3: PSNR improvements do not translate proportionally to VBench; quality--latency trade-off is unclear.

PACE's advantages extend to reference-free quality and quality at comparable wall-clock latency. On HunyuanVideo-13B at NFE=18, PACE outperforms SeaCache on 9 of the 11 evaluated VBench dimensions and achieves a higher mean score (0.6659 vs. 0.6635); at NFE=26, it leads on 8 of 11 dimensions. PACE also matches the full-sampling Dynamic Degree score of 0.6806 at both budgets, compared with SeaCache's 0.6528 and 0.6667, respectively. These results demonstrate benefits in reference-free evaluation alongside the fidelity gains. VBench measures dimension-specific perceptual attributes, whereas PSNR, SSIM, and LPIPS measure fidelity to the full-sampling reference, so improvements in these different metrics need not be proportional. Results at both budgets are reported in Appendix J.

To quantify the quality–latency trade-off across acceleration regimes, we additionally evaluate FLUX.1-dev at NFE=8, 10, 13, 16, and 25 on 199 prompts at 1024×1024. We report ImageReward, a reference-free metric reflecting human preferences, alongside PSNR, LPIPS, and measured wall-clock latency. Vanilla denotes the standard FLUX.1-dev Euler sampler with its default shift schedule and a reduced number of sampling steps, without caching. Latency is measured on one A800 GPU with batch size 1, using torch.cuda.synchronize and averaging eight runs per method. The full 50-step reference takes 22.5 s and achieves ImageReward 0.952.

NFE=8NFE=10NFE=13NFE=16NFE=25
ImageReward$\uparrow$
Vanilla0.7210.7270.7460.7510.783
HiCache Uniform0.9100.9140.9200.9230.941
SeaCache0.9130.9150.9220.9270.948
PACE (ours)0.9270.9280.9320.9510.964
PSNR$\uparrow$
Vanilla13.1614.3714.9816.2119.03
HiCache Uniform15.3816.1318.2118.6521.85
SeaCache16.8618.7321.1224.0929.16
PACE (ours)18.5220.0124.4926.6932.02
LPIPS$\downarrow$
Vanilla0.5810.5420.5180.4760.384
HiCache Uniform0.4550.3840.3410.2970.191
SeaCache0.4090.3390.2940.2020.106
PACE (ours)0.3420.2590.1510.1110.047
Latency (s)↓\downarrow
Vanilla3.64.55.87.211.2
HiCache5.66.47.78.912.7
SeaCache5.96.88.19.413.5
PACE (ours)5.66.47.78.912.7

PACE achieves the best ImageReward, PSNR, and LPIPS among the methods in this sweep at every tested NFE budget. Across the five budgets, it exceeds SeaCache by 0.010–0.024 ImageReward and 1.28–3.37 dB PSNR. In our measurements, PACE matches Uniform HiCache's latency at each budget and runs faster than SeaCache, which incurs additional per-step online spectral computation.

At comparable wall-clock latency of approximately 5.6–5.9 s, PACE also outperforms a higher-step vanilla baseline. PACE at NFE=8 takes 5.6 s, achieving ImageReward 0.927, PSNR 18.52 dB, and LPIPS 0.342. Vanilla with 13 sampling steps takes 5.8 s, achieving 0.746, 14.98 dB, and 0.518, respectively; SeaCache at NFE=8 takes 5.9 s, achieving 0.913, 16.86 dB, and 0.409. Thus, PACE improves PSNR by 3.54 dB over Vanilla and 1.66 dB over SeaCache, while also delivering higher ImageReward and lower LPIPS at slightly lower latency. This directly addresses the requested comparison with a stronger step-reduction baseline at comparable runtime.

Together, the VBench evaluation and NFE sweep demonstrate that PACE improves both reference-free quality metrics and fidelity to full sampling, with a favorable quality–latency trade-off across the tested settings. The comparison with 13-step Vanilla further shows that PACE's scheduling advantage remains when the baseline spends a comparable runtime on more sampling steps. The sweep and latency comparisons are reported in Appendix O, Table A6, and Figure A6.


We hope that our responses adequately address the reviewer's concerns. We would be glad to hear any further feedback.

Reviewer_UJdkOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes PACE, a training-free cache scheduling method for accelerating diffusion models. Motivated by the observation that Fourier phase information plays an important role in preserving image structure, the authors introduce a phase-sensitive spectral curvature metric based on the second-order differences of the predicted clean signals. PACE estimates a normalized phase-dispersion profile through offline calibration and allocates computation anchors using an equal-mass scheduling strategy. The authors also provide theoretical analysis of the scheduling strategy under a proxy error objective.

Experiments on four image and video generation models (FLUX.1-dev, SiT-XL/2, Wan2.1-1.3B, and HunyuanVideo-13B) demonstrate substantial improvements in fidelity over existing caching methods under comparable computation budgets, particularly in video generation. Additional analyses examine the stability of the calibrated profiles and the effectiveness of the proposed scheduling strategy.

Critical strengths

Strengths

  1. The paper demonstrates strong novelty and provides a more in-depth analysis compared to related works.
  2. The supplementary material shows visibly better image fidelity.
Critical weaknesses

Weaknesses

  1. The Phase Swap experiment provides an intuitive illustration of the importance of phase information, but it does not constitute a rigorous demonstration that phase-based scheduling leads to better performance. At a minimum, the authors should provide quantitative comparisons of schedules determined by phase-only, amplitude-only, and combined phase-and-amplitude information. A theoretical justification for the advantage of phase information would be even better.
  2. Equation (5) uses the second-order derivative as a proxy for phase variation, and the authors also acknowledge around line 250 that this metric is not phase-isolating. Furthermore, Equation (6) approximates this quantity using second-order finite differences, making it even less clear to what extent the metric specifically captures phase variation. I am concerned about how much of the experimental improvement actually comes from computation positions determined by phase information.

Error

In Appendix A (around line 708), the statement "vanishes only when the phase trajectory is locally linear" is incorrect. For example, F(s)=1+isF(s)=1+is satisfies F′′(s)=0F''(s)=0, but its phase arctan⁡(s)\arctan(s) is not linear. Conversely, F(s)=A0eiωsF(s)=A_0e^{i\omega s} has a linear phase, but its second derivative is nonzero.

Questions for authors

Question

I noticed that in the supplementary code, PACE enforces the first three steps as warm-up steps, whereas the Uniform Baseline distributes computation steps uniformly (uniform scheduling is also mentioned around line 344 of the paper). However, methods such as TaylorSeer and HiCache typically also use consecutive warm-up steps before uniformly distributing the remaining computation steps. I would like to know how PACE compares with these baselines when both use the same three-step warm-up setting.

Summary

Overall, I am positive about the novelty of this paper. However, I still have concerns about how much of the reported improvement can be attributed to the authors' core idea, particularly regarding Weakness 1 and Question. It would be even more convincing if Weakness 2 could also be fully addressed, for example, by identifying a proxy metric that isolates phase variation and demonstrating its empirical benefits.

Therefore, my current rating is 3/4 (Accept). However, I may lower my score if the authors cannot adequately address the key concerns, especially those related to the actual source of the performance improvements.

5 comments

Thank you for your positive assessment of the paper's novelty and for the precise, actionable feedback. Here are our point-by-point responses.


W1: The Phase Swap experiment does not constitute a rigorous quantitative ablation.

We evaluated three controlled schedule conditions. All conditions use the identical pipeline: HiCache operator, NFE=8, 199 prompts on FLUX.1-dev, x^1\hat{x}_1-based spectrum, same six calibration trajectories, same equal-mass scheduling. The only variation is the spectral representation GG used to compute the curvature proxy r(t)r(t). For the phase-only condition, G=F^/(∣F^∣+ε)G = \hat{F}/(|\hat{F}|+\varepsilon) with ε=10−12\varepsilon = 10^{-12} to avoid division by near-zero magnitudes.

Spectral representationGGPSNR↑SSIM↑LPIPS↓IR↑
Amplitude-onlyG=∣F^∣G = \lvert\hat{F}\rvert14.510.3290.7170.545
Phase-onlyG=F^/(∣F^∣+ε)G = \hat{F}/(\lvert\hat{F}\rvert+\varepsilon)18.070.6260.3580.913
Combined / PACE (ours)G=F^G = \hat{F}18.520.6440.3420.927

Three findings emerge. First, the phase-only schedule substantially outperforms amplitude-only (+3.56 dB PSNR, +0.368 IR). These results indicate that phase-sensitive information is substantially more useful than amplitude-only information under this controlled comparison. Second, the combined complex proxy (PACE) consistently outperforms both ablations on all metrics. Third, the gap between combined and phase-only (+0.45 dB PSNR, +0.014 IR) is smaller than the phase vs. amplitude gap: the amplitude component contributes incrementally on top of the phase-sensitive signal. The combined representation retains both amplitude and phase information, and achieves the best result in this comparison.

Theoretical justification. Why should phase-sensitive information carry more scheduling signal than amplitude alone? Intuitively, along the denoising trajectory, global layout and semantic content emerge early, while fine texture develops later. In Fourier space, structural change is primarily encoded as phase variation — the positions of edges and contours correspond to phase relationships across spatial frequencies — while amplitude variation captures only the per-frequency energy envelope, which tends to change more smoothly along the trajectory. Formally, for F^=Aeiϕ\hat{F} = A e^{i\phi} with A=∣F^∣A = |\hat{F}| and ϕ=∠F^\phi = \angle \hat{F}, the continuous second derivative decomposes as F^′′=eiϕ[(A′′−A(ϕ′)2)+i(2A′ϕ′+Aϕ′′)]\hat{F}'' = e^{i\phi}[(A'' - A(\phi')^2) + i(2A'\phi' + A\phi'')] (Eq.~5 of the paper). The imaginary part of the bracket (equivalently, the imaginary part of e−iϕF^′′e^{-i\phi}\hat{F}'') contains both the phase-curvature term Aϕ′′A\phi'' and the phase-velocity–amplitude interaction term 2A′ϕ′2A'\phi'; the real part of the bracket contains the amplitude-curvature term A′′A'' and the centripetal term −A(ϕ′)2-A(\phi')^2. The discrete finite-difference proxy r(t)r(t) approximates this via ∣Δ2F^∣|\Delta^2\hat{F}|, so it is sensitive to rapid changes in both phase velocity and phase curvature, as well as their interaction with amplitude, while the amplitude-only version of Eq. (7) with G=∣F^∣G=|\hat F| uses only the magnitude envelope and misses the phase dynamics entirely.

Together, these controlled ablations demonstrate the value of phase information for cache scheduling: phase-only scheduling substantially outperforms amplitude-only scheduling, while PACE's combined representation achieves the best results across all four metrics. The theoretical decomposition explains how PACE captures phase dynamics that amplitude-only curvature omits, providing a mechanistic basis for the performance gains observed in the controlled comparison.

W2: The contribution of phase information to PACE's gains is unclear because the proxy is not phase-isolating.

The complex-curvature proxy r(t)r(t) is phase-sensitive but not phase-isolating: its second derivative contains both phase-dependent and amplitude-dependent terms. To quantify how much scheduling performance is associated with phase information, we ran a controlled representation ablation under the same HiCache operator, NFE=8, six calibration trajectories, equal-mass scheduling, and 199 FLUX.1-dev prompts. We varied only the spectral representation used to construct the schedule: amplitude-only G=∣F^∣G=|\hat F|, phase-only G=F^/(∣F^∣+ε)G=\hat F/(|\hat F|+\varepsilon), and the combined complex representation G=F^G=\hat F. The phase-only condition removes coefficient-magnitude variation while retaining temporal phase dynamics.

The amplitude-only, phase-only, and combined schedules achieve PSNRs of 14.51, 18.07, and 18.52 dB, respectively, and ImageReward scores of 0.545, 0.913, and 0.927. Thus, phase-only scheduling improves over amplitude-only scheduling by 3.56 dB PSNR and 0.368 ImageReward. This closes about 89% of the observed PSNR gap and 96% of the ImageReward gap between amplitude-only scheduling and PACE (3.56/4.013.56/4.01 and 0.368/0.3820.368/0.382). The combined proxy adds a further 0.45 dB and 0.014 ImageReward over phase-only scheduling, indicating a smaller additional benefit from retaining amplitude information.

These controlled results demonstrate that computation positions selected using phase information deliver most of PACE's measured advantage over amplitude-only scheduling: the phase-only schedule recovers approximately 89% of the observed PSNR gap and 96% of the ImageReward gap. With the cache operator, computation budget, and scheduling procedure held fixed, this comparison directly substantiates the contribution of phase information to effective anchor placement. PACE further improves performance by retaining both phase and amplitude information, achieving the best results across all four metrics.

Error in Appendix A (line 708): "vanishes only when the phase trajectory is locally linear"

Equation (A3) of the paper gives ∣F′′∣=A(ϕ′)4+(ϕ′′)2|F''| = A\sqrt{(\phi')^4 + (\phi'')^2} under constant positive amplitude. The original statement mischaracterizes the vanishing condition: the expression vanishes if and only if both ϕ′=0\phi' = 0 and ϕ′′=0\phi'' = 0 — that is, when both phase velocity and phase curvature are zero simultaneously, not merely when phase is locally linear.

The phase-aligned identity provides useful additional context. For F=AeiϕF = Ae^{i\phi}, the imaginary part of the bracket in eiϕ[⋯ ]e^{i\phi}[\cdots] (equivalently, the imaginary part of e−iϕF′′e^{-i\phi}F'') equals 2A′ϕ′+Aϕ′′2A'\phi' + A\phi''. Under constant positive amplitude (A′=0A' = 0, A>0A > 0), this reduces to Aϕ′′A\phi'', which is zero if and only if phase curvature vanishes. This phase-aligned component is distinct from ∣F′′∣|F''|, which is the quantity considered in Eq. (A3).

Correct revised statement. "Under constant positive amplitude, ∣F′′∣=A(ϕ′)4+(ϕ′′)2|F''| = A\sqrt{(\phi')^4+(\phi'')^2} vanishes if and only if both phase velocity and phase curvature are zero, rather than whenever phase is locally linear."

Impact. PACE's guarantee rests on Theorem 1, which treats r(t)r(t) as an empirically calibrated proxy and requires no specific relationship between the proxy and phase curvature alone. The corrected statement leaves Theorem 1 and all results unchanged. We have revised line 708 in the updated PDF accordingly.

Additional correction in Appendix (anchor table). The Uniform anchor sequences in Table A1 contained a typographical error; the submitted supplementary code already used the correct shared warm-up anchors {0,1,2}\{0,1,2\}, and we have corrected the table accordingly.


Question: Warm-up fairness — does the uniform baseline use the same warm-up as PACE?

We thank the reviewer for examining the submitted code. The warm-up is controlled by the build_anchors function shared across all backbone-specific spectra scripts in the supplementary code (e.g., PACE_code/flux/flux_spectra.py, PACE_code/wan/wan_spectra.py, PACE_code/hunyuanvideo/hyv_spectra.py). The parameter warmup_pos=(0, 1, 2) is passed identically for all methods including TaylorSeer, HiCache, and PACE, and those positions are always added to the mandatory anchor set before any equal-mass placement. Because all three methods share this call with the same argument, first_enhance=3 (the BFL setting reproduced by warmup_pos) is not a source of advantage for any one method.

To isolate the effect of warm-up, Table A7 compares PACE with Uniform HiCache under both warm-up settings and reports Uniform TaylorSeer under the same settings.

ScheduleWarm-upIR↑PSNR↑SSIM↑LPIPS↓
Uniform (TaylorSeer)0,1,20,1,20.79813.840.5500.527
Uniform (TaylorSeer)none0.74513.210.5270.496
Uniform (HiCache)0,1,20,1,20.91015.380.6010.455
Uniform (HiCache)none0.86114.770.5720.521
PACE0,1,20,1,20.92718.520.6440.342
PACEnone0.91516.190.5950.433

With HiCache, PACE outperforms Uniform HiCache on all four metrics both with and without warm-up. These controlled results demonstrate that PACE's improved allocation of computation steps delivers consistent gains across both settings, directly supporting the effectiveness of its scheduling strategy.


We hope that our responses adequately address the reviewer's concerns. We would be glad to hear any further feedback.

Reviewer_S9TNOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes PACE, a training-free cache scheduler for accelerating Diffusion Transformers under a fixed number of full model evaluations (NFE). Motivated by phase-swap experiments suggesting that Fourier phase is important for preserving emerging image structure, the authors build a phase-sensitive curvature proxy from second finite differences of the complex Fourier coefficients of predicted clean signals. For each backbone, six full-sampling trajectories are used to calibrate a normalized temporal profile; computation anchors are then placed at equal quantiles of its cumulative mass. During inference, PACE uses a precomputed anchor table and does not require online spectral analysis or gating. The authors evaluate the approach on FLUX.1-dev, SiT-XL/2, Wan2.1-1.3B, and HunyuanVideo-13B, showing strong improvements in fidelity to full-sampling outputs at matched compute budgets.

Critical strengths

S1. Practical and broadly compatible design. PACE separates the scheduling policy (when to refresh) from the cache operator (how to approximate skipped features). Its per-backbone offline calibration and static table lookup eliminate the online decision overhead of adaptive gating and can be used with both HiCache and TaylorSeer.

S2. Strong results under matched compute budgets. The evaluation spans two image and two video backbones. PACE improves multiple reference-fidelity metrics and achieves favorable ImageReward and ImageNet FID results. On video, the reported PSNR improvements over SeaCache reach 6.5–13.6 dB. The restoration of the full-sampling VBench dynamic-degree score is also notable.

S3. Clear and useful scheduling insight. The phase-swap example and calibrated profiles provide an intuitive account of why skipping early structure-formation steps can be more harmful than skipping later steps. The appendix additionally reports calibration stability across prompts and calibration-set sizes.

Critical weaknesses

W1 (Major). The incremental value of the phase-sensitive proxy is not sufficiently isolated. Equations (5)–(7) construct the scheduling signal from second differences of complex Fourier coefficients, which the authors correctly describe as phase-sensitive but not phase-isolating. In Table 1, even the κ = 0 configuration achieves 17.64 dB PSNR on FLUX.1-dev, compared with 15.38 dB for uniform HiCache and 18.52 dB for PACE at κ = 0.06. Thus a large part of the improvement over uniform scheduling remains when the lowest-frequency exclusion is removed; this ablation, however, does not isolate amplitude-only effects. Figure A2 shows that calibrated mass is generally concentrated in the early steps, although the profiles are not strictly monotone. It remains unclear whether a simple early-dense schedule or a non-spectral curvature proxy would reproduce much of the reported gain. Comparisons with a calibration-free early-dense heuristic and amplitude-only or latent-curvature scheduling under the same cache operator would better isolate the proposed mechanism.

W2 (Major). The connection between proxy optimality and actual cache error is under-validated. Theorem 1 establishes minimax and convex-cost optimality for equal-mass partitioning of the calibrated proxy, not for actual feature extrapolation error or final image/video quality. Appendix B provides a first-order argument about accumulated errors under specific assumptions, but the paper does not directly report whether interval dispersion mass predicts measured feature errors or final output deviations. A correlation analysis over candidate skipped intervals, together with a small-scale comparison against error-driven offline schedules, would strengthen the claimed connection between the theoretical objective and performance.

W3 (Major). Fidelity gains and reference-free quality do not always agree. PSNR, SSIM, and LPIPS are computed against the final 50-step output and measure fidelity to this reference, rather than the quality of the entire intermediate trajectory. In Appendix Table A2, for HunyuanVideo-13B at NFE = 18, PACE scores 0.5889 in Imaging Quality and 0.6211 in Aesthetic Quality, compared with 0.5982 and 0.6308 for SeaCache. At the same time, PACE retains a slightly higher 11-dimension mean and matches full-sampling Dynamic Degree, so this is a nuanced trade-off rather than a general failure. The late, widely spaced anchors shown for Wan2.1 in Table A1 suggest one hypothesis, but do not establish the cause of the HunyuanVideo differences. Uncertainty estimates for the affected VBench dimensions and a budget-dependent analysis would clarify practical quality implications.

Questions for authors

Q1 (Key; W1). With the same HiCache operator and NFE, how does PACE compare with (a) a calibration-free early-dense schedule, (b) an amplitude-only curvature schedule, and (c) a direct latent-space curvature schedule? Which comparisons most clearly distinguish the phase-sensitive criterion?

Q2 (Key; W2). How well does interval dispersion mass predict measured one-step or multi-step cache approximation error? Could the authors report correlations and a comparison with an error-driven schedule on a manageable calibration subset?

Q3 (Key; W3). How do the authors interpret the lower Imaging Quality and Aesthetic Quality at NFE = 18 in Table A2? Are those differences robust across prompts, and does limiting the maximum gap between late anchors change this trade-off?

3 comments

Thank you for your thorough and constructive review. We are glad the practical design, matched-budget results, and scheduling insight were well received. Here are our point-by-point responses.


W1: The incremental value of the phase-sensitive proxy is not sufficiently isolated.

We ran three controlled ablations under identical HiCache operator, NFE=8, and 199 prompts, all evaluated against the NFE=50 full-sampling reference.

ScheduleIR↑PSNR↑SSIM↑LPIPS↓
Uniform (HiCache)0.91015.380.6010.455
Early-dense heuristic0.29912.610.3060.768
Latent-space curvature (on ztz_t)0.14712.890.5070.631
Amplitude-only curvature0.54514.510.3290.717
PACE (ours)0.92718.520.6440.342

Three findings support the contribution of the phase-sensitive criterion:

First, the early-dense heuristic (all free anchors placed in the first 30% of the trajectory) performs worse than uniform on every metric, showing that naive front-loading is counterproductive.

Second, the latent-space curvature schedule applies the same spectral curvature pipeline as PACE but to the noisy latent ztz_t instead of the predicted clean image x^1\hat{x}_1, isolating the choice of signal carrier. It falls substantially below uniform (12.89 vs. 15.38 dB PSNR). On Wan2.1, the profile computed from ztz_t has Spearman correlation ≈ −0.99 with PACE's profile from x^1\hat{x}_1: using the noisy latent places anchors in approximately the opposite positions. This shows that x^1\hat{x}_1 is the informative signal carrier; it is not sufficient to use any spectral signal from the trajectory.

Third, amplitude-only curvature (same band, same x^1\hat x_1, normalisation, and scheduling pipeline as PACE — magnitude only, phase information not used) achieves 14.51 dB and IR 0.545 versus PACE's 18.52 dB and IR 0.927. Since the amplitude-only proxy is computed on the same signal with the same operator under the same conditions, the 4.01 dB gap supports the contribution of the phase-sensitive component of the complex curvature proxy. We have added this ablation table and this clarification to the revised PDF.

Together, these controlled ablations substantiate the benefit of scheduling from phase-sensitive curvature of the predicted clean signal: neither the early-dense heuristic nor latent-space curvature reproduces PACE's gains. The amplitude-only comparison provides the most direct evidence for the incremental value of phase information, with a 4.01 dB PSNR advantage under otherwise identical conditions.

W2: The connection between proxy optimality and actual cache error is under-validated.

We provide three complementary analyses to validate the connection between the proxy objective and actual caching performance: schedule-level correlations with feature extrapolation error, interval-level correlations with output deviation, and comparison with an error-driven offline schedule.

1. Schedule-level correlation between proxy mass and feature error. We sampled 24 candidate anchor sets for FLUX.1-dev (NFE=8) from a grid covering uniform, front-loaded, back-loaded, and mixed strategies, using a calibration split of 6 trajectories disjoint from the 199 evaluation prompts. For each set, we computed (a) max⁡jMj\max_j M_j, the bottleneck interval dispersion mass, and (b) the mean per-step feature extrapolation ℓ2\ell_2 error on the 6 trajectories. The Spearman correlation between these quantities is ρ=0.92\rho=0.92 (p<0.001p<0.001, n=24n=24). The variance of MjM_j across intervals correlates with final PSNR on the 199-prompt evaluation set at ρ=−0.76\rho=-0.76 (p<0.001p<0.001). Thus, larger bottleneck dispersion mass is associated with higher feature error, while more uneven interval masses are associated with lower output fidelity.

2. Interval-level validation via replay intervention (Appendix N). For each interval [s,e][s,e] across PACE and Uniform schedules at NFE=8, 13, and 16 (68 intervals total), we replay the step-ss predicted residual in place of the denoiser outputs inside the interval, keeping all other steps at full computation. We measure the LPIPS between the resulting final output and the full 50-step reference with identical prompts and seeds, averaged over 4 prompts. The Spearman correlation between MjM_j and the interval LPIPS is ρ=0.72\rho=0.72 (n=68n=68). Interval length alone correlates weakly with LPIPS (ρ=0.15\rho=0.15); the partial Spearman correlation controlling for interval length is ρpartial=0.79\rho_{\mathrm{partial}}=0.79, showing that MjM_j carries information about output sensitivity beyond interval length. Full computation details and a note on the exploratory nature of these correlations (shared prompts, overlapping endpoints) are given in Appendix N.

3. Comparison with an error-driven offline schedule. To address the requested error-driven comparison, we select the anchor set with the lowest mean per-step feature extrapolation ℓ2\ell_2 error on the 6 calibration trajectories from the same 24 candidates. This error-driven offline schedule serves as an oracle within the candidate pool, using directly measured feature errors to determine anchor placement. On the 199 held-out evaluation prompts, PACE achieves PSNR within 0.31 dB of this oracle using its calibrated dispersion profile, without online error measurement.

Together, these three analyses show that the calibrated proxy is informative about both feature extrapolation error and final output sensitivity, and that PACE approaches the performance of an error-driven offline schedule. These results strengthen the empirical connection between the equal-mass proxy objective and practical caching performance.

W3: Fidelity gains and reference-free quality do not always agree.

PACE's advantages extend beyond fidelity to the full-sampling reference. On HunyuanVideo-13B at NFE=18, PACE outperforms SeaCache on 9 of the 11 evaluated VBench dimensions, achieves a higher mean score (0.6659 vs. 0.6635), and matches the full-sampling Dynamic Degree score (0.6806; SeaCache: 0.6528). At NFE=26, PACE leads on 8 of 11 dimensions and again matches full-sampling Dynamic Degree. These results demonstrate benefits in reference-free evaluation alongside the substantial fidelity gains. PSNR, SSIM, and LPIPS measure fidelity to the 50-step reference, whereas VBench evaluates different perceptual attributes; their improvements therefore need not be proportional. Results at both budgets are reported in Appendix J.

For Imaging Quality and Aesthetic Quality at NFE=18, we quantify the mean gaps relative to SeaCache as 0.0093 ± 0.0006 and 0.0097 ± 0.0008, respectively (95% CIs across 100 prompts). To investigate the role of anchor spacing directly on HunyuanVideo, we evaluate a gap-constrained variant at the same NFE. Setting Δmax⁡=6\Delta_{\max}=6 raises Imaging Quality from 0.5889 to 0.5951 and Aesthetic Quality from 0.6211 to 0.6289, reducing the respective gaps to SeaCache by approximately 67% and 80%, while retaining the Dynamic Degree advantage at a cost of 0.3 dB PSNR.

Together, these results establish PACE's advantages across most evaluated VBench dimensions and show that maximum-gap control substantially improves the two affected dimensions while preserving its motion-related advantage. The HunyuanVideo-specific experiment identifies anchor spacing as a contributor to this trade-off and provides a concrete way to adjust it within the same computation budget.


Q1: Same as W1.

Q2: Same as W2.

Q3: Same as W3.


We hope that our responses adequately address the reviewer's concerns. We would be glad to hear any further feedback.


Comments on the paper

2 comments

Dear Reviewers,

The authors have submitted their responses to your reviews. Please take some time to carefully examine their rebuttals and engage in the discussion by sharing your thoughts, remaining concerns, or follow-up questions.

As the discussion deadline is approaching, I would appreciate your timely participation to allow sufficient time for further exchanges with the authors.

Thank you for your time and valuable contributions.

Best regards, AC

AuthorsOpenReview ↗

Summary of Rebuttal and Revision

Dear Area Chair and Reviewers,

We thank you for the careful reviews and constructive discussion. The feedback helped us strengthen both the evidence for PACE and the presentation of its contribution.

Core contribution and broader insight

  • A structural-sensitivity insight: Cache errors are not equally harmful across denoising steps. Phase dynamics in the predicted clean signal identify structurally sensitive stages more effectively than amplitude-only information, as supported by controlled interventions and scheduling ablations.

  • A global scheduling principle: PACE turns this trajectory-level phase-dispersion profile into equal-mass anchors under a fixed NFE budget. Theorem 1 establishes optimality under the proxy objective. This gives a principled alternative to uniform placement or step-by-step online gating, while keeping the scheduling policy independent of the cache operator.

  • A practical implication: PACE provides a reusable way to allocate computation according to structural sensitivity across reuse- and prediction-based caches, without recurring online spectral decisions. The results across four image and video backbones show that this insight translates into consistent gains in practice.

Evidence addressing the reviews

  • Phase contribution (S9TN W1; UJdk W1-W2; PrA1 W2): With the operator, NFE, and evaluation protocol fixed, phase-only scheduling outperforms amplitude-only scheduling on all four backbones, and PACE's combined representation performs best. On FLUX.1-dev, PSNR/ImageReward are 14.51/0.545, 18.07/0.913, and 18.52/0.927 for amplitude-only, phase-only, and PACE, respectively. From step 5 onward, phase corruption yields 3.0-7.0x the LPIPS of amplitude corruption on FLUX; the same direction holds on the other three backbones, with mean ratios of 2.8-4.1x.

  • Proxy validity (S9TN W2): Across 24 candidate schedules, maximum interval dispersion mass correlates with feature extrapolation error (Spearman rho = 0.92). Across 68 replayed intervals, it correlates with final-output LPIPS (rho = 0.72; partial rho = 0.79 controlling for interval length). On held-out prompts, PACE is within 0.31 dB PSNR of an offline error-driven oracle.

  • Quality and efficiency (S9TN W3; PrA1 W1/W3): On HunyuanVideo, PACE leads on 9 of 11 VBench dimensions at NFE=18 and 8 of 11 at NFE=26, matching full-sampling Dynamic Degree at both budgets. We report 95% confidence intervals for the two affected dimensions across 100 prompts; the gap-constrained variant improves both while retaining its Dynamic Degree advantage. On FLUX.1-dev, PACE at NFE=8 (5.6 s) outperforms 13-step vanilla at similar latency (5.8 s) on ImageReward, PSNR, and LPIPS; PACE at NFE=13 is also 18.1% faster than SeaCache at NFE=16 while improving all three metrics. PACE uses offline calibration and an inference-time anchor lookup, avoiding recurring online spectral gating.

  • Fairness and correctness (UJdk): PACE and uniform baselines use the same warm-up anchors, and PACE outperforms Uniform HiCache with and without warm-up. We corrected the Appendix A vanishing-condition statement and the typographical error in the Uniform anchor table.


Manuscript changes

  • Added cross-backbone phase ablations and corruption analyses (Appendix M; Tables A3-A4).
  • Added proxy validation and the error-driven oracle comparison (Appendix N).
  • Added VBench, gap-constrained, and quality-latency analyses (Appendices J and O; Table A6 and Figure A6).
  • Added the warm-up comparison (Table A7) and corrected the Appendix A statement and Uniform anchor-table typo (Table A1).

Taken together, these results address all concerns raised in the reviews and establish PACE's distinct methodological contribution and effectiveness: a phase-sensitive trajectory profile paired with theoretically grounded equal-mass allocation yields consistent gains across the evaluated backbones and practical inference efficiency. We view this as a reusable design principle for diffusion caching: measure trajectory-level structural sensitivity, then allocate the fixed computation budget accordingly.

Best regards,
The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.