- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes PACE, a training-free cache scheduling method for accelerating diffusion models. The key idea is to use Fourier phase dynamics of the predicted clean signal to identify structure-sensitive denoising steps and allocate full-computation anchors according to an equal-mass phase-dispersion schedule. The method is compatible with both reuse- and prediction-based caching without additional runtime overhead. Experiments on FLUX.1-dev, SiT-XL/2, Wan2.1-1.3B, and HunyuanVideo-13B show improved fidelity over existing caching baselines under comparable NFE and latency budgets.
Critical strengths
-
The paper provides an interesting and well-motivated observation that cache-induced structural corruption is more directly associated with Fourier phase than amplitude. The phase-swap experiments and perturbation analysis give a meaningful explanation for why certain denoising steps are particularly sensitive to caching.
-
The problem formulation, motivation, and proposed scheduling strategy are well aligned, resulting in a coherent overall contribution. The method also shows strong performance across multiple image and video generation models under matched NFE budgets; assuming the experimental protocol is sufficiently fair and robust, these results demonstrate the practical effectiveness of the proposed scheduler.
Critical weaknesses
-
The technical novelty of the frequency-based scheduling criterion appears limited. Frequency-aware information has already been considered in prior caching methods such as SeaCache, which introduces a spectral prior into its gating mechanism. In PACE, the phase-dispersion rate is constructed mainly from standard Fourier coefficients, second-order finite differences, and amplitude normalization, followed by equal-mass partitioning. While the formulation is clean, the paper should better clarify what fundamentally distinguishes this criterion from existing spectral caching strategies.
-
Moreover, the central observation that phase is more indicative of cache-induced structural corruption than amplitude is motivated convincingly by the presented examples, but broader quantitative evidence across models, prompts, and failure cases would strengthen this claim.
-
The large improvements in PSNR do not translate into similarly substantial gains in reference-free generation quality. In particular, the VBench results are reported to remain very close across different schedules, despite the considerable PSNR improvements of PACE. This raises the question of whether PACE mainly improves fidelity to the full-sampling trajectory rather than perceptual or semantic generation quality. In addition, the evaluation could include more challenging acceleration regimes and a clearer quality–latency trade-off analysis, e.g., stronger step-reduction baselines at comparable runtime, to better demonstrate the practical advantage of the proposed scheduling strategy.
Questions for authors
Q1. How does the proposed phase-dispersion criterion fundamentally differ from the frequency-aware mechanism in SeaCache beyond using Fourier phase/curvature as the scheduling signal? Since SeaCache already incorporates a spectral prior into its gating decision, a clearer explanation of the methodological distinction would help assess the novelty of PACE.
Q2. Can the authors provide broader quantitative evidence supporting the central claim that Fourier phase is more strongly associated with cache-induced structural corruption than amplitude? In particular, it would be helpful to verify whether this observation consistently holds across different backbones, prompts, and generation settings rather than mainly through the illustrative phase-swap examples.
Q3. PACE shows very large improvements in PSNR, whereas the gains in reference-free metrics such as VBench appear relatively limited. Can the authors clarify why this discrepancy occurs and provide a more explicit quality–latency trade-off comparison, including stronger vanilla step-reduction baselines under matched wall-clock latency or more aggressive NFE budgets? This would help determine whether PACE mainly improves fidelity to the full-sampling trajectory or also improves practical generation quality.
7 comments
W2: The claim that phase is more indicative of structural corruption needs broader quantitative support.
We provide two pieces of evidence across all four backbones:
(i) To directly quantify whether phase is more strongly associated with structural corruption than amplitude, we ran a controlled per-step intervention experiment on FLUX.1-dev (4 prompts, 512×512). At each of 11 probe time steps , we corrupt the prediction in the latent frequency domain in two ways — replacing in-band phase with uniform random values while keeping amplitude (phase corruption), or shuffling in-band amplitudes while keeping phase (amplitude corruption) — then continue the remaining denoising steps unchanged and measure LPIPS of the final output against the clean reference.
| Denoising step | Phase corruption LPIPS↑ | Amplitude corruption LPIPS↑ | Ratio | |
|---|---|---|---|---|
| 0 | 1.000 | 1.046 | 1.049 | 1.0× |
| 5 | 0.944 | 0.633 | 0.091 | 7.0× |
| 10 | 0.883 | 0.138 | 0.040 | 3.5× |
| 15 | 0.814 | 0.078 | 0.026 | 3.0× |
| 20 | 0.738 | 0.058 | 0.016 | 3.6× |
| 25 | 0.653 | 0.039 | 0.009 | 4.3× |
| 30 | 0.556 | 0.034 | 0.007 | 4.9× |
| 35 | 0.446 | 0.029 | 0.006 | 4.8× |
| 40 | 0.320 | 0.024 | 0.004 | 6.0× |
| 45 | 0.173 | 0.019 | 0.003 | 6.3× |
| 49 | 0.037 | 0.009 | 0.002 | 4.5× |
At step 0 (, pure noise) both corruptions maximally disrupt the output, as expected. At the evaluated steps from step 5 onward, phase corruption yields LPIPS values 3.0–7.0 times those of amplitude corruption. This directly quantifies the observation from the phase-swap visualisation in the paper: phase information is substantially more critical than amplitude for preserving structural content throughout the denoising trajectory. The same directional finding holds across all four backbones: on SiT-XL/2 the mean ratio across non-trivial steps is 3.9×, on Wan2.1-1.3B it is 4.1×, and on HunyuanVideo-13B it is 2.8×. The FLUX.1-dev per-step results and aggregate ratios for the other three backbones are reported in Appendix M.
(ii) The controlled ablation on FLUX.1-dev (HiCache operator, NFE=8, 199 prompts) directly compares the amplitude-only proxy against PACE under otherwise identical conditions:
| Schedule | IR↑ | PSNR↑ | SSIM↑ | LPIPS↓ |
|---|---|---|---|---|
| Amplitude-only curvature | 0.545 | 14.51 | 0.329 | 0.717 |
| PACE (ours) | 0.927 | 18.52 | 0.644 | 0.342 |
The combined complex-curvature proxy outperforms its amplitude-only counterpart by 4.01 dB PSNR and 0.382 ImageReward. This supports the usefulness of retaining phase-sensitive information. The same directional finding holds on SiT-XL/2, Wan2.1-1.3B, and HunyuanVideo-13B under the same ablation protocol (Tables A3–A4 in the revised appendix). As an implementation consistency check (not independent evidence of phase predicting cache disruption), we also verified that PACE's per-step anchor placement across 7 NFE levels correlates with at () versus () for , confirming the schedule places anchors where the phase-sensitive proxy is high as intended.
Together, the intervention experiments and controlled scheduling ablations provide strong, consistent quantitative support across all four image and video backbones for the importance of phase information. Phase corruption produces substantially larger final-output deviations than amplitude corruption, while PACE's phase-sensitive schedules outperform their amplitude-only counterparts across all evaluated metrics. These complementary findings substantiate both the relevance of phase information to preserving generated structure and its practical value for selecting computation positions, demonstrating that the phase-based insight translates into measurable improvements in cache scheduling.
W3: Same as Q3 below.
Q1: Same as W1.
Q2: Same as W2.
Q3: PSNR improvements do not translate proportionally to VBench; quality--latency trade-off is unclear.
PACE's advantages extend to reference-free quality and quality at comparable wall-clock latency. On HunyuanVideo-13B at NFE=18, PACE outperforms SeaCache on 9 of the 11 evaluated VBench dimensions and achieves a higher mean score (0.6659 vs. 0.6635); at NFE=26, it leads on 8 of 11 dimensions. PACE also matches the full-sampling Dynamic Degree score of 0.6806 at both budgets, compared with SeaCache's 0.6528 and 0.6667, respectively. These results demonstrate benefits in reference-free evaluation alongside the fidelity gains. VBench measures dimension-specific perceptual attributes, whereas PSNR, SSIM, and LPIPS measure fidelity to the full-sampling reference, so improvements in these different metrics need not be proportional. Results at both budgets are reported in Appendix J.
To quantify the quality–latency trade-off across acceleration regimes, we additionally evaluate FLUX.1-dev at NFE=8, 10, 13, 16, and 25 on 199 prompts at 1024×1024. We report ImageReward, a reference-free metric reflecting human preferences, alongside PSNR, LPIPS, and measured wall-clock latency. Vanilla denotes the standard FLUX.1-dev Euler sampler with its default shift schedule and a reduced number of sampling steps, without caching. Latency is measured on one A800 GPU with batch size 1, using torch.cuda.synchronize and averaging eight runs per method. The full 50-step reference takes 22.5 s and achieves ImageReward 0.952.
| NFE=8 | NFE=10 | NFE=13 | NFE=16 | NFE=25 | |
|---|---|---|---|---|---|
| ImageReward$\uparrow$ | |||||
| Vanilla | 0.721 | 0.727 | 0.746 | 0.751 | 0.783 |
| HiCache Uniform | 0.910 | 0.914 | 0.920 | 0.923 | 0.941 |
| SeaCache | 0.913 | 0.915 | 0.922 | 0.927 | 0.948 |
| PACE (ours) | 0.927 | 0.928 | 0.932 | 0.951 | 0.964 |
| PSNR$\uparrow$ | |||||
| Vanilla | 13.16 | 14.37 | 14.98 | 16.21 | 19.03 |
| HiCache Uniform | 15.38 | 16.13 | 18.21 | 18.65 | 21.85 |
| SeaCache | 16.86 | 18.73 | 21.12 | 24.09 | 29.16 |
| PACE (ours) | 18.52 | 20.01 | 24.49 | 26.69 | 32.02 |
| LPIPS$\downarrow$ | |||||
| Vanilla | 0.581 | 0.542 | 0.518 | 0.476 | 0.384 |
| HiCache Uniform | 0.455 | 0.384 | 0.341 | 0.297 | 0.191 |
| SeaCache | 0.409 | 0.339 | 0.294 | 0.202 | 0.106 |
| PACE (ours) | 0.342 | 0.259 | 0.151 | 0.111 | 0.047 |
| Latency (s) | |||||
| Vanilla | 3.6 | 4.5 | 5.8 | 7.2 | 11.2 |
| HiCache | 5.6 | 6.4 | 7.7 | 8.9 | 12.7 |
| SeaCache | 5.9 | 6.8 | 8.1 | 9.4 | 13.5 |
| PACE (ours) | 5.6 | 6.4 | 7.7 | 8.9 | 12.7 |
PACE achieves the best ImageReward, PSNR, and LPIPS among the methods in this sweep at every tested NFE budget. Across the five budgets, it exceeds SeaCache by 0.010–0.024 ImageReward and 1.28–3.37 dB PSNR. In our measurements, PACE matches Uniform HiCache's latency at each budget and runs faster than SeaCache, which incurs additional per-step online spectral computation.
At comparable wall-clock latency of approximately 5.6–5.9 s, PACE also outperforms a higher-step vanilla baseline. PACE at NFE=8 takes 5.6 s, achieving ImageReward 0.927, PSNR 18.52 dB, and LPIPS 0.342. Vanilla with 13 sampling steps takes 5.8 s, achieving 0.746, 14.98 dB, and 0.518, respectively; SeaCache at NFE=8 takes 5.9 s, achieving 0.913, 16.86 dB, and 0.409. Thus, PACE improves PSNR by 3.54 dB over Vanilla and 1.66 dB over SeaCache, while also delivering higher ImageReward and lower LPIPS at slightly lower latency. This directly addresses the requested comparison with a stronger step-reduction baseline at comparable runtime.
Together, the VBench evaluation and NFE sweep demonstrate that PACE improves both reference-free quality metrics and fidelity to full sampling, with a favorable quality–latency trade-off across the tested settings. The comparison with 13-step Vanilla further shows that PACE's scheduling advantage remains when the baseline spends a comparable runtime on more sampling steps. The sweep and latency comparisons are reported in Appendix O, Table A6, and Figure A6.
We hope that our responses adequately address the reviewer's concerns. We would be glad to hear any further feedback.
We appreciate the careful reading and the recognition of the paper's coherent problem formulation. Here are our point-by-point responses.
W1: The technical novelty of the frequency-based criterion appears limited relative to SeaCache.
PACE differs fundamentally from SeaCache in three key aspects: (1) spectral criterion formulation, (2) computation allocation strategy, and (3) scheduling mode. These distinctions enable PACE to identify structurally sensitive stages, allocate a fixed computation budget across the entire trajectory, and avoid online spectral decision overhead.
1. Spectral criterion formulation. PACE aims to identify stages where skipping computation can disrupt structural development. It constructs an amplitude-normalized sensitivity profile from the complex second differences of structure-band Fourier coefficients of the predicted clean signal x^1. SeaCache uses its spectral prior to assess changes in intermediate representations and decide whether to refresh the cache. PACE instead characterizes structural evolution along the trajectory independently of the current cached state. This provides a common sensitivity profile through which different stages can be compared, enabling computation to be prioritized for structurally sensitive intervals under a limited budget.
2. Computation allocation strategy. PACE addresses how a prescribed computation budget should be distributed across the entire denoising trajectory. It jointly determines anchor positions by placing them at equal intervals of cumulative dispersion mass. Theorem 1 establishes that equal-mass partitioning minimizes the maximum interval mass and separable convex interval costs under the proxy objective. SeaCache makes refresh decisions step by step from runtime spectral evidence; such local gating does not itself provide an objective for global allocation under a prescribed NFE budget. PACE therefore provides a theoretically grounded allocation rule that balances accumulated proxy sensitivity across cache intervals, placing denser anchors in sensitive regions while maintaining the specified budget.
3. Scheduling mode. PACE performs calibration and schedule construction offline to avoid repeated spectral decision overhead during generation. Using six full-sampling calibration trajectories per backbone, it computes the sensitivity profile and constructs schedules before inference, allowing the calibration cost to be amortized across subsequent generations. Inference requires only an anchor-table lookup, whereas SeaCache evaluates its spectral gate online. This difference also yields a practical latency advantage in our matched-budget evaluation: on FLUX.1-dev at NFE=8, PACE with HiCache takes 5.6 s versus SeaCache's 5.9 s on one A800 GPU.
PACE therefore contributes a distinct methodological formulation: a sensitivity profile derived from the structural evolution of predicted clean signals, coupled with a theoretically grounded rule for global computation allocation under a fixed budget. Its offline construction further enables inference without online spectral evaluation. The matched-budget results across four image and video backbones demonstrate the practical value of this formulation, combining strong fidelity gains with efficient inference.