PACE: Phase-Dispersion-Aware Cache Scheduling for Accelerating Diffusion Models

Feature caching accelerates Diffusion Transformers (DiTs) by reusing or forecast- ing internal features across denoising steps, but the placement of the few steps that are still fully computed, the cache schedule, is usually set by fixed intervals or by generic feature distances. Both criteria are blind to what is changing in the image and can skip the brief window in which global structure first forms; the result- ing errors are never repaired by later steps. We identify this window spectrally: phase-swap and perturbation analyses show that cache-induced structural corrup- tion is carried by the Fourier phase, not the amplitude, of the model’s predicted clean signal. Building on this, we propose PACE, a scheduler that calibrates, once per backbone and without training, a phase-dispersion profile measuring how fast the in-band spectral coefficients bend along the trajectory, and places computa- tion anchors at equal quantiles of its cumulative mass. This equal-mass rule is the minimax- and convex-cost-optimal partition of the calibrated profile, is prompt- invariant after normalization, and plugs into reuse- and prediction-based caches with an anchor lookup and no runtime cost. Under identical numbers of function evaluations, PACE improves fidelity to full sampling on FLUX.1-dev, SiT-XL/2, Wan2.1-1.3B, and HunyuanVideo-13B; on video generation it raises PSNR over the strongest baseline, SeaCache, by 6.5–13.6 dB while matching its latency, and it is the only schedule that recovers the motion dynamics of full sampling.