WHEN TO SYNCHRONIZE, HOW MUCH TO REPAIR: CACHE CONTROL FOR AUTOREGRESSIVE VIDEO DIFFUSION

Autoregressive (AR) video generation makes long-horizon visual synthesis possible, but every generated chunk requires repeated forward passes through a large diffusion transformer (DiT). Training-free caching reduces this cost by reusing intermediate computation. However, existing cache controllers often couple two decisions: when the cache is synchronized exactly and how much approximate computation is spent in between. Our characterization suggests that these decisions should be separated. For reactive staleness signals, exact synchronization should be scheduled before measured drift accumulates, while measured drift should decide only how much repair to spend between synchronizations. Based on this separation, we introduce Frequency-Step Cache (FSC), a training-free controller that performs exact synchronization at full-computation anchors on a dense-to-sparse schedule and adaptive repair between them. A lightweight latent-space risk combining frequency-resolved drift with token age routes each non-anchor step from whole-step replay to selective token repair or full repair without changing the synchronization schedule. We further show on SkyReels-V2 that whole-step reuse provides the main acceleration, while selective token computation is most effective as fidelity repair. In a controlled ablation, using one reactive signal for both synchronization and repair reduces fidelity at matched latency, with the separation advantage also observed on FramePack and Matrix-Game. Across multiple AR video backbones, FSC provides favorable speed–quality trade-offs. On SkyReels-V2, FSC-Speed reaches a 2.36× speedup, above MotionCache’s 2.275×, while improving SSIM by 0.019 and PSNR by 0.5 dB. At MotionCache’s speed, FSC-Quality improves SSIM by 0.031 and PSNR by 1.0 dB, while lowering LPIPS by 0.024. Without retuning, FSC-Quality transfers to the 14B SkyReels-V2 model at 2.12× speedup, compared with 1.79× for MotionCache.