acceptodds
Under review as a conference paper at ICLR 2027

One Anchor is Not Enough: Trajectory Trace Caching for Video Diffusion Acceleration

Abstract

Video diffusion transformers spend almost all of their inference cost on repeated forward passes of a large denoiser, and training-free caching reduces that cost by reusing network outputs across denoising steps. Existing caches keep only the latest output residual and reuse it as a frozen point estimate, so they ignore that this residual rotates, decays, and oscillates along the sampling path, and they trigger reuse from open-loop heuristics that are never checked against the error the reuse actually caused. The cost is largest in continuous video editing, where successive instructions are applied to one clip, every round reruns the full sampling loop, and the accelerated output of one round becomes the input of the next, so cache error is amplified rather than merely repeated. We observe that the conditional and unconditional residuals share a low-dimensional subspace whose reduced coefficients evolve almost linearly, so a skipped step becomes advancing a small state and projecting it back rather than reconstructing a high-dimensional output. We introduce **AlignVACE**, a training-free cache that maintains an online mode basis and a stability-projected reduced generator, reconstructs a whole run of skipped steps with one matrix exponential over the entire gap, and routes skips by a risk budget built from out-of-subspace energy and reduced prediction error, calibrated online against the realized error of the guided output. Because the stored trace is compressed, the operating point we use holds *less* device memory than a single uncompressed anchor. Across four editing categories, four-round edit chains, two editing backbones, and text-to-video and text-to-image generation, **AlignVACE** reaches speedup at dB on object editing and stays dB above the strongest baseline at the fourth edit round, while using GB of cache against to GB for prior caches. Our code is available at <https://anonymous.4open.science/r/AlignVACE-86BA/>.

Then back it, or bet against it.

Related papers

Open the market on this paper to see 7 more related papers.