acceptodds
Published as a conference paper at NeurIPS 2026

Conformal Cache: Reliable Proxy-Discrepancy Caching for Fast Generative Inference

Wenxiao Wu, Yachen Gao, Yukang Feng, Chengming Xu, Moran Li, jingyu li, Ming Xie, Xiaobin Hu, Xinwei Sun, Jing-Hao Xue, Nong Sang, Yanwei Fu

Abstract

While diffusion and flow-based generative models have achieved impressive performance in high-fidelity image and video synthesis, this capability comes at the cost of substantial inference overhead during iterative sampling. Cache-based acceleration alleviates this cost by reusing intermediate computations across adjacent denoising or transport steps, typically relying on lightweight proxy discrepancies to decide when cached outputs should be refreshed. However, such proxy-discrepancy rules are inherently unreliable: the proxy is only a point estimate of the inaccessible oracle discrepancy and can misalign across prompts, timesteps, and generation dynamics. This mismatch induces asymmetric failures: proxy underestimation may cause unsafe reuse of stale cached computations and degrade generation quality, while proxy overestimation leads to overly conservative refreshes and reduced acceleration. To address these issues, this paper introduces Conformal Cache, dubbed CCache, a training-free conformal calibration framework for reliable cache-based generation acceleration. Our key insight is to formulate cache reuse as a one-sided risk-control problem and adaptively calibrate a timestep-wise upper correction for the signed proxy-to-oracle gap based on split conformal prediction. CCache is plug-and-play and can be integrated with different proxy-discrepancy caching baselines without structural modifications or parameter updates to the generative model. Extensive experiments on image and video generation demonstrate that \methodname improves generation quality and cache reliability while preserving favorable speed-quality trade-offs.

Related papers

See more related submissions when the market opens.