Entropy Forcing: Entropy-Routed Subset Selection for KV Cache in Streaming Video Generation
Abstract
Autoregressive (AR) video diffusion has emerged as a promising paradigm for streaming and open-ended long-video synthesis, with generation quality depending heavily on effective KV cache management under a fixed budget. Existing methods typically rely on attention-based importance, selecting KV entries with the highest attention scores for subsequent generation. However, the reliability of this importance signal depends on how distinctly the attention scores separate different KV entries. Specifically, well-separated scores yield clear rankings, whereas clustered scores provide limited discrimination and make selection less reliable. In such cases, existing methods lack an appropriate criterion for selecting a suitable subset of KV entries. To address these limitations, we propose **Entropy Forcing**, a training-free KV cache selection method for AR video diffusion. First, instead of scoring KV entries independently, we introduce a coverage-aware objective that explicitly accounts for redundancy among retained historical representations. Second, we use normalized Rényi-2 entropy as a global routing signal to adaptively determine how much the selection should rely on attention-based importance versus representation coverage. Finally, we exhaustively score and rank candidate KV subsets under the given budget, with parallel evaluation enabling efficient inference. Experiments on long-video generation show that Entropy Forcing improves motion dynamics while maintaining competitive visual quality and temporal consistency compared with recent state-of-the-art methods. Notably, it consistently achieves the best overall average performance across the evaluated metrics.
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.