Memory Forcing: Attendable Mid-Horizon History for Few-Step Streaming Video

Autoregressive video diffusion realizes causal streaming generation without a bidirectional pass over the full clip. Those histories are stored as keys and values in a KV cache, which later denoising steps attend without re-encoding the past. To keep attention cost from growing with clip length, most current systems keep only a recent FIFO of fixed length and drop the oldest keys when the window is full, and we call this mid-horizon forgetting, where lost events sit between the start of the clip and the recent representations that the FIFO still holds. We present Memory Forcing, a few-step streaming method that focuses on those lost events rather than only the recent window. To this end, we introduce Archive & Working Banks so those events remain attendable, local motion stays in the recent window, and later queries can still retrieve a subject or scene after it leaves and returns. Further, to give those banks distinct places on a time axis whose indices otherwise grow outside the range seen in training, we introduce Bank-aware RoPE so later queries read the start, the recent representations, and mid-horizon events in the right place. Extensive experiments show that Memory Forcing outperforms existing methods on long-horizon clips and stays more stable as generation extends from 5s to 60s. To move past the public 1.3B ceiling on this few-step forcing line, we also introduce, to our knowledge, the first public 5B streaming model.