DeltaVideo: Learning Residual Updates for Long-Term Video Memory

World modeling requires maintaining persistent information about scenes and objects as the environment evolves. Causal streaming video generation offers a promising route toward this goal, yet limited attention windows restrict access to distant history, making long-term consistency difficult to maintain. Existing context-based memory methods extend this accessible history, but often retain redundant visual information across neighboring frames, leading to inefficient use of memory capacity. Efficiently retaining this history therefore requires exploiting the substantial visual redundancy across frames. We introduce Delta Video, a learnable online memory mechanism that learns residual memory updates, suppressing information already captured by the current state while preserving newly observed content. For memory updating, its frame-wise Delta rule predicts incoming visual representations from the current memory state and writes only the prediction residual, suppressing redundant updates for already well-represented information. Retrieved historical context is injected into attention layers to guide ongoing generation. To further teach the memory to handle redundant observations, we introduce redundancy-aware post-training that enforces consistency in both the recurrent memory state and its readout under constructed redundant history, encouraging stable and redundancy-aware memory preservation. On 30-second autoregressive generation, Delta Video improves Continuity of Memory by 14.1% over the strongest baseline. Controlled analyses further show that redundant observations induce 53.0% smaller changes in subsequent memory reads, providing direct evidence that Delta Video selectively suppresses already-represented information. Project page: https://delta-video.github.io/.