acceptodds
Withdrawn from ICLR 2027

Memory Forcing: Attendable Mid-Horizon History for Few-Step Streaming Video

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Autoregressive video diffusion realizes causal streaming generation without a bidirectional pass over the full clip. Those histories are stored as keys and values in a KV cache, which later denoising steps attend without re-encoding the past. To keep attention cost from growing with clip length, most current systems keep only a recent FIFO of fixed length and drop the oldest keys when the window is full, and we call this mid-horizon forgetting, where lost events sit between the start of the clip and the recent representations that the FIFO still holds. We present Memory Forcing, a few-step streaming method that focuses on those lost events rather than only the recent window. To this end, we introduce Archive & Working Banks so those events remain attendable, local motion stays in the recent window, and later queries can still retrieve a subject or scene after it leaves and returns. Further, to give those banks distinct places on a time axis whose indices otherwise grow outside the range seen in training, we introduce Bank-aware RoPE so later queries read the start, the recent representations, and mid-horizon events in the right place. Extensive experiments show that Memory Forcing outperforms existing methods on long-horizon clips and stays more stable as generation extends from 5s to 60s. To move past the public 1.3B ceiling on this few-step forcing line, we also introduce, to our knowledge, the first public 5B streaming model.

Primary area:
generative models
Keywords:
long video generation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_RHBNOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Memory Forcing, a few-step streaming video generation framework designed to address mid-horizon forgetting in autoregressive diffusion models. Built upon Rolling Forcing, the method introduces Archive & Working Banks to preserve intermediate historical information within a bounded KV cache, and Bank-aware RoPE to dynamically assign temporal positions to different memory banks. Experiments on Wan2.1-1.3B and Wan2.2-5B demonstrate improved long-horizon generation quality and stability across multiple benchmarks and video durations.

Critical strengths
  1. Practical and efficient memory design. The proposed memory mechanism preserves intermediate historical information within a fixed KV-cache budget, providing a practical approach to improving long-range consistency without unbounded memory growth.
Critical weaknesses
  1. Limited methodological novelty. The proposed three-bank memory organization and Bank-aware RoPE share similar ideas with recent or concurrent works such as MemRoPE[1], Anchor Forcing[2], and Rolling Sink[3]. Although the specific implementation differs, the main contributions appear to be incremental extensions of existing memory management and positional encoding strategies. The paper could better clarify its distinctive technical contributions relative to these closely related approaches.
  2. Limitations of the memory selection strategy. The proposed similarity-based eviction relies on a local comparison between working-memory chunks and the incoming chunk, rather than their relevance to future queries. Consequently, historically important information may be permanently discarded, while less useful content is retained. Moreover, the archive uses FIFO eviction once full, which may further undermine long-term memory retention. This raises concerns about whether the proposed strategy can reliably preserve important events over extended video generation.
  3. The paper's title appears incomplete, as "Few-Step Streaming Video Generation" would more accurately describe the task. Moreover, the qualitative visualizations are limited and lack sufficiently detailed comparisons to convincingly demonstrate long-term consistency.

[1] Kim, Youngrae, et al. "MemRoPE: Training-Free Infinite Video Generation via Evolving Memory Tokens." In ECCV, 2026.

[2] Yang, Yang, et al. "Anchor Forcing: Anchor Memory and Tri-Region RoPE for Interactive Streaming Video Diffusion." In ECCV, 2026.

[3] Li, Haodong, et al. "Rolling Sink: Bridging Limited-Horizon Training and Open-Ended Testing in Autoregressive Video Diffusion." arXiv preprint arXiv:2602.07775, 2026.

Questions for authors
  1. Can the authors further clarify the novelty and distinctive contributions of Memory Forcing compared with prior works?
  2. How does the similarity-based eviction strategy ensure that important historical information is preserved, given that current similarity does not necessarily reflect its relevance to future queries?
Reviewer_wq9zOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes Memory Forcing, a cache management approach for streaming video generation that aims to preserve useful mid-horizon history under a fixed memory budget. Building on Rolling Forcing, it partitions the historical KV cache into three banks: a sink bank for initial context, an archive bank for selected historical chunks, and a working bank for actively updated context. Chunks that are least similar to newly generated content are promoted from the working bank to the archive, with FIFO eviction when the archive reaches capacity. The method also introduces a bank-aware RoPE assignment that dynamically rotates cached, unrotated keys using compact temporal coordinates, together with context-aligned cache insertion. Experiments on 1.3B and 5B models report improved long-video quality, with the clearest gains on the 1.3B model.

Critical strengths
  1. Reasonable motivation. Preserving mid-horizon context under a fixed memory budget addresses a meaningful challenge in streaming video generation. And the three-bank cache and bank-aware RoPE provide a clear approach to combining retained history with recent context.

  2. Good empirical results. The 1.3B model shows improved long-video quality and reduced degradation as video duration increases.

Critical weaknesses
  1. The paper explicitly claims to support subject and scene reappearance after leaving the context window. However, the evaluation primarily measures general video quality and consistency, without a targeted leave-and-return assessment to substantiate this claim.

  2. The ablations do not clearly isolate the benefits of the method. It is unclear whether Working-only and Archive + Working use the same total cache budget. Moreover, FIFO is tested only at inference after training with similarity-based selection, potentially confounding its comparison with train-inference consistency.

Questions for authors
  1. Which examples or results explicitly demonstrate recovery of a subject or scene after it leaves the recent context window?

  2. Beyond the aggregate scores in Table 2(b), what evidence explains why Interpolated RoPE makes better use of mid-horizon history than the other three mappings? Could the authors provide a controlled leave-and-return comparison or an analysis of archived-history usage to substantiate this interpretation? Please also clarify the rationale for compressing archive positions into [0,1], including the effect of sharing boundary positions with the sink and working bank.

Reviewer_r6kZOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper addresses "mid-horizon forgetting" in few-step streaming video diffusion: because the KV cache keeps only a sink plus a recent FIFO window, events that leave the window can no longer be retrieved. Based on Rolling Forcing, the proposed Memory Forcing splits the cache into sink, archive and working banks, and moves the least-similar working chunk into the archive. It also introduces Bank-aware RoPE, which reassigns temporal indices at every attention so they stay within the trained range and the three banks stay distinguishable. On VBench-based evaluations from 5s to 60s, the method is at or near the top on longer clips with among the smallest degradation.

Critical strengths
  1. The paper names a real limitation of sink-plus-FIFO caches, “mid-horizon forgetting,” and addresses it with a three-bank cache and similarity-based archiving that needs no extra memory module and keeps the cache size fixed. This matters because the fix is a drop-in change to the existing Rolling Forcing pipeline, so it is practical and easy to adopt.
  2. Memory Forcing is at or near the top at 15s–60s and has among the smallest 5s→60s drops across four prompt suites. Ablations on the banks, the four RoPE index choices and the similarity-based write each show a consistent gain. This supports the paper’s central claim that long-video quality improves, not just short-clip quality.
Critical weaknesses
  1. The paper is about retrieving mid-horizon events, such as a subject that leaves and returns, yet all numbers are aggregate VBench Quality/Total scores, which mostly reflect per-frame quality. Leave-and-return is shown only in a few qualitative frames, with no identity-consistency metric and no targeted benchmark. Given that, and gains that are often under 1 point with no variance reported, the evidence does not establish that the archive actually does what the paper claims.
  2. The paper itself says relative re-indexing of RoPE is prior work (Infinity-RoPE), and AnchorForcing already uses an anchor memory with tri-region RoPE, which leaves the bank partition and similarity-based write as fairly incremental.
Questions for authors
  1. Can you give a quantitative leave-and-return evaluation, such as identity/scene similarity between pre-exit and post-return frames (DINO/CLIP), against Rolling Forcing and AnchorForcing?
  2. How many seeds or samples were used per prompt, and what is the variance of the scores, given that most long-horizon gains are under 1 point?
  3. How does Memory Forcing differ from AnchorForcing’s anchor memory and tri-region RoPE in mechanism, and how do they compare head-to-head at the same cache budget?
  4. What exactly does the similarity function sim compute (which features, how pooled over tokens)?
  5. Does the FIFO-only inference ablation also require a model retrained with FIFO writes, or is the train/test mismatch confounding Table 2c?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.