acceptodds
Withdrawn from ICLR 2027

Context Persistence Calibration for Interactive Long Video Generation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Interactive long video generation entails adapting to successive prompts while preserving coherent content over extended horizons. Causal autoregressive models are well suited to this setting, as sequential generation and KV cache reuse naturally support streaming synthesis. Existing methods primarily preserve historical context through visual KV caches, while paying comparatively limited attention to the persistence of textual conditioning. Yet textual conditioning provides direct semantic guidance by specifying the desired visual content. In interactive settings, the initial prompt often establishes global attributes such as subject appearance, scene, and visual style. As generation proceeds under successive prompts, relying primarily on cached visual states to preserve these global attributes becomes increasingly unreliable, degrading long-range visual consistency and semantic coherence. To address this limitation, we propose Context Persistence Calibration (CPC), a training-free framework that maintains global semantics while accommodating evolving instructions. Specifically, we jointly contextualize the initial and current prompts and retain the leading singular components of the contextualized initial-prompt representation. We term this procedure Semantic Anchor, which constructs a rank-constrained textual reference at each subsequent interaction stage. To further mitigate sink-collapse and the resulting abrupt scene resets, we introduce Adaptive Sink Regulation, which adaptively regulates excessive contributions from the frame sink. CPC operates entirely at inference time without additional training, learnable parameters, or cache expansion. Experiments on two interactive long video evaluation sets, InterVidProM and InterStoryBench, demonstrate improved long-range visual consistency and semantic coherence, reduced sink-collapse, and continued responsiveness to evolving prompts.

Primary area:
generative models
Keywords:
Interactive Long Video Generation, Autoregressive Video Diffusion, Text to Video, Generative Model

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_7NJkOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper looks at interactive long video generation with causal AR video models, where the user keeps sending new prompts.

The authors argue that current methods keep history mostly through the visual KV cache, so once the first prompt is replaced, the global attributes it set up (subject look, scene, style) slowly fade.

They propose CPC, a training-free add-on to LongLive with two parts. Semantic Anchor encodes the first prompt together with the current one and keeps a low-rank, down-scaled copy of the first prompt's tokens in the text condition. Adaptive Sink Regulation scales down the frame sink keys per head when the sink gets more attention than the oldest rolling block, to stop the sudden scene resets known as sink collapse.

Validation is on two evaluation sets the authors build with an LLM (InterVidProM and InterStoryBench), using VBench-Long dimensions, CLIP score, two new reset metrics and a 30-person user study against eight baselines.

The clearest result is that LongLive's reset problem mostly goes away, with small gains in subject consistency. Keeping the initial prompt alive is a reasonable idea, but both parts are close to known tricks, and the evidence for the text part is thin.

Critical strengths
  1. A simple, training-free fix for a very visible failure. Adaptive Sink Regulation almost removes LongLive's scene resets (SRR goes from 32.73 to 0.31 at 60s) with no new parameters and no cache growth, and the user study agrees. This matters because resets are the most obvious failure in long interactive rollouts, and the fix is easy to drop into LongLive style systems.

  2. The paper treats persistence of the text condition as its own axis, separate from visual memory, and compares with a broad and recent set of AR baselines over two horizons. This is useful because it gives a cheap, clear starting point for multi-prompt generation that others can build on.

Critical weaknesses
  1. The main claim, that the initial text context has to persist, is not really backed by the numbers. In Table 3, Semantic Anchor alone barely changes VBench, makes resets slightly worse and lowers CLIP, and full CPC has lower CLIP than LongLive in all three settings. Persistence itself is only shown on one video (Fig. 2) and a 5-point t-SNE. There is no dataset-level persistence metric, direct concatenation is only shown qualitatively in the appendix, and the SVD ratio and scaling factor are never ablated. This matters because most of the measured gain comes from fixing the backbone's own sink collapse, which Rolling Forcing already avoids, and we can't tell what the SVD step adds beyond simply re-adding P1.

  2. Novelty is limited, and the closest prior work isn't cited. Semantic Anchor is very close to One-Prompt-One-Story (ICLR 2025), which also concatenates an identity prompt with frame prompts, edits the token embeddings with SVD, and even reports the same kind of embedding compactness evidence as Fig. 3. SuppressEOT (ICLR 2024) also applies SVD to text embeddings. Adaptive Sink Regulation is a heuristic sink damping in the line of Deep Forcing, LoL and attention sink calibration in LLMs. Everything is tested on one backbone with single runs. Positioned against this work, the contribution looks like two known tricks combined on top of LongLive, which is the main reason for my score.

Questions for authors
  1. Can you report quantitative results for direct P1 + Pi concatenation, for concatenation with the 0.8 scaling but no SVD, and for sweeps over the truncation ratio and lambda? How were 60% and 0.8 chosen, and on which data? This decides whether Semantic Anchor adds anything beyond re-adding P1.
Reviewer_ZiJnOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies the loss of persistent context during interactive long-video generation. It proposes CPC, an inference-time modification of LongLive with two components. Semantic Anchor jointly encodes the initial and current prompts and applies truncated SVD to the initial-prompt representations. Adaptive Sink Regulation scales frame-sink keys according to their attention mass relative to the oldest rolling-cache block.Experiments on InterVidProM and InterStoryBench report improved visual consistency and fewer detected resets, with additional support from a human preference study.

Critical strengths

CPC addresses a concrete failure mode without retraining the backbone or enlarging the visual KV cache. The reported improvement over LongLive is substantial for reset behavior: on 60-second InterVidProM, SRR decreases from 32.73 to 0.31 in the table’s units, while PSNR-RFS increases from 48.70 to 99.57. This makes the method a potentially useful addition to an existing generation pipeline.

The evaluation includes two prompt collections, a longer 120-second setting, component ablations, and a human preference study. CPC is preferred to LongLive in 80.7% of subject-consistency decisions and 70.2% of instruction-following decisions.

Critical weaknesses

With Semantic Anchor alone, VBench-Long Avg. changes only from 84.83 to 84.97, while the reset metrics become slightly worse. Figures 9–10 show failures of direct concatenation, but do not provide a quantitative comparison with CPC under otherwise matched settings. A full-rank control with the same weights, a separately encoded initial-prompt control, and a small retention-ratio sweep would help establish the contribution.

Equation (9) multiplies sink keys by a coefficient below one, but this does not always reduce their softmax attention: a negative query-key logit moves toward zero and can receive more weight.This does not invalidate the reported empirical gains, but the intended suppression should be verified by measuring sink attention before and after regulation across sequences and heads.

Questions for authors

1.Have SRR and PSNR-RFS been checked against human annotations of unintended resets? Both use the first frame as the reference. A deliberate return to an earlier view could be penalized, while a reset to a later state could be missed. Appendix C.6 establishes threshold stability, but not agreement with perceived resets.

2.What are the generation speed, peak memory, and prompt-switch latency relative to LongLive on the same hardware? An unchanged visual cache size does not establish unchanged inference cost. Please explain how attention-mass estimation and key regulation are implemented with efficient attention kernels.

3.How does CPC behave when a later prompt deliberately changes clothing, visual style, or another attribute established in the first prompt?

Reviewer_jMnPOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper tries to keep characters and scenes consistent when users change prompts during long-video generation. It keeps a compressed version of the initial prompt using SVD and combines it with the current prompt. It also adjusts attention to the first-frame cache to reduce sudden returns to the starting scene. The method is applied to LongLive without extra training.

Critical strengths
  • The method does not need extra training or a larger visual KV cache. This is useful in practice.
  • The paper is easy to follow, and the examples explain the problem clearly.
  • The attention adjustment helps reduce sudden scene resets, and the ablation results support this.
Critical weaknesses
  • The method mainly remembers information from the first prompt. What happens if a new character appears later and needs to return much later? What if the user changes an attribute from the initial prompt, such as a character’s clothes? These seem like important cases for interactive generation, but the current tests do not clearly show how well CPC handles them.
  • Most of the reduction in scene resets seems to come from sink regulation. The gain from the Semantic Anchor alone is small, and the full method does not improve the CLIP score over LongLive. I would like stronger evidence that the text anchor helps preserve identity while still following the new prompt.
  • Both reset metrics compare generated frames with the first frame. An intended return to the starting scene could therefore be counted as a failure, while a reset to another scene could be missed. A manual check of some detected resets would help show whether these metrics measure the intended problem.
Questions for authors

I would like to understand why the authors chose this way of keeping memory. Another option is to store characters, objects, and scenes as reusable assets, then retrieve them when needed, possibly with an agent. What do we gain by keeping the initial prompt instead? Is it faster, cheaper, or easier to control? The paper includes LongLive-RAG, but I would still like a clearer comparison of other choices.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.