acceptodds
Under review as a conference paper at ICLR 2027

StreamingR2V: Towards Long and Interactive Reference-to-Video Generation

Abstract

Reference-to-video (R2V) generation renders user-specified subjects faithfully, but only within a short clip whose reference set is fixed before generation begins. In practice, however, subjects often enter or leave partway through a shot, and users need interactive control over the cast as the video unfolds over a long horizon. We therefore extend R2V to (SR2V), which keeps the horizon open and lets references be added or removed as generation proceeds, and identify two challenges that both peak at the moment of interaction. First, history frames carry the identities already rendered, which override the expressiveness of newly inserted references. Second, the chunk where the reference set changes must render a subject unseen in history, which can raise its error, and may accumulate more acutely during generation. We accordingly propose with two complementary designs. The reconstructs each history token from the references and subtracts the reference-explainable identity from its value, leaving the attention map untouched. The adds history-error banks indexed by rollout depth, approximating error accumulation at one forward pass per iteration. On an SR2V benchmark with diverse events of subject entry and exit, StreamingR2V matches or exceeds leading commercial models and several baselines across a wide range of metrics, especially on long-term consistency and interaction compliance. The anonymous demo is available at: https://sr2v-demo.github.io/.

open until 14 Dec 2026

est. 32% chance this paper gets accepted at ICLR 2027.

Reject 68%Accept 32%

What do you think this paper will get?

All positions stay anonymous.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.