acceptodds
Under review as a conference paper at ICLR 2027

MemID: Memorizing Identities in Streaming Narrative Video Generation

15 views

Abstract

Memory is central to streaming autoregressive (AR) video generation, where later events of a multi-prompt narrative must stay consistent with content that has fallen outside the bounded context window. Existing memory methods mainly focus on what to remember, yet they either overwrite early identity anchors through continuous memory updates or rely on pretrained LLMs/VLMs as external retrievers in inference. More importantly, they overlook where to render each identity: even with correct identity references, overlapping attention between different identities causes ID fusion, while fragmented attention within the same identity leads to ID duplication. We propose MemID, an identity-aware streaming AR video generation framework that addresses both aspects. For what to remember, MemID introduces an ID-preserving Memory that compresses each prompt-conditioned event into representative frames, freezes it after the event ends, and retrieves only prompt-relevant entries for current generation. For where to render, MemID is trained with ID-aware Attention Losses that separate different identities while encouraging compact and temporally coherent attention for each identity. Extensive experiments demonstrate that MemID substantially improves long-range identity consistency and maintains accurate instance counts in multi-character narratives, while preserving competitive generation quality. Code and models will be publicly released.

Primary area:
generative models
Keywords:
Video Generation, Identity Consistency

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_qygFOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies identity consistency in streaming video generation with changing prompts. MemID combines an event-based memory bank with two attention losses. The memory bank selects representative frames through clustering, freezes the memory of completed events, and retrieves historical KV features for subsequent generation. The two losses discourage attention overlap between different subjects and suppress competing spatial regions for the same subject. The model is initialized from a LongLive Stage 1 checkpoint and trained with DMD and the proposed losses. Experiments on 60-second multi-prompt videos report improvements in identity consistency and subject counting.

Critical strengths
  1. The paper addresses a relevant problem: retrieving the correct historical reference does not necessarily prevent the generator from mixing identities or duplicating subjects. The attention losses directly target these failures without requiring spatial annotations or additional optimization at inference time.
  2. The ablations show that both attention losses contribute to the final performance, and using them together gives the best identity scores. The method also avoids external LLM/VLM calls at inference, which is a practical advantage over IAMFlow.
Critical weaknesses
  1. My main concern is the limited novelty of the memory design. The memory component reorganizes existing retrieval-based memory by grouping frame-level KV features into prompt-defined events, selecting representative frames with K-Means, freezing completed events, and retrieving entries using text-video similarity. MemFlow already performs prompt-conditioned retrieval of historical KV features, while IAMFlow already uses identity-aware retrieval. The main differences therefore appear to be event-wise grouping, clustering, and freezing, which I view as incremental changes rather than a substantial new memory mechanism. The Exclusive Loss also shares its main idea with prior work. Separate-and-Enhance: Compositional Finetuning for Text-to-Image Diffusion Models (Bao et al., SIGGRAPH 2024) already uses a training-time separation loss to reduce overlap between object attention maps. MemID applies this idea to identity attention in streaming video generation, but it is unclear what the methodological advance is beyond this adaptation. The Tracking Loss is more distinctive. Table 3 shows that it is useful as a complete objective, but does not separate the effect of temporal association from that of suppressing competing spatial regions.
  2. The memory ablations do not identify which part of the design drives the reported gains. Table 2 replaces the entire proposed memory with a MemFlow-style memory, so the difference between 65.34 and 55.38 combines several changes, including event-wise freezing, frame selection, and the amount of history retained. Figure 8 does not resolve this ambiguity. Its fixed-capacity baselines are continuously overwritten, whereas the full method retains compressed memories from every completed event. The comparison therefore changes both the update rule and the size of the candidate history. The w/o K-Means variant partly isolates frame selection, but the contribution of event-wise freezing remains unclear. Table 2 also lacks the fourth matched condition: MemFlow-style memory with DMD-only training. It therefore shows the benefit of the proposed memory when the attention losses are present, and the benefit of the losses when the proposed memory is present, but does not separate their individual contributions from their interaction. The DMD-only variant scores below LongLive (53.02 vs. 62.45), but LongLive is not a matched baseline for this ablation. The results therefore support the full system, while the attribution of its gains to the two components remains incomplete.
Questions for authors
  1. In Figure 8, the full method retains compressed memories from every completed event, whereas the fixed-capacity banks are continuously overwritten. How should the observed gain be attributed between event-wise organization and the larger amount of history retained in the candidate pool?
  2. Table 2 reports the full model and two single-component ablations, but no matched baseline with neither proposed component. What can be concluded about the individual contributions of the memory and attention losses, rather than their interaction?
  3. How does the Exclusive Loss differ methodologically from the training-time separation loss in Separate-and-Enhance, aside from its application to streaming video? For the Tracking Loss, what evidence separates the effect of temporal association from that of suppressing competing spatial regions?
Reviewer_1E7ROpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes MemID, an identity-aware streaming autoregressive video generation framework for maintaining character consistency across long narratives. The method introduces ID-preserving Memory to preserve historical identity information and ID-aware Attention Losses to reduce identity fusion and duplication. Experiments demonstrate improvements in identity consistency and instance-count accuracy over existing streaming video generation methods.

Critical strengths
  • The paper writing is clear, and the motivation of addressing both temporal identity drift and spatial identity confusion in long-video generation is reasonable.
  • The proposed ID-preserving memory mechanism is clean, retaining historical identity information through compression and retrieval without requiring external LLMs/VLMs at inference.
  • The experimental results are promising, showing improvements over existing baselines in identity consistency and instance counting.
Critical weaknesses
  • The experimental evaluation is limited to a single narrative multi-prompt benchmark. It would be useful to evaluate more complex narratives such as character reappearances and scene transitions to better demonstrate the effectiveness of the proposed method.
  • The efficiency analysis is insufficient. Although the paper claims real-time inference, the paper is missing detailed comparisons of inference latency, memory consumption, and scalability with video length.
  • The related work section could be refined to further position the paper with respect to recent video generation methods on selective memory and consistency designs, such as
    • OneStory: Coherent multi-shot video generation with adaptive memory (CVPR 2026)
    • VGGRPO: Towards World-Consistent Video Generation with 4D Latent Reward (ECCV 2026)
Questions for authors

Please see the weakness above.

Reviewer_wHFzOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper presents MemID, an identity-aware streaming autoregressive video generation framework for long-form multi-prompt narratives. The method combines an ID-preserving memory, which stores and freezes representative frames from completed events, with ID-aware attention losses that reduce inter-ID attention overlap and intra-ID attention fragmentation. Experiments show improved long-range identity consistency and instance counting while maintaining competitive overall video quality.

Critical strengths
  1. The paper identifies two clear sources of identity inconsistency: temporal memory drift and spatial attention interference, and designs corresponding solutions for each.
  2. The proposed event-wise memory is simple and effective, preserving early identity anchors without continuously overwriting them with potentially drifted generations.
  3. The experimental evaluation includes strong recent baselines, time-wise identity consistency analysis, and qualitative results that clearly illustrate the main failure modes.
Critical weaknesses
  1. The gains from the memory module may partly come from simply preserving earlier, cleaner reference frames. The current ablations do not include a strong baseline that explicitly keeps early anchors under the same memory budget.
  2. The evaluation is limited to 60-second videos. Since completed-event memories are permanently retained, it is unclear how memory size, retrieval cost, and generation speed scale to substantially longer streams.
  3. Entity supervision relies on LLM-extracted foreground entities, but the reliability of this extraction is not evaluated. Errors in entity identification could directly affect the proposed attention losses.
  4. The proposed identity metrics rely on a multi-stage automatic pipeline involving LLM entity parsing, GroundingDINO, SAM, and DINO features. The accuracy and robustness of this evaluation pipeline are not independently validated.
  5. Some implementation details remain unclear, particularly how retrieved K/V memories are injected into the DiT and which representations are used for text-video retrieval.
Questions for authors
  1. Can the authors compare against a baseline that explicitly preserves early clean reference frames under the same memory budget?
  2. How do memory size, retrieval latency, and generation speed scale with substantially longer videos and more events?
  3. How accurate is the LLM-based entity extraction used for training, and was it manually or automatically verified?
  4. Has the proposed ID Consistency / ID Counting evaluation pipeline been validated against human annotations or another independent evaluator?
  5. Could the authors clarify how retrieved K/V memories are injected into the DiT and which layer representations are used for retrieval?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.