acceptodds
Under review as a conference paper at ICLR 2027

MemFlow: Flowing Adaptive Memory for Consistent and Efficient Long Video Narratives

Abstract

The core challenge for streaming video generation is maintaining content consistency over long context, which poses high requirement for the memory design. Most existing solutions maintain the memory by compressing historical frames with predefined strategies. However, different to-generate video chunks should refer to different historical cues, which is hard to satisfy with fixed strategies. In this work, we propose MemFlow to address this problem. Specifically, before generating the coming chunk, we dynamically retrieve the most relevant historical frames as context with the text prompt of this chunk from the global memory bank. This design not only accurately sources the context needed to maintain visual consistency, but also ensures semantic coherence even as new events unfold or scenes transition. In addition, during generation, we only activate the most relevant tokens in memory for each query in the attention layers, which effectively guarantees the generation efficiency. After tuning the model on curated memory-intensive scenarios to unlock its ability to leverage distant historical cues, MemFlow achieves outstanding long-context consistency with minimal computation overhead (7.9% speed reduction compared with the memory-free baseline) and is designed to be compatible with any streaming video generation model with KV cache.

Primary area:
generative models
Keywords:
Long video generation, memory mechanism

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_Nm3iOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3(+1)
Soundness
3(+1)
Read review
Paper summary

This work introduces the MEMFLOW framework, which alleviates the limitations of existing approaches that rely on predefined cache compression strategies. While various retrieval methods, such as those based on 3D spatial indexing, already exist. This specific approach dynamically retrieves the most relevant historical frames from a global memory bank to serve as context for generating the next video segment, guided by the segment's corresponding text prompt. Experiments demonstrate that this method effectively preserves context.

Critical strengths
  1. The two-stage memory selection process in this method is well-founded. NAM first determines the content to be retrieved based on the text, while SMA subsequently filters these memories according to the current generation state—leveraging semantic conditions and the visual generation state, respectively.
  2. The idea behind this paper is easy to understand, and the writing is clear.
Critical weaknesses
  1. There are directly verifiable numerical discrepancies between the main text and the appendices. The results for LongLive differ between Table 2 in the main text and Tables B/C in the appendix, whereas the results for MEMFLOW are consistent. The results for LongLive differ between Table 2 in the main text and Tables B/C in the appendix, whereas the results for MEMFLOW are consistent. In both Table 2 (main text) and Tables B/C (appendix), MEMFLOW achieves a Quality Score of 85.95 and a Semantic Score of 81.90. For LongLive, however, the values ​​in Table 2 are 86.97 (Quality Score) and 76.47 (Semantic Score), while in Tables B/C (appendix), they are 83.96 (Quality Score) and 80.15 (Semantic Score). These discrepancies raise concerns regarding the reliability of the data presented in the paper.
  2. The baselines used for comparison in the experiments are outdated. The authors should include more recent baselines. Furthermore, the effectiveness of the model should be evaluated using architectures with a larger number of parameters. Many of the issues observed with the 1.4B-parameter model might not persist when scaled up to larger models.
  3. The paper introduces two changes. It modifies the model's underlying mechanism and, additionally, incorporates specific training datas, such as the reappearance of objects after they have vanished and the revisiting of previously encountered scenes. However, the paper does not evaluate the specific contribution of this training data. If LongLive were trained using the same MST prompt data, without the inclusion of NAM and SMA, what level of performance improvement would the model achieve?
Questions for authors
  1. Explain the inconsistencies in the numerical values ​​that appear in the paper.
  2. Verify the effectiveness of the method on a larger model.
  3. Conduct additional MST ablation experiments.
4 comments

R3-W1 & Q1: Numerical discrepancies

Thank you for checking the numbers carefully. The discrepancy comes from two different sources for the LongLive numbers and does not affect our conclusion. The LongLive row in Table 2 reports the numbers published in the LongLive paper, whereas the per-dimension LongLive results in Tables B/C come from our own evaluation of the released checkpoint. MemFlow is evaluated by us in both places, which is why its numbers match. The same holds for the 30s setting (Table 4 versus Tables B/C). We could not exactly match the published LongLive numbers with the released checkpoint. The gap may come from evaluation details that are not fully specified, such as the prompt rewriting applied to the prompts before generation and random seeds, and GPU hardware differences. Under either set of LongLive numbers, MemFlow consistently achieves a higher total score at both 5s and 30s, so this discrepancy does not change our conclusion about the advantage of our method. In the revision, we will add a clarification in the appendix stating that the LongLive results in Tables 2 and 4 are taken from the LongLive paper, whereas those in Tables B/C come from our own evaluation of the released checkpoint.

R3-W2 (part 1): More recent baselines

We added recent methods under two protocols.

(1) Our multi-prompt 60s. Four recent methods that support streaming long video generation with prompt switching, evaluated on our 60s multi-prompt test set:

Table a: Comparison of recent baselines for 60s multi-prompt generation

Method#ParamsQuality Score ↑Aesth. Drift ↓Subj. Consist. CLIP ↑Subj. Consist. DINO ↑Scene Consist. CLIP ↑Scene Consist. DINO ↑Prompt Align. ↑Transition Align. ↑
LongLive1.3B1.3\mathrm{B}0.74800.74800.0673‾\underline{0.0673}0.31470.31470.27000.27000.5052‾\underline{0.5052}0.4644‾\underline{0.4644}0.18030.18030.10790.1079
Anchor Forcing1.3B1.3\mathrm{B}0.77870.77870.07430.07430.54100.54100.4676‾\underline{0.4676}0.45440.45440.35860.35860.1814‾\underline{0.1814}0.7826‾\underline{0.7826}
OmniMem1.3B1.3\mathrm{B}0.75110.75110.07720.07720.51680.51680.44830.44830.45450.45450.35990.35990.17450.17450.65760.6576
LongLive 2.05B5\mathrm{B}0.75460.75460.09460.09460.5907\mathbf{0.5907}0.45310.45310.42740.42740.32950.32950.16730.16730.8696\mathbf{0.8696}
Helios (distilled)14B14\mathrm{B}0.7816‾\underline{0.7816}0.08730.08730.23840.23840.18450.18450.40520.40520.32830.32830.15290.15290.59240.5924
MemFlow1.3B1.3\mathrm{B}0.7849\mathbf{0.7849}0.0314\mathbf{0.0314}0.5689‾\underline{0.5689}0.4884\mathbf{0.4884}0.5150\mathbf{0.5150}0.4674\mathbf{0.4674}0.1827\mathbf{0.1827}0.72160.7216

MemFlow achieves the best quality score, the lowest aesthetic drift and the best prompt alignment, and clearly leads the four recent methods in scene consistency under both features as well as in subject consistency under DINO. LongLive 2.0, built on a 5B backbone, obtains higher subject consistency under CLIP and the highest transition alignment. This is expected: before its final Distribution Matching Distillation (DMD), LongLive 2.0 is trained on large-scale long, multi-shot video data, which improves cross-shot consistency, whereas MemFlow is only fine-tuned at the DMD stage on a multi-switch prompt set and builds on a smaller base model. Helios, despite its 14B backbone and competitive quality score, shows the weakest subject consistency, suggesting that model capacity alone does not preserve identities across prompt switches. Thus MemFlow remains competitive against more recent baselines: it is the best or second best on seven of the eight metrics with a 1.3B model.

(2) Public single-prompt protocol. 60s generation on the first 128 MovieGen prompts with VBench-Long metrics, as used by recent long-video works (e.g., Self-Forcing++, Reward Forcing). All methods are evaluated by us under the same protocol:

Table b: Comparison of recent baselines for single-prompt 60s generation on MovieGen prompts

Method#ParamsQuality Score ↑Aesth. Drift ↓Subject Consist. ↑Background Consist. ↑Motion Smooth. ↑Dynamic Degree ↑Aesthetic Quality ↑Imaging Quality ↑Overall Consist. ↑
SkyReels-V21.3B1.3\mathrm{B}0.76730.76730.15280.15280.96190.96190.96410.96410.9877‾\underline{0.9877}0.43750.43750.53610.53610.59230.59230.22050.2205
Self Forcing1.3B1.3\mathrm{B}0.77390.77390.12310.12310.96680.96680.96290.96290.98150.98150.33280.33280.57000.57000.66360.66360.21790.2179
FramePack13B13\mathrm{B}0.80940.80940.02620.02620.96790.96790.96200.96200.9916\mathbf{0.9916}0.62470.62470.58870.58870.65990.65990.24710.2471
LongLive1.3B1.3\mathrm{B}0.80360.80360.0146‾\underline{0.0146}0.97520.97520.9668‾\underline{0.9668}0.9877‾\underline{0.9877}0.40490.40490.6287\mathbf{0.6287}0.6964‾\underline{0.6964}0.2509‾\underline{0.2509}
Anchor Forcing1.3B1.3\mathrm{B}0.8097‾\underline{0.8097}0.02550.02550.96520.96520.96120.96120.97330.97330.65540.65540.61350.61350.68860.68860.24780.2478
OmniMem1.3B1.3\mathrm{B}0.80940.80940.02420.02420.97080.97080.96260.96260.98100.98100.57680.57680.61430.61430.69040.69040.24580.2458
LongLive 2.05B5\mathrm{B}0.80260.80260.06610.06610.9758‾\underline{0.9758}0.95770.95770.98430.98430.7331‾\underline{0.7331}0.52570.52570.65330.65330.22700.2270
Helios (distilled)14B14\mathrm{B}0.79130.79130.10930.10930.96070.96070.96530.96530.98290.98290.7339\mathbf{0.7339}0.56520.56520.56280.56280.22720.2272
MemFlow1.3B1.3\mathrm{B}0.8188\mathbf{0.8188}0.0090\mathbf{0.0090}0.9790\mathbf{0.9790}0.9695\mathbf{0.9695}0.98110.98110.57550.57550.6211‾\underline{0.6211}0.7167\mathbf{0.7167}0.2522\mathbf{0.2522}

On this public single-prompt protocol, MemFlow achieves the best results on six of the nine metrics, including subject consistency, background consistency, and overall consistency. Even the larger LongLive 2.0 and Helios models do not obtain a higher quality score. These results show that MemFlow retains an advantage in general long-video quality over recent state-of-the-art methods, including those built on larger backbones.

In particular, MemFlow produces higher dynamics than LongLive (0.5755 vs. 0.4049), consistent with MST's exposure to multiple prompt switches during training. FramePack obtains higher motion smoothness than MemFlow. Its allocation of more context capacity to recent, more important frames may benefit motion smoothness. However, its generation speed is far below that of MemFlow (0.92 FPS vs. 18.7 FPS, as reported in Table 4 of the main paper).

R3-W2 (part 2) & Q2: Effectiveness on a larger model

We built a 5B version of MemFlow on LongLive 2.0, whose base model is Wan2.2-TI2V-5B. LongLive 2.0 uses a Frame Sink and a Shot Sink: the former is the same as in LongLive, and the latter keeps the KV cache of the first chunk of the clip generated under the current prompt, holds it throughout that clip, and replaces it with the first chunk of the next clip after a prompt switch. We replace the Shot Sink with Narrative Adaptive Memory (NAM) and Sparse Memory Activation (SMA) and fine-tune the LoRA with Memory-Forced Streaming Tuning (MST).

Table c: Cross-backbone validation for 60s multi-prompt generation.

ModelBase model#ParamsMemoryQuality Score ↑Aesth. Drift ↓Subj. Consist. CLIP ↑Subj. Consist. DINO ↑Scene Consist. CLIP ↑Scene Consist. DINO ↑Prompt Align. ↑Transition Align. ↑
LongLiveWan2.1-T2V1.3B1.3\mathrm{B}Frame Sink0.74800.74800.0673‾\underline{0.0673}0.31470.31470.27000.27000.5052‾\underline{0.5052}0.4644‾\underline{0.4644}0.1803‾\underline{0.1803}0.10790.1079
MemFlowWan2.1-T2V1.3B1.3\mathrm{B}Frame Sink + NAM + SMA0.7849‾\underline{0.7849}0.0314\mathbf{0.0314}0.56890.56890.4884‾\underline{0.4884}0.5150\mathbf{0.5150}0.4674\mathbf{0.4674}0.1827\mathbf{0.1827}0.72160.7216
LongLive 2.0Wan2.2-TI2V5B5\mathrm{B}Frame Sink + Shot Sink0.75460.75460.09460.09460.5907‾\underline{0.5907}0.45310.45310.42740.42740.32950.32950.16730.16730.8696‾\underline{0.8696}
LongLive 2.0 + our memoryWan2.2-TI2V5B5\mathrm{B}Frame Sink + NAM + SMA0.7890\mathbf{0.7890}0.07880.07880.6237\mathbf{0.6237}0.5050\mathbf{0.5050}0.40420.40420.33740.33740.16890.16890.8862\mathbf{0.8862}

On this backbone, our design brings the same pattern of gains in subject consistency and transition alignment as on the 1.3B LongLive, which verifies the effectiveness of our method on a larger model. Overall, seven of the eight metrics improve on the 5B backbone. The gains in subject consistency and transition alignment are smaller than at 1.3B, because LongLive 2.0 has been trained on large-scale long, multi-shot data and is a much stronger base model. Our memory design supplies more relevant context needed when earlier content returns, and MST teaches the model to use it and follow more complex narrative changes.

In addition, in the 60s multi-prompt comparison above, the larger baselines (LongLive 2.0 at 5B and Helios at 14B) remain clearly below the 1.3B MemFlow in scene consistency, and Helios also in subject consistency. Scaling up the model alone does not resolve long-range consistency under prompt switches, so the issue is not specific to small models.

R3-W3 & Q3: Ablation on MST

Thank you for the suggestion. To evaluate the specific contribution of the training data, we ran exactly this experiment: LongLive trained with the same MST prompts and schedule, without NAM and SMA.

Table d: MST ablation for 60s multi-prompt generation.

MethodMemoryTraining ParadigmQuality Score ↑Aesth. Drift ↓Subj. Consist. CLIP ↑Subj. Consist. DINO ↑Scene Consist. CLIP ↑Scene Consist. DINO ↑Prompt Align. ↑Transition Align. ↑
LongLiveFrame Sinkw/o MST0.74800.74800.06730.06730.31470.31470.27000.27000.50520.50520.46440.46440.18030.18030.10790.1079
LongLive + MST (new)Frame SinkMST0.75210.75210.06010.06010.32390.32390.30120.30120.5103‾\underline{0.5103}0.4669‾\underline{0.4669}0.1815‾\underline{0.1815}0.50710.5071
MemFlow w/o MSTFrame Sink + NAM + SMAw/o MST0.7700‾\underline{0.7700}0.0485‾\underline{0.0485}0.5684‾\underline{0.5684}0.4185‾\underline{0.4185}0.46910.46910.38040.38040.18100.18100.6818‾\underline{0.6818}
MemFlowFrame Sink + NAM + SMAMST0.7849\mathbf{0.7849}0.0314\mathbf{0.0314}0.5689\mathbf{0.5689}0.4884\mathbf{0.4884}0.5150\mathbf{0.5150}0.4674\mathbf{0.4674}0.1827\mathbf{0.1827}0.7216\mathbf{0.7216}

(1) MST training alone improves LongLive, mainly in transition alignment: the model learns to follow more complex narrative changes, such as subject removal and re-introduction. It also improves scene consistency and subject consistency to some extent. Its scene consistency is close to MemFlow, but its subject consistency remains far below MemFlow, showing that MST training alone cannot make up for the absence of the memory design in preserving subject identity. (2) LongLive + MST still remains well below MemFlow in subject consistency, because a first-chunk sink cannot store subjects that are introduced later; the cue needed at re-appearance is not in the context, and simply training cannot recover it. (3) Our model with the memory design but without MST (third row) shows a deterioration across all visual and semantic consistency metrics compared with full MemFlow. MST further improves fine-grained subject identity (DINO), scene consistency, and transition alignment on top of the gains already provided by the memory design, while subject consistency under CLIP remains nearly unchanged. These results suggest that MST helps the model make fuller use of the retrieved memory and manage complex content transitions. The training paradigm and the memory mechanism are complementary, and the full model performs best among these variants.

Reviewer_aNhmOpenReview ↗
Rating
3
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper addresses identity inconsistency and scene drift caused by prompt switching in streaming long-video generation. Building on LongLive, the authors propose MEMFLOW, which consists of NAM for prompt-guided historical frame retrieval, SMA for efficient memory selection, and MST for improving long-term memory utilization through training scenarios involving character reappearance and scene revisitation. Experiments show improved long-term consistency in 60-second interactive video generation while maintaining an inference speed of 18.7 FPS.

Critical strengths
  1. Different prompts require different historical information, while fixed Frame Sink strategies struggle with character reappearance and scene switching. The motivation for dynamic memory retrieval is clear and practically relevant.
  2. The experiments are fairly comprehensive. Beyond standard video quality metrics, the authors introduce long-term consistency metrics for character reappearance and scene switching. The quantitative results, qualitative comparisons, and user study are generally convincing.
  3. The method improves long-term consistency with only a 7.9% reduction in inference speed, making it practical for streaming video generation.
Critical weaknesses
  1. The semantic retrieval formulation in NAM appears to contain a mathematical error, which may result in identical relevance scores for all historical frames and undermine the effectiveness of the core module.
  2. The designs of NAM, SMA, and MST are relatively straightforward, mainly combining existing retrieval, sparse selection, and training strategies. I find the overall technical novelty somewhat limited.
  3. Several key design choices, including memory updating, retrieval, and positional handling of historical KV caches, lack sufficient experimental validation or implementation details.
  4. The compared baselines are relatively outdated. Comparisons with more recent methods and evaluations on public long-video benchmarks are missing.
Questions for authors

see the weakness above

6 comments

R2-W1: Error in Eq. (1)

Thank you for the careful reading. You are right that Eq. (1) in the original submission is wrong: Am,iA_{m,i} is defined per frame, so the softmax reads as a normalization over the nn tokens of a single frame. Every row then sums to 1, and the averaged score equals 1/n1/n for all frames.

The error is in the equation, which incorrectly described our implementation; the method itself is not affected. In our implementation, no softmax is applied when computing the retrieval score, and all results in the paper were produced with this implementation. Let Qtextl,h∈Rt×d\mathbf{Q}^{l,h}_{\text{text}} \in \mathbb{R}^{t \times d} denote the current prompt's retrieval queries at layer ll and head hh. These queries reuse the normalized text keys from the model's cross-attention module. Here, tt is the number of text tokens, HH is the number of heads, dd is the feature dimension per head, bmb_m is the number of valid prototype frames in the bank at generation iteration mm, and nn is the number of visual tokens per frame. We compute raw scaled dot-product scores over all valid prototype frames and aggregate them:

Aml,h=Qtextl,h(Kml,h)⊤d∈Rt×bmn,Kml,h=[Km,1l,h;… ;Km,bml,h]∈Rbmn×d,\mathbf{A}_m^{l,h} = \frac{\mathbf{Q}_{\text{text}}^{l,h} (\mathbf{K}_m^{l,h})^\top}{\sqrt{d}} \in \mathbb{R}^{t \times b_m n}, \quad \mathbf{K}_m^{l,h} = [\mathbf{K}_{m,1}^{l,h}; \dots; \mathbf{K}_{m,b_m}^{l,h}] \in \mathbb{R}^{b_m n \times d},

Sm,il=1H,t,n∑h=1H∑τ=1t∑j∈ΩiAml,h[τ,j],\mathcal{S}_{m,i}^l = \frac{1}{H\\,t\\,n} \sum_{h=1}^{H} \sum_{\tau=1}^{t} \sum_{j \in \Omega_i} \mathbf{A}_m^{l,h}[\tau, j],

where Ωi\Omega_i is the index set of the nn tokens of frame ii. Thus, Sm,il\mathcal{S}_{m,i}^l averages the raw scaled scores over text tokens, attention heads, and tokens within each frame, independently at each layer ll. No softmax normalization is applied to these scores, so they are not constrained to 1/n1/n and can vary across frames and with the prompt. We have corrected Eq. (1) and its description in the revision.

R2-W2: Technical novelty

We respectfully argue that the contribution is not retrieval or sparse selection in isolation, but making retrieval-based memory work inside a streaming video generator, which requires co-design:

(1) Prompt-conditioned retrieval without auxiliary encoders or additional trainable parameters. We explore retrieval directly within a streaming DiT, using its existing projections and cached visual keys instead of auxiliary encoders or additional trainable retrieval parameters. Specifically, Narrative Adaptive Memory (NAM) projects the current narrative prompt with the DiT's existing cross-attention key projection, scores historical visual keys through text–visual dot products, and retrieves the top-K frames. By comparison, StoryMem uses CLIP-based similarity for memory keyframe selection and updating, whereas OneStory introduces a learned, caption-conditioned Frame Selection module. Related internal KV-cache retrieval has been explored in Video-LLMs and our contribution is to apply such retrieval to recall previously generated content to improve long-range consistency in streaming video generation.

(2) A memory architecture alone is not enough for long-range consistency. As Table 5 ("w/o MST") shows, attaching the memory without Memory-Forced Streaming Tuning (MST) degrades consistency compared with the full MemFlow model. MST introduces memory-intensive training samples and thereby effectively compels the model to utilize the memory for long-range consistency. This shows that only the joint design (memory architecture + memory-forced training) can maximize the improvement in consistency, which is also one of the core contributions of our work.

(3) Efficiency and Practical value. Together with Sparse Memory Activation (SMA), which bounds the number of memory tokens in attention, MemFlow achieves 18.7 FPS compared with 20.3 FPS for LongLive's Frame Sink baseline, a 7.9% reduction in throughput but much better consistency. Our MemFlow framework is compatible with any streaming generator that uses a KV cache. It adds no trainable retrieval module, and SMA keeps the cost of the memory small, so it can be integrated into existing systems without modifying the model architecture and improves the long-range consistency of the base model at a small cost in inference speed.

R2-W3: Ablations and details of key design choices

We have added the missing details to the revision and summarize them here.

Memory updating. The global bank is updated every 18 latent frames (6 chunks) during the rollout: the KV cache of the first latent frame of the newly generated chunk is stored as a prototype. We compare three choices of prototype (first frame, middle frame, and mean-pooled frame): they perform comparably, with no consistent winner, which supports the temporal-redundancy assumption in Sec. 3.2:

Table a: Memory prototype ablation for 60s multi-prompt generation.

Memory prototypeQuality Score ↑Aesth. Drift ↓Subj. Consist. CLIP ↑Subj. Consist. DINO ↑Scene Consist. CLIP ↑Scene Consist. DINO ↑Prompt Align. ↑Transition Align. ↑
First frame (ours)0.7849‾\underline{0.7849}0.0314\mathbf{0.0314}0.5689‾\underline{0.5689}0.4884\mathbf{0.4884}0.5150‾\underline{0.5150}0.4674‾\underline{0.4674}0.1827‾\underline{0.1827}0.7216\mathbf{0.7216}
Middle frame0.7871\mathbf{0.7871}0.0338‾\underline{0.0338}0.5734\mathbf{0.5734}0.47920.47920.50870.50870.4731\mathbf{0.4731}0.18160.18160.70940.7094
Mean-pooled0.78120.78120.03560.03560.56270.56270.4835‾\underline{0.4835}0.5213\mathbf{0.5213}0.46020.46020.1838\mathbf{0.1838}0.7153‾\underline{0.7153}

Retrieval and selection. Retrieval is performed independently at each transformer layer and only when the prompt switches: the K=3K = 3 frames retrieved serve as the memory for all chunks generated under that prompt, until the next switch. They are merged with the Frame Sink (the first-chunk KV cache), and SMA activates k=3k = 3 frames from this combined 6 latent-frame candidate set. Sec. 3.3 has been updated to state more clearly that the sink is part of the SMA candidates. We ablate the sensitivity to KK and kk below.

Table b: Ablation on retrieved-memory size

Retrieved frame number KK (k=3k = 3)Quality Score ↑Aesth. Drift ↓Subj. Consist. CLIP ↑Subj. Consist. DINO ↑Scene Consist. CLIP ↑Scene Consist. DINO ↑Prompt Align. ↑Transition Align. ↑
33 (ours)0.7849‾\underline{0.7849}0.0314\mathbf{0.0314}0.5689‾\underline{0.5689}0.4884\mathbf{0.4884}0.5150‾\underline{0.5150}0.4674\mathbf{0.4674}0.1827‾\underline{0.1827}0.7216\mathbf{0.7216}
660.7896\mathbf{0.7896}0.0352‾\underline{0.0352}0.5781\mathbf{0.5781}0.4698‾\underline{0.4698}0.5213\mathbf{0.5213}0.4641‾\underline{0.4641}0.1838\mathbf{0.1838}0.7064‾\underline{0.7064}
990.77910.77910.04030.04030.55120.55120.44310.44310.50970.50970.45830.45830.18160.18160.69430.6943
All (entire bank)0.77640.77640.04250.04250.54680.54680.43520.43520.50620.50620.45270.45270.18120.18120.68970.6897

Effect of KK. With kk fixed, KK only controls how many text-retrieved candidates are passed to SMA. The results show that K=6K = 6 is comparable to K=3K = 3, with no consistent winner, while K=9K = 9 and the entire bank lower subject consistency and transition alignment.

The drop arises in segments where the prompt brings back a subject that appeared earlier but is absent from the current scene. In this case the two stages behave differently. NAM ranks the bank by the text prompt, so the frames showing the subject are prioritized. SMA selects by the visual query of the chunk being generated; this chunk does not contain the subject yet, so its query resembles the ongoing scene, and SMA tends to favor frames close to the current scene. With a small KK, the candidates are the top-ranked frames and mostly show the subject, so SMA activates, among relevant frames, those closest to the current scene. With a larger KK, the candidates also include lower-ranked frames that do not show the subject, and SMA activates some of them in place of the frames that do. The model then has a weaker reference for the returning subject: its identity drifts, which lowers subject consistency, and it is sometimes not brought back as scripted, which lowers transition alignment. A few extra candidates (K=6K = 6) have a limited effect, and with the entire bank the text-based filtering is removed altogether. The last two rows are close because, on 60s videos, the bank rarely holds more than 9 frames at a prompt switch.

Overall, K=3K = 3 keeps the candidate set relevant to the current prompt with a lower chance of passing irrelevant frames to SMA, which supports our choice of the retrieved-memory size.

Table c: Ablation on activated-memory size

Activated frame number kk (K=3K = 3)Quality Score ↑Aesth. Drift ↓Subj. Consist. CLIP ↑Subj. Consist. DINO ↑Scene Consist. CLIP ↑Scene Consist. DINO ↑Prompt Align. ↑Transition Align. ↑FPS ↑
k=3k = 3 (ours)0.7849‾\underline{0.7849}0.0314\mathbf{0.0314}0.5689\mathbf{0.5689}0.4884\mathbf{0.4884}0.5150‾\underline{0.5150}0.4674‾\underline{0.4674}0.1827‾\underline{0.1827}0.7216\mathbf{0.7216}18.7\mathbf{18.7}
All candidates, k=6k = 6 (Table 5, "w/o SMA")0.7878\mathbf{0.7878}0.0337‾\underline{0.0337}0.5613‾\underline{0.5613}0.4727‾\underline{0.4727}0.5221\mathbf{0.5221}0.4684\mathbf{0.4684}0.1843\mathbf{0.1843}0.7018‾\underline{0.7018}17.6‾\underline{17.6}

Effect of kk. The candidate set passed to SMA has six frames (three retrieved frames and the three-frame Frame Sink). Activating k=3k = 3 of them instead of all six is faster (18.7 vs. 17.6 FPS) and gives a comparable quality score and consistency, offering a better trade-off between generation quality and efficiency.

Positional handling. All frames keep their original temporal positions. The current chunk uses its position in the full video; historical caches keep the positions they had when generated. No re-indexing is applied. This preserves the temporal order between memory and the present, and avoids position collisions with the local window. The same scheme is used in training and inference.

R2-W4: Recent baselines and public long-video benchmarks

Our setting is interactive long video generation with streaming prompts, so we extend the comparison under two protocols.

(1) Multi-prompt 60s. We added four recent state-of-the-art methods that support streaming long video generation with prompt switching, evaluated on our 60s multi-prompt test set:

Table d: Comparison of recent baselines for 60s multi-prompt generation

Method#ParamsQuality Score ↑Aesth. Drift ↓Subj. Consist. CLIP ↑Subj. Consist. DINO ↑Scene Consist. CLIP ↑Scene Consist. DINO ↑Prompt Align. ↑Transition Align. ↑
LongLive1.3B1.3\mathrm{B}0.74800.74800.0673‾\underline{0.0673}0.31470.31470.27000.27000.5052‾\underline{0.5052}0.4644‾\underline{0.4644}0.18030.18030.10790.1079
Anchor Forcing1.3B1.3\mathrm{B}0.77870.77870.07430.07430.54100.54100.4676‾\underline{0.4676}0.45440.45440.35860.35860.1814‾\underline{0.1814}0.7826‾\underline{0.7826}
OmniMem1.3B1.3\mathrm{B}0.75110.75110.07720.07720.51680.51680.44830.44830.45450.45450.35990.35990.17450.17450.65760.6576
LongLive 2.05B5\mathrm{B}0.75460.75460.09460.09460.5907\mathbf{0.5907}0.45310.45310.42740.42740.32950.32950.16730.16730.8696\mathbf{0.8696}
Helios (distilled)14B14\mathrm{B}0.7816‾\underline{0.7816}0.08730.08730.23840.23840.18450.18450.40520.40520.32830.32830.15290.15290.59240.5924
MemFlow1.3B1.3\mathrm{B}0.7849\mathbf{0.7849}0.0314\mathbf{0.0314}0.5689‾\underline{0.5689}0.4884\mathbf{0.4884}0.5150\mathbf{0.5150}0.4674\mathbf{0.4674}0.1827\mathbf{0.1827}0.72160.7216

MemFlow achieves the best quality score, the lowest aesthetic drift and the best prompt alignment, and clearly leads the four recent methods in scene consistency under both features as well as in subject consistency under DINO. LongLive 2.0, built on a 5B backbone, obtains higher subject consistency under CLIP and the highest transition alignment. This is expected: before its final Distribution Matching Distillation (DMD), LongLive 2.0 is trained on large-scale long, multi-shot video data, which improves cross-shot consistency, whereas MemFlow is only fine-tuned at the DMD stage on a multi-switch prompt set and builds on a smaller base model. Helios, despite its 14B backbone and competitive quality score, shows the weakest subject consistency, suggesting that model capacity alone does not preserve identities across prompt switches. Overall, compared with these recent methods, MemFlow provides the most balanced long-range consistency with a 1.3B model.

Reviewer_oyoHOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

MemFlow addresses identity inconsistency when characters disappear and later reappear in long videos through three complementary components: Narrative Adaptive Memory (NAM), which retrieves historical KV memory using the current text prompt; Sparse Memory Activation (SMA), which further selects relevant memory during generation to reduce computation; and Memory-Forced Streaming Tuning (MST), which trains the model to use distant historical context through character reappearance and scene revisitation scenarios. Its core contribution is integrating narrative-conditioned memory retrieval, memory overhead control, and learned memory utilization into a streaming video generation framework.

Critical strengths
  1. Retaining only the first chunk makes it difficult to remember characters introduced later. Retrieving historical context according to the current narrative directly addresses this limitation.
  2. Removing semantic retrieval or memory-forced streaming tuning degrades performance, providing some evidence for the effectiveness of the proposed mechanisms.
Critical weaknesses
  1. The main text describes training with multiple prompt switches and scenarios requiring distant historical context, whereas the appendix describes paired prompts with a single prompt switch.
  2. Evaluation primarily focuses on a single backbone and specially curated 60-second scripts, without longer-video evaluations or cross-backbone validation.
  3. The current results support effectiveness on the specific task, but provide limited evidence for general long-term memory capabilities.
Questions for authors
  1. What training configuration is actually used for MST, and where do the gains come from?
  2. Could the authors compare random sampling, recent-frame sampling, and semantic retrieval under the same memory-frame budget and training configuration? Moreover, report both retrieval hit rate and final consistency.
3 comments

R1-W1: Training configuration

The main text is correct. Sec. 4.1 describes the configuration we actually used: Memory-Forced Streaming Tuning (MST) is run on 60s self-rollouts with five prompt switches per sequence. The "Streaming Tuning" paragraph in Appendix C.2 (paired prompts with a single switch) is a leftover from an early version that followed LongLive's implementation, and it does not match our final training. We have corrected Appendix C.2 in the revision (marked in blue) so that the two descriptions agree.

R1-W2: Longer-video evaluation and cross-backbone validation

Cross-backbone. MemFlow introduces the memory at the Distribution Matching Distillation (DMD) stage and uses MST to teach the model to exploit it for consistent long video generation. A suitable model for cross-backbone validation is therefore LongLive 2.0, which follows a different training path from LongLive: starting from a multi-step bidirectional base model (Wan2.2-TI2V-5B), it trains a multi-step autoregressive (AR) model and a few-step LoRA separately and then combines them into a few-step AR model. LongLive 2.0 keeps a Frame Sink and a Shot Sink (the first chunk of the clip generated under the current prompt). Starting from the released LongLive 2.0 weights, we replace the Shot Sink with Narrative Adaptive Memory (NAM) and Sparse Memory Activation (SMA) and fine-tune the LoRA with MST.

Table a: Cross-backbone validation on 60s multi-prompt generation.

ModelBase model#ParamsMemoryQuality Score ↑Aesth. Drift ↓Subj. Consist. CLIP ↑Subj. Consist. DINO ↑Scene Consist. CLIP ↑Scene Consist. DINO ↑Prompt Align. ↑Transition Align. ↑
LongLiveWan2.1-T2V1.3B1.3\mathrm{B}Frame Sink0.74800.74800.0673‾\underline{0.0673}0.31470.31470.27000.27000.5052‾\underline{0.5052}0.4644‾\underline{0.4644}0.1803‾\underline{0.1803}0.10790.1079
MemFlowWan2.1-T2V1.3B1.3\mathrm{B}Frame Sink + NAM + SMA0.7849‾\underline{0.7849}0.0314\mathbf{0.0314}0.56890.56890.4884‾\underline{0.4884}0.5150\mathbf{0.5150}0.4674\mathbf{0.4674}0.1827\mathbf{0.1827}0.72160.7216
LongLive 2.0Wan2.2-TI2V5B5\mathrm{B}Frame Sink + Shot Sink0.75460.75460.09460.09460.5907‾\underline{0.5907}0.45310.45310.42740.42740.32950.32950.16730.16730.8696‾\underline{0.8696}
LongLive 2.0 + our memoryWan2.2-TI2V5B5\mathrm{B}Frame Sink + NAM + SMA0.7890\mathbf{0.7890}0.07880.07880.6237\mathbf{0.6237}0.5050\mathbf{0.5050}0.40420.40420.33740.33740.16890.16890.8862\mathbf{0.8862}

On this backbone, our design brings the same pattern of gains in subject consistency and transition alignment as on LongLive, showing that the memory supplies the context needed when earlier content returns and that MST teaches the model to use it and follow more complex narrative changes.

Longer videos. We built a 90s test set with the similar structure: six prompts and five switches, with each prompt lasting 15s instead of 10s. It goes beyond the 60s training horizon and lengthens the absence before re-appearance.

Table b: Comparison for longer video generation (90s, 6 prompts × 15s).

MethodQuality Score ↑Aesth. Drift ↓Subj. Consist. CLIP ↑Subj. Consist. DINO ↑Scene Consist. CLIP ↑Scene Consist. DINO ↑Prompt Align. ↑Transition Align. ↑
SkyReels-V20.76160.76160.08600.08600.4752‾\underline{0.4752}0.33020.33020.36910.36910.25120.25120.18130.18130.7409‾\underline{0.7409}
Self Forcing0.75010.75010.14320.14320.44890.44890.3557‾\underline{0.3557}0.37500.37500.31710.31710.14770.14770.57610.5761
FramePack0.7777‾\underline{0.7777}0.0324\mathbf{0.0324}0.42680.42680.34060.34060.40220.40220.28450.28450.1914\mathbf{0.1914}0.55980.5598
LongLive0.74180.74180.07640.07640.32940.32940.28600.28600.5011‾\underline{0.5011}0.4464‾\underline{0.4464}0.17510.17510.11710.1171
MemFlow0.7787\mathbf{0.7787}0.0460‾\underline{0.0460}0.5767\mathbf{0.5767}0.4920\mathbf{0.4920}0.5326\mathbf{0.5326}0.5008\mathbf{0.5008}0.1815‾\underline{0.1815}0.7446\mathbf{0.7446}

As shown in Table b, MemFlow achieves the best quality score and outperforms all other methods in subject and scene consistency under both CLIP and DINO features, although the 90s videos exceed the 60s horizon used in MST. LongLive is the closest in scene consistency, as its Frame Sink anchors the first scene, but falls far behind in subject consistency and transition alignment, in line with the 60s results in Table 1. With a much larger model size of 13B(compared to our 1.3B), FramePack shows lower aesthetic drift than MemFlow, which ranks second, and slightly better prompt alignment, but its subject and scene consistency are far below ours. These results show that MemFlow keeps its consistency advantage on longer videos.

Curated scripts. The curated scripts are used only for the multi-prompt setting, to stress disappearance and re-appearance to evaluate long-term memory capabilities. Single-prompt results are already reported in Tables 2 and 4 on the official VBench prompts. We further follow the evaluation protocol of Self-Forcing++ and Reward Forcing for single-prompt 60s generation; the results are given in our response to W3.

R1-W3: Evaluation for general long-term memory capabilities

To examine the method beyond our curated narrative task, we evaluated single-prompt 60s generation on 128 MovieGen prompts with VBench-Long metrics, a protocol used by recent long-video works (e.g., Self-Forcing++, Reward Forcing):

Table c: Comparison for single-prompt 60s generation on MovieGen prompts

MethodQuality Score ↑Aesth. Drift ↓Subject Consist. ↑Background Consist. ↑Motion Smooth. ↑Dynamic Degree ↑Aesthetic Quality ↑Imaging Quality ↑Overall Consist. ↑
SkyReels-V20.76730.76730.15280.15280.96190.96190.96410.96410.9877‾\underline{0.9877}0.43750.43750.53610.53610.59230.59230.22050.2205
Self Forcing0.77390.77390.12310.12310.96680.96680.96290.96290.98150.98150.33280.33280.57000.57000.66360.66360.21790.2179
FramePack0.8094‾\underline{0.8094}0.02620.02620.96790.96790.96200.96200.9916\mathbf{0.9916}0.6247\mathbf{0.6247}0.58870.58870.65990.65990.24710.2471
LongLive0.80360.80360.0146‾\underline{0.0146}0.9752‾\underline{0.9752}0.9668‾\underline{0.9668}0.9877‾\underline{0.9877}0.40490.40490.6287\mathbf{0.6287}0.6964‾\underline{0.6964}0.2509‾\underline{0.2509}
MemFlow0.8188\mathbf{0.8188}0.0090\mathbf{0.0090}0.9790\mathbf{0.9790}0.9695\mathbf{0.9695}0.98110.98110.5755‾\underline{0.5755}0.6211‾\underline{0.6211}0.7167\mathbf{0.7167}0.2522\mathbf{0.2522}

This protocol contains no prompt switch, so it examines whether our memory design and MST affect general long-video generation. As shown in Table c, MemFlow achieves the best quality score, the lowest aesthetic drift and the best consistency among the compared methods. In particular, MemFlow produces clearly higher dynamics than LongLive, which follows from training with multiple prompt switches in MST. FramePack obtains higher motion smoothness and dynamic degree: it allocates more context capacity to recent, more important frames, which benefits motion smoothness, and it does not go through a DMD distillation stage, which weakens dynamics, so its dynamic degree is higher, while its generation speed (0.92 FPS, Table 4 in the main paper) is far below that of MemFlow (18.7 FPS). Similar conclusions hold for the single-prompt 5s and 30s evaluations on VBench. This experiment shows that general long-video quality is preserved; evidence on long-term memory beyond 60s is given in Table b in our response to W2.

R1-Q1: MST configuration and source of the gains

Configuration. As clarified in our response to W1, MST uses 60s rollouts with five prompt switches and memory-intensive prompts (Sec. 4.1).

Source of the gains. MemFlow differs from LongLive in two respects: the memory design (NAM + SMA) and the training paradigm (MST, i.e., training with multiple prompt switches on memory-intensive prompts). Table 5 in the main paper already ablates MST on top of our memory. To further separate the two factors, we additionally trained LongLive with exactly the same MST prompts and schedule, without NAM and SMA:

Table d: MST ablation on 60s multi-prompt generation.

MethodMemoryTraining ParadigmQuality Score ↑Aesth. Drift ↓Subj. Consist. CLIP ↑Subj. Consist. DINO ↑Scene Consist. CLIP ↑Scene Consist. DINO ↑Prompt Align. ↑Transition Align. ↑
LongLiveFrame Sinkw/o MST0.74800.74800.06730.06730.31470.31470.27000.27000.50520.50520.46440.46440.18030.18030.10790.1079
LongLive + MST (new)Frame SinkMST0.75210.75210.06010.06010.32390.32390.30120.30120.5103‾\underline{0.5103}0.4669‾\underline{0.4669}0.1815‾\underline{0.1815}0.50710.5071
MemFlow w/o MSTFrame Sink + NAM + SMAw/o MST0.7700‾\underline{0.7700}0.0485‾\underline{0.0485}0.5684‾\underline{0.5684}0.4185‾\underline{0.4185}0.46910.46910.38040.38040.18100.18100.6818‾\underline{0.6818}
MemFlowFrame Sink + NAM + SMAMST0.7849\mathbf{0.7849}0.0314\mathbf{0.0314}0.5689\mathbf{0.5689}0.4884\mathbf{0.4884}0.5150\mathbf{0.5150}0.4674\mathbf{0.4674}0.1827\mathbf{0.1827}0.7216\mathbf{0.7216}

(1) MST training alone improves LongLive, mainly in transition alignment: the model learns to follow more complex narrative changes, such as subject removal and re-introduction. It also improves scene consistency and subject consistency to some extent. Its scene consistency is close to MemFlow, but its subject consistency remains far below MemFlow, showing that MST training alone cannot make up for the absence of the memory design in preserving subject identity. (2) LongLive + MST still remains well below MemFlow in subject consistency, because a first-chunk sink cannot store subjects that are introduced later; the cue needed at re-appearance is not in the context, and simply training cannot recover it. (3) Our model with the memory design but without MST (third row) shows a deterioration across all visual and semantic consistency metrics compared with full MemFlow. MST further improves fine-grained subject identity (DINO), scene consistency, and transition alignment on top of the gains already provided by the memory design, while subject consistency under CLIP remains nearly unchanged. These results suggest that MST helps the model make fuller use of the retrieved memory and manage complex content transitions. The training paradigm and the memory mechanism are complementary, and the full model performs best among these variants.

R1-Q2: Memory selection strategies

We trained three models that differ only in how the memory frames are selected, under the same memory-frame budget and training configuration:

  • Random sampling: frames sampled uniformly from the global memory bank.
  • Recent-frame sampling: the most recent frames in the global memory bank that precede the local window.
  • Semantic retrieval (ours): NAM retrieval.

Hit rate is the fraction of the selected frames in which the re-appearing subject is actually visible, measured on the 90s test set when the re-appearance segment is generated. The value is verified manually. The consistency metrics are also measured on the 90s test set.

Table e: Ablation for memory selection mechanism

Memory SelectionHit rate ↑Quality Score ↑Aesth. Drift ↓Subj. Consist. CLIP ↑Subj. Consist. DINO ↑Scene Consist. CLIP ↑Scene Consist. DINO ↑Prompt Align. ↑Transition Align. ↑
Random sampling0.46‾\underline{0.46}0.7503‾\underline{0.7503}0.07670.07670.4460‾\underline{0.4460}0.3467‾\underline{0.3467}0.5030‾\underline{0.5030}0.44400.44400.1801‾\underline{0.1801}0.6181‾\underline{0.6181}
Recent-frame sampling0.000.000.74210.74210.0648‾\underline{0.0648}0.33150.33150.30750.30750.49810.49810.4692‾\underline{0.4692}0.1801‾\underline{0.1801}0.55390.5539
Semantic retrieval (ours)0.73\mathbf{0.73}0.7787\mathbf{0.7787}0.0460\mathbf{0.0460}0.5767\mathbf{0.5767}0.4920\mathbf{0.4920}0.5326\mathbf{0.5326}0.5008\mathbf{0.5008}0.1815\mathbf{0.1815}0.7446\mathbf{0.7446}

With the 15s absence of the subject in the 90s set, the most recent frames no longer contain the subject, so the hit rate of recent-frame sampling drops to 0, while random sampling stays near 0.5. Semantic retrieval achieves the highest hit rate and the best performance, in particular for subject identity. The gain therefore comes from retrieving the right frames rather than from the additional memory budget itself.

Comments on the paper

2 comments

Status after the discussion. Two of the three reviewers (Reviewer oyoH and Reviewer aNhm) gave positive ratings in their original reviews (weak acceptance). Reviewer Nm3i noted in the follow-up that the numerical discrepancies have been addressed to some extent and adjusted the soundness and presentation scores, while keeping the overall rating. The main remaining concern of this reviewer is that the gains of our memory on the 5B backbone are smaller than those on the 1.3B backbone. We addressed it from two sides in our follow-up response:

  • The original benchmark leaves limited room on the affected metrics. The gain in overall quality is of similar size on both backbones. The smaller gains are confined to subject consistency and transition alignment, on which the 5B base model already starts from high values (e.g., 0.87 in transition alignment). To indicate the scale of the subject-consistency metric, we computed it on real videos that follow the same person: they reach only 0.79 with CLIP features and 0.67 with DINO features. A gain of the size reported at 1.3B would require the 5B model to reach or exceed these real-video values, so it is not attainable on this benchmark.

  • On a more demanding benchmark, the same 5B backbone shows much larger gains. We evaluated the 5B model with and without our memory on 100 sequences of 90s with nine successive prompts, which involve repeated subject re-appearance and scene changes. There the base model is no longer close to these limits, and our memory improves subject consistency by 0.12 (CLIP) and 0.11 (DINO) and transition alignment by 0.21, compared with 0.03, 0.05 and 0.02 on the original benchmark. The smaller gains on the original benchmark therefore reflect the room left by that benchmark, not a weaker effect of the memory on a larger model.

We sincerely appreciate the reviewers' constructive feedback, which has helped us strengthen our work.

Best regards,

The Authors

First and foremost, we sincerely thank all Reviewers, Area Chairs, and Program Chairs for their time and thoughtful feedback. We have provided detailed point-by-point responses to all comments from (Reviewer oyoH, Reviewer aNhm, and Reviewer Nm3i).

We are encouraged by the reviewers' positive recognition of our work:

  • Methodology: Retrieving historical context according to the current narrative directly addresses the limitation of keeping only the first chunk, and the motivation is clear and practically relevant. The two-stage memory selection is well-founded: NAM decides what to retrieve from the text, and SMA filters the retrieved memory according to the current generation state.
  • Evaluation: The experiments are fairly comprehensive, with long-term consistency metrics for character re-appearance and scene switching, and the quantitative results, qualitative comparisons and user study are generally convincing. The method improves long-term consistency with only a 7.9% reduction in inference speed, which makes it practical for streaming generation.
  • Presentation: Clear and easy to follow.

In response to the reviewers' valuable suggestions, we have conducted additional experiments and clarified our methodology and contributions. The major questions we addressed are summarized below:

  • Recent baselines and a public long-video benchmark (Reviewer aNhm, Reviewer Nm3i, Reviewer oyoH). The concern was that the compared baselines were outdated and that the evaluation relied on our curated scripts. We added four recent state-of-the-art methods that support prompt switching on our 60s multi-prompt test set. With a 1.3B model, MemFlow is the best on six of the eight metrics. Its scene consistency under DINO features is about 30% above the best of the four recent methods. We also added a public single-prompt 60s protocol used by recent long-video works with nine methods, where MemFlow ranks first on six of the nine metrics. This shows that the memory design does not come at the cost of general long-video quality.

  • Longer videos and a larger backbone (Reviewer oyoH, Reviewer Nm3i). The concern was whether the method still works beyond the 60s horizon and on a model larger than 1.3B. We evaluated 90s videos (six prompts of 15s each), which exceed the 60s training horizon and lengthen the absence before a subject returns. Among the five compared methods, MemFlow is the best on six of the eight metrics. We also transferred the design to the 5B LongLive 2.0 backbone by replacing its Shot Sink with our memory and fine-tuning with MST. Seven of the eight metrics improve over the base model, including the quality score, subject consistency and transition alignment.

  • Correctness of Eq. (1) and consistency of the reported details (Reviewer aNhm, Reviewer oyoH, Reviewer Nm3i). Eq. (1) was written incorrectly and has been corrected. The error was confined to the equation; our implementation was correct.

  • Technical contribution and the source of the gains (Reviewer aNhm, Reviewer oyoH, Reviewer Nm3i). We clarified that the contribution is not retrieval or sparse selection in isolation, but making retrieval-based memory work inside a real-time streaming generator: retrieval reuses the generator's own features, thus needs no auxiliary encoder and no trainable retrieval module, and it is combined with sparse memory activation for efficiency and memory-forced training (MST) for fine-tuning. The ablation in Table 5 of the paper shows that both the memory and the training are needed. Removing MST while keeping the memory lowers scene consistency and fine-grained subject consistency (from 0.49 to 0.42 under DINO features), and replacing semantic retrieval with the entire memory bank lowers subject consistency under DINO features from 0.49 to 0.38.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.