acceptodds
Under review as a conference paper at ICLR 2027

MemActive: Training-Free Memory Activation for Consistent Interactive Streaming Video Generation

Abstract

Interactive streaming video generation requires following evolving prompts while recalling entities such as characters or scenes beyond the recent context. However, frozen generators can fail to recover visual content even when relevant historical memory is supplied. We term this failure the memory-utilization gap. In a controlled experiment with fixed memory, temporal RoPE compression improves consistency with the supplied history, accompanied by an increase in memory attention mass. Motivated by this finding, we introduce MemActive, a training-free framework combining Route-Specific Query RoPE (RSQ-RoPE) with Semantic Structured Memory (SSM). RSQ-RoPE adjusts memory-route positions according to entity-description compatibility while retaining native local-route positions. SSM supplies entity-specific memory for retrieval and recombination across prompts. Quantitative and qualitative experiments under hard cuts and smooth transitions show that MemActive preserves recurring content over long temporal gaps. These consistency gains further extend to four additional training and training-free streaming backbones.

Primary area:
generative models
Keywords:
streaming video generation, memory utilization

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_zRHVOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

MemActive investigates why a frozen streaming video generator may fail to use relevant historical memory even when it is available. Its Semantic Structured Memory stores character- and scene-specific KV states from their first occurrence. Route-Specific Query RoPE then compresses query–memory temporal offsets according to text-description compatibility, while retaining native positional relations for local attention. Both routes share a softmax normalization. The method requires no additional generator training. Evaluation covers hard-cut and smooth-transition prompt sequences, controlled positional-remapping experiments, activation ablations, comparisons with additive attention bias, a blinded human study, and transfer to four additional backbones.

Critical strengths
  1. The paper distinguishes memory availability from memory utilization and tests this distinction directly. The fixed-memory experiment across 100 cases changes the query–memory positional mapping while holding the supplied memory and random seed fixed. The SSM-only versus full-method comparison further supports the importance of activation: hard-cut inter-clip consistency increases from 79.24 to 85.99. The paper also correctly acknowledges that shared normalization can change local attention mass despite leaving local logits unchanged.

  2. The empirical validation is comparatively broad for a training-free intervention. The evaluation includes two 100-sequence suites, each with twelve five-second clips, multiple inference-time baselines, a blinded study with 30 participants, and additional backbone tests. The comparison against additive attention bias is a useful control. Together, these results support practical usefulness beyond a single qualitative example or checkpoint.

Critical weaknesses
  1. The principal entity-recall metric has important validity gaps for the claimed task. Appendix E.2 groups clips using character labels from the prompts, associates detected person crops using DINO similarity, and evaluates consistency using DINO features. This procedure does not independently verify that a detected person corresponds to the requested named entity, so visually consistent but incorrectly bound characters may escape detection. More seriously, when no person is detected, the metric falls back to the full frame rather than explicitly penalizing missing character evidence. A recurring background can therefore produce high similarity even when the requested person is absent. Only the middle frame of each clip is sampled, which also leaves within-clip disappearance or identity changes unmeasured. These issues do not prove that the reported improvements are artifacts, but they mean the headline inter-clip score is insufficient to establish reliable entity recall. The blinded human study is valuable supporting evidence, yet it does not separately quantify missing entities, identity swaps, or prompt-to-entity binding errors.

  2. The evidence does not sufficiently distinguish useful interactive memory from stronger preservation of a fixed appearance anchor. SSM stores the first occurrence of each entity and never updates it; CLIP description similarity controls activation of this fixed state. The evaluation does not explicitly separate stable identity from attributes that the current prompt requires to change. Moreover, the paper’s own ablations show consequential trade-offs: in the hard-cut setting, adding the full method to the reset-controlled baseline increases inter-clip consistency from 78.67 to 85.99, but Dynamic Degree decreases from 43.92 to 30.75 and text alignment from 9.14 to 8.84 (Table 3). These changes do not establish that reduced dynamics are necessarily undesirable, but require task-specific evidence that requested actions and state changes are preserved. Cut accuracy also decreases from 100.00 for the reset-only baseline to 98.13 for the full method, demonstrating that improved cut accuracy over native baselines cannot be assigned to memory activation alone. The missing discussion of EM-Vid [4], which already combines training-free entity-indexed memory, structured scripts, and selective appearance control, further leaves the novelty and practical benefit of the proposed combination incompletely established.

  3. Cite and discuss [1–3], covering complementary approaches to local/global memory, adaptive multi-shot conditioning, and long-horizon consistency. More directly, EM-Vid [4], publicly available since May 2026, should be included because it also uses a training-free entity-indexed memory and structured multi-shot scripts, while explicitly addressing selective appearance preservation. Its latent-patch representation differs from the proposed KV-state memory, and it does not establish that RSQ-RoPE has already been proposed. Nevertheless, this overlap makes a precise component-level comparison necessary: what is new in entity organization, what is new in activation, and what evidence demonstrates improved controllability?

The fixed-memory intervention and training-free implementation are interesting. However, the central recall metric does not adequately establish entity presence and correct identity binding, and the evaluation does not resolve preservation versus intended change. Together with incomplete positioning against closely related entity-memory work, these concerns prevent me from concluding that the reported consistency gains establish the claimed interactive-memory advantage.

References

[1] Zhaochong An, Menglin Jia, Haonan Qiu, et al. OneStory: Coherent Multi-Shot Video Generation with Adaptive Memory. CVPR, 2026.

[2] Hongjun Wang, Lin Liu, Jianguo Li, and Tao Lin. Dual-Granularity Memory for Efficient Video Generation. CVPR, 2026.

[3] Jingyu Lin, Xinyi Shang, Peng Sun, Cunjian Chen, and Zhiqiang Shen. DynaMem: Consistent Long Video Generation via Hierarchical Memory and Motion Priors. ICML, 2026.

[4] Jente Vandersanden, Matheus Gadelha, Chun-Hao P. Huang, Hyeonho Jeong, and Yulia Gryaditskaya. EM-Vid: Training-Free Entity-Centric Memory for Efficient and Consistent Multi-Shot Video Generation. arXiv:2605.23610, 2026.

Questions for authors
  1. How often does the inter-character metric use the full-frame fallback, separately for each method? Can you report entity-presence accuracy, identity-swap errors, and prompt-to-character binding accuracy using independently verified correspondences? Do the rankings persist with several sampled frames per clip and explicit treatment of missing entities?

  2. Can you evaluate prompts that require an entity to retain identity while changing clothing, appearance, action, or scene state, reporting preservation and requested-change success separately? How does the CLIP-based compatibility rule perform on small textual changes with large visual consequences? Please distinguish this evaluation from generic text similarity or motion magnitude.

  3. Please separate the contributions of boundary reset, SSM, and RSQ-RoPE under matched conditions, and provide paired uncertainty estimates. If the contribution is claimed to arise specifically from improved within-memory token matching, can a control matched for total memory attention mass support that interpretation? Otherwise, the narrower conclusion that the positional intervention improves selected consistency measures is more appropriate.

These questions address the validity and interpretation of the current results, rather than optional future capabilities.

9 comments
AuthorsOpenReview ↗

Point-by-point response to zRHV

Dear Reviewer zRHV,

Thank you for your constructive suggestions and for highlighting the distinction between memory availability and utilization, the controlled fixed-memory experiment, and the breadth of the evaluation supporting the method's practical usefulness. Below, we respond to your comments point by point. A typeset version of this response appears in Appendix L.4 of the revised paper, with tables and supporting figures.

AuthorsOpenReview ↗

zRHV-W1 & Q1: Fallback, Five-Frame Scoring, and VLM Character Checks

MemActive retains the highest Inter-Character and Inter-Clip Consistency in both transition modes under five-frame scoring with missing detections penalized. Table 5 reports the original midpoint fallback counts for each method. MemActive used full-frame fallback for 4 of 1,200 hard-cut frames and 1 of 1,200 smooth-transition frames. In the stricter evaluation, we retain all pairs and assign zero character similarity at sampled positions where either clip lacks a valid person crop.

A separate VLM audit matches generated characters to named entities independently of DINO-based crop matching. MemActive ranks first or second in character presence, achieves binding rates of 90% under hard cuts and 93% under smooth transitions, and matches the lowest reported identity-swap score in both modes (Table 6). In the VLM identity-preservation evaluation, it exceeds Echo-Forcing by 5 percentage points under hard cuts and 7 under smooth transitions (Table 7). See Common Question 3 for the full results.

AuthorsOpenReview ↗

zRHV-W2(a): Fixed anchor and memory updating

Static first-occurrence memory achieves higher Inter-Clip Consistency and Identity Preservation than latest-occurrence replacement in both transition modes.

We compare static memory with a latest-occurrence updating policy (Table 10). Under the update policy, the KV from the middle latent frame of each new character appearance replaces the character's previously stored memory. Compared with this policy, static memory improves Inter-Clip Consistency from 82.69 to 85.99 for hard cuts and 91.22 to 91.98 for smooth transitions, while increasing Identity Preservation from 65% to 72% and 76% to 79%, respectively, with comparable Instruction Following rates in both transition modes. The fixed first-occurrence memory serves as a stable identity reference, while its influence is adaptively modulated according to the compatibility between the current and stored character descriptions.

Table 10. Static versus updated memory.

SettingMemory policyText Alignment↑Dynamic Degree↑Motion Smoothness↑Intra-Clip Consistency↑Inter-Clip Consistency↑Shot Cut Accuracy↑Identity Preservation (%)↑Instruction Following (%)↑
HardUpdate8.8025.7399.4094.4782.6995.22659
HardStatic (Ours)8.8430.7599.4194.4085.9998.13729
SmoothUpdate8.8016.0099.3495.9691.22N/A768
SmoothStatic (Ours)9.5319.3399.3496.1891.98N/A798
Reviewer_wmePOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper has two components: SSM stores separate historical KV memories for characters and scenes, while RSQ-RoPE compresses the temporal positional distance between queries and long-term memory, preserving the native positional relations for recent frames. This method is training-free. The evaluation showing improved cross-clip consistency.

Critical strengths

This work identifies ineffective use of historical memory as a source of inconsistency and develops a method based on this finding. The connection between the analysis and the method offers a useful direction for long video generation. This method improves long video consistency without additional training and shows gains across generation settings and models, supporting its transferability.

Critical weaknesses

The paper includes action changes, camera-related scenes, and evaluations of prompt following, but two points remain unclear. First, if the SSM reference shows a person standing and a later prompt asks them to crouch or run, the compression factor remains unchanged when the character description stays the same. The reported averages do not show how well identity is preserved while the new action is followed. Second, there are no separate results for long videos with large camera movements and substantial changes in viewing angle. It remains unclear how well the method recalls characters and scenes when they reappear from viewpoints that differ substantially from the memory frame.

Questions for authors

Regarding the two concerns in Weaknesses: 1.When the requested action differs substantially from the stored pose, how does MemActive compare with the baselines on identity consistency and action following? 2.In long videos with large camera movements and substantial viewpoint changes, how does MemActive compare with the baselines on consistency when characters or scenes reappear?

5 comments
AuthorsOpenReview ↗

Point-by-point response to wmeP

Dear Reviewer wmeP,

Thank you for your constructive suggestions and for highlighting the connection between our memory-utilization analysis and the training-free method, and for noting that the gains across settings and backbones support its transferability. Below, we respond to your comments point by point. A typeset version of this response appears in Appendix L.3 of the revised paper, with references to the supporting tables and figures.

AuthorsOpenReview ↗

wmeP-W1(a) & Q1: Identity consistency and action following

MemActive achieves the highest Identity Preservation Rate, Average Score, and Joint Success on action-change prompts in both transition modes. Under hard cuts, it achieves 41% Joint Success versus 36% for Infinity-RoPE, with higher Identity Preservation (95% vs. 69%) and lower Action Following (42% vs. 58%).

When only the action instruction changes, the entity description and thus α remain unchanged. The current prompt still specifies the new action, while α controls the strength of memory activation.

Figure 3(a) in Appendix L shows a standing-to-seated transition under hard cuts and a standing-to-crouched transition under smooth transitions. Running prompts can produce walking-like motion. See Common Question 2 for the complete comparison (Table 4).

AuthorsOpenReview ↗

wmeP-W1(b) & Q2: Identity consistency and viewpoint following

MemActive achieves the highest Identity Preservation Rate on viewpoint-change prompts in both transition modes. It reaches 88% under hard cuts and 86% under smooth transitions, compared with 69% and 80% for the strongest baseline in each mode (Table 4). In the hard-cut example in Figure 3(b) in Appendix L, the recurring characters remain recognizable in front and rear views, and the park remains recognizable across clips through visual cues such as the path, lawn, and trees.

Across methods using the shared Self-Forcing backbone, following the requested viewpoint remains challenging. MemActive follows the requested viewpoint in 6% of hard-cut recalls and 0% of smooth-transition recalls, with joint success rates of 4% and 0%, respectively. The vanilla Self-Forcing backbone also achieves 0% viewpoint following on the same prompts without memory conditioning. See Common Question 2 for the full comparison.

Reviewer_TP8KOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper aims to tackle the interactive streaming video generation, which requires following prompts while recalling entities beyond the recent context. In particular, the authors identified the memory-utilization gap and proposed MemActive framework, which is a training-free framework combining RSQ-RoPE and SSM. In a sense, the motivation is clear and performance is good.

Critical strengths
  1. The motivation is reasonable and the authors analyzed the motivation very clear in Section 3. The authors identified the memory-utilization gap in frozen streaming generation and show RoPE remapping improves recall in a controlled experiment.
  2. The RSQ-RoPE is novel which adjusts query-memory positional relations without altering the native local-route mapping.
  3. Extensive experiments demonstrate the effectiveness of the proposed method.
Critical weaknesses
  1. Table 4 compares RSQ-RoPE with Attention bias across different values of \gamma. Attention Bias can obtain better performance when \gamma is set as 4 for smooth setting. Hence, it remains unclear whether Attention Bias can achieve comparable performance with a more carefully tuned \gamma. Moreover, I am also concerned about whether the improvement of RSQ-RoPE comes from increased memory attention mass or from more effective utilization of memory tokens.
  2. MemActive tackled the confliction between the stored appearance and new prompts with the CLIP-based text similarity. However, I am concerned about this manner may fail to distinguish identity-related features from appearance attributes which should be modified.
  3. The paper focus on interactive streaming video generation, where the inference efficiency can be important. However, Table 2 shows that MemActive introduces more computational cost and GPU memory overhead, which may limit its practicality. Moreover, what is the efficiency with an increasing number of characters.
  4. SSM construct entity-specific memory with the central latent frame of its first occurrence. I am wondering this first occurrence may not provide a good appearance, such as under bad viewpoint or occlusion. Could the authors provide how such frame selection affects the consistency. Meanwhile, is it possible to investigate multi-frame memory would provide additional merits?
Questions for authors

See Weakness above

7 comments
AuthorsOpenReview ↗

Point-by-point response to TP8K

Dear Reviewer TP8K,

Thank you for your constructive suggestions and for highlighting the clear motivation, the controlled experiment showing how positional remapping improves recall, and the novelty of RSQ-RoPE. Below, we respond to your comments point by point. A typeset version of this response appears in Appendix L.2 of the revised paper, with tables and supporting figures.

AuthorsOpenReview ↗

TP8K-W1: Attention-bias tuning and memory-mass control

RSQ-RoPE achieves higher Inter-Clip Consistency at every tested bias strength in both transition modes. Under smooth transitions, bias at γ = 4 achieves higher Text Alignment and Dynamic Degree. Extending the sweep to γ = 6, 8, and 10 leaves RSQ-RoPE ahead on Inter-Clip Consistency. Its advantage also persists under matched memory attention mass, with gains of 2.89 points under hard cuts and 0.80 points under smooth transitions. See Common Question 1 for the full results.

AuthorsOpenReview ↗

TP8K-W2: Change following and identity consistency

Adaptive activation better balances recurring identity preservation and prompt-specified appearance changes: it improves identity preservation over the backbone and instruction following over fixed-α activation.

We evaluate appearance-change prompts using a VLM to measure identity preservation and instruction following separately. Figure 2(a) in Appendix L shows clothing changes in which MemActive preserves the recurring character's appearance cues while applying the requested edit. Table 3 shows that adaptive activation achieves the highest Joint Success in both transition modes. Under hard cuts, Joint Success reaches 90%, compared with 86% for the backbone and 87% for fixed-α activation. Please see Common Question 2 for the complete evaluation and results.

Comments on the paper

4 comments
AuthorsOpenReview ↗

General Response to Reviewers and Area Chair

Dear Reviewers and Area Chair,

Thank you for the constructive reviews and valuable suggestions. The three main concerns raised across the reviews are summarized below, together with our responses:

1. Can tuning attention bias or matching total memory attention reproduce the consistency gains of RSQ-RoPE?

RSQ-RoPE achieves higher Inter-Clip Consistency than additive attention bias at every tested bias strength in both transition modes. With attention mass matched for each entity memory group, RSQ-RoPE improves mean Inter-Clip Consistency by 2.89 points under hard cuts and 0.80 points under smooth transitions over the bias control. These gains support the role of within-group attention redistribution in improving consistency.

2. Can MemActive preserve identity while following requested changes?

Adaptive activation improves Joint Success over both the backbone and fixed-α activation in all four appearance and scene-state settings. MemActive achieves the highest Joint Success on action-change prompts in both transition modes. On viewpoint-change prompts, it achieves the highest Identity Preservation Rate in both modes.

3. Do the consistency results hold under stricter evaluation and independent character matching?

MemActive retains the highest Inter-Character and Inter-Clip Consistency in both transition modes under five-frame scoring with missing detections penalized. A separate VLM assessment also ranks MemActive first in identity preservation, at 72% under hard cuts and 79% under smooth transitions.

Detailed analyses of these three common questions are provided below, and the complete common-question and reviewer-specific point-by-point responses, including supporting figures, are provided in Appendix L (“Response to Reviewers”) of the revised manuscript.

AuthorsOpenReview ↗

Common Question 1 — Can tuning attention bias or matching total memory attention reproduce the consistency gains of RSQ-RoPE? (Related to R1-TP8K-W1 and R3-zRHV-Q3(b))

RSQ-RoPE achieves higher Inter-Clip Consistency than additive attention bias across all tested strengths in both transition modes. This advantage persists at matched total memory attention mass, showing that memory attention mass alone does not account for the observed consistency gains. RSQ-RoPE can also adjust the relative attention weights of individual tokens within each entity memory group, enabling more selective use of retrieved memory.

Attention-bias tuning. At γ = 4 under smooth transitions, Attention Bias achieves higher Text Alignment and Dynamic Degree but lower Motion Smoothness, Intra-Clip Consistency, and Inter-Clip Consistency than RSQ-RoPE. We further extend the sweep to γ = 6, 8, 10. All three settings score below RSQ-RoPE on Inter-Clip Consistency in both transition modes (Table 1 and Figure 1 in Appendix L).

Matched memory attention mass. For each query and attention head at each step, we calibrate a uniform logit bias for each entity-memory group to match the total attention mass assigned by RSQ-RoPE. Table 2 shows that RSQ-RoPE achieves higher mean Inter-Clip Consistency by +2.89 points under hard cuts and +0.80 points under smooth transitions. These results suggest that the consistency gains cannot be explained solely by increased total memory attention mass and are consistent with a contribution from within-group token-level attention allocation.

We have added both the extended bias sweep and the matched-mass comparison to the main paper.

Table 1. Comparison of RSQ-RoPE with Attention Bias at Different Strengths.

SettingMemory ActivationγText Alignment↑Dynamic Degree↑Motion Smoothness↑Intra-Clip Consistency↑Inter-Clip Consistency↑SCA↑
Hard cutAttention Bias67.6524.7598.9489.0783.1294.19
Hard cutAttention Bias88.1328.3398.6087.4880.0285.05
Hard cutAttention Bias108.4921.8398.4988.2179.4977.65
Hard cutOurs--8.8430.7599.4194.4085.9998.13
SmoothAttention Bias69.2827.7598.3492.5586.03N/A
SmoothAttention Bias89.4916.0097.9389.2580.97N/A
SmoothAttention Bias109.2517.7597.9789.5381.78N/A
SmoothOurs--9.5319.3399.3496.1891.98N/A

Table 2. Attention bias matched to RSQ-RoPE memory attention mass.

SettingMethodText Alignment↑Dynamic Degree↑Motion Smoothness↑Intra-Clip Consistency↑Inter-Clip Consistency↑Shot Cut Accuracy↑
HardMatched bias8.8126.4099.4094.4483.1096.33
HardOurs8.8430.7599.4194.4085.9998.13
SmoothMatched bias8.8017.4499.3395.8791.18N/A
SmoothOurs9.5319.3399.3496.1891.98N/A
AuthorsOpenReview ↗

Common Question 2 — Can MemActive preserve identity while following requested changes? (Related to R1-TP8K-W2; R2-wmeP-W1(a,b), Q1, Q2; R3-zRHV-W2(b), Q2(a,b))

Adaptive activation achieves higher Joint Success than both the backbone and fixed-α activation in all four appearance and scene-state settings. For action changes, MemActive achieves the highest Joint Success in both transition modes. On viewpoint-change prompts, it achieves the highest Identity Preservation Rate, at 88% under hard cuts and 86% under smooth transitions.

Evaluation. We construct 25 additional cases (12 clips each) per transition mode for each of four change types: appearance, scene state, action, and viewpoint. We use Gemini 3.5 Flash to evaluate Identity Preservation Rate and Instruction Following Rate, and report their arithmetic mean as the Average Score. Joint Success requires both criteria to be satisfied in the same recall. For appearance and scene-state changes, we compare the backbone, fixed-α activation, and adaptive activation to isolate the effect of the adaptive strategy. For action and viewpoint changes, we compare MemActive with the baselines.

Appearance and scene-state changes. Adaptive activation improves identity preservation over the backbone and instruction following over fixed-α activation in all four settings. Under hard-cut scene-state changes, it improves Instruction Following from 42% to 74% and Joint Success from 40% to 49% compared with fixed-α activation. The improvements also hold under smooth transitions, with Instruction Following increasing from 8% to 15% and Joint Success from 3% to 9% (Table 3). Figure 2(a,b) in Appendix L shows side-by-side examples of clothing and scene-state edits for the backbone, fixed-α activation, and adaptive activation.

Action changes. MemActive achieves the highest Identity Preservation Rate, Average Score, and Joint Success in both transition modes. Under hard cuts, it achieves 41% Joint Success versus 36% for Infinity-RoPE, with higher identity preservation (95% vs. 69%) despite lower action following (42% vs. 58%). Under smooth transitions, its Joint Success reaches 30%, compared with 23% for the best baseline. Figure 3(a) in Appendix L shows examples of the requested seated and crouched poses. In the running examples, outputs from MemActive and the baselines resemble walking (Table 4).

Viewpoint changes. MemActive achieves the highest identity preservation on viewpoint-change prompts in both transition modes. Figure 3(b) in Appendix L shows front-to-back and left-to-right viewpoint prompts. The outputs illustrate how challenging it is for methods using the shared Self-Forcing backbone to reach the requested viewpoint (Table 4).

To test viewpoint control without memory conditioning, we use prompts that specify the same viewpoint changes to generate five-second videos with the vanilla Self-Forcing backbone. Its Instruction Following Rate is 0% on these prompts. This result shows that the base generator struggles to follow the requested viewpoint changes even without memory conditioning. A backbone with stronger camera control could build on MemActive's identity-preservation gains by following viewpoint instructions more reliably.

Table 3. VLM evaluation of appearance and scene-state changes.

SettingMethodIdentity Preservation (%)↑Instruction Following (%)↑Average Score (%)↑Joint Success (%)↑
Appearance change (Hard cut)Backbone88989386
Fixed-α919693.587
Ours919894.590
Appearance change (Smooth)Backbone9014528
Fixed-α931051.56
Ours9115539
Scene-state change (Hard cut)Backbone56766645
Fixed-α70425640
Ours617467.549
Scene-state change (Smooth)Backbone3816277
Fixed-α448263
Ours401527.59

Table 4. VLM evaluation of action and viewpoint changes.

SettingMethodIdentity Preservation (%)↑Instruction Following (%)↑Average Score (%)↑Joint Success (%)↑
Action change (Hard cut)Infinity-RoPE695863.536
Echo-Forcing704758.532
Ours954268.541
Action change (Smooth)Infinity-RoPE63234315
Echo-Forcing801547.513
SWIFT822754.516
MemRoPE76325423
Ours94326330
Viewpoint change (Hard cut)Infinity-RoPE691441.54
Echo-Forcing571435.51
Ours886474
Viewpoint change (Smooth)Infinity-RoPE800400
Echo-Forcing72136.50
SWIFT69034.50
MemRoPE72136.50
Ours860430
AuthorsOpenReview ↗

Common Question 3 — Do the consistency results hold under stricter evaluation and independent character matching? (Related to R3-zRHV-W1 and R3-zRHV-Q1)

MemActive retains the highest Inter-Character and Inter-Clip Consistency in both transition modes under five-frame scoring with missing detections penalized. It also achieves the highest VLM-assessed Identity Preservation Rate, at 72% under hard cuts and 79% under smooth transitions.

Robustness to missing crops and temporal sampling. In the original midpoint evaluation, MemActive used full-frame fallback for 4 of 1,200 hard-cut frames and 1 of 1,200 smooth-transition frames (Table 5). We evaluate both midpoint and five-frame protocols with missing detections explicitly penalized: at each sampled position, we retain all clip pairs and assign zero character similarity when either clip lacks a valid person crop. We evaluate five temporal locations (10%, 30%, 50%, 70%, and 90% of each clip) and average the scores across these positions. Under this stricter protocol, MemActive achieves five-frame Inter-Clip Consistency scores of 83.14 and 90.28 under hard cuts and smooth transitions, respectively (Table 5).

Independent character audit. The VLM matches generated characters to named entities based on the prompts and visual content, independently of DINO-based crop matching. MemActive ranks first or second in character presence. It ranks second in prompt-to-character binding in both modes, with rates of 90% under hard cuts and 93% under smooth transitions. It matches the lowest reported identity-swap score in both modes (Table 6).

Identity preservation and instruction following. We further use a VLM to compare recurring characters with earlier generated references and assess whether the current clips follow their corresponding instructions. Under a criterion that requires all applicable action, scene, and camera instructions to be satisfied, MemActive achieves instruction-following rates of 9% under hard cuts and 8% under smooth transitions. As discussed in Common Question 2, complex actions and viewpoint changes remain challenging across methods using the shared Self-Forcing backbone. MemActive achieves the highest VLM-assessed Identity Preservation Rate in both modes, exceeding Echo-Forcing by 5 and 7 percentage points, respectively (Table 7).

Table 5. Full-frame fallback and robustness to five-frame evaluation.

SettingMethodFallback frames (original midpoint evaluation)Midpoint Inter-Character ↑Midpoint Inter-Clip ↑Five-frame Inter-Character ↑Five-frame Inter-Clip ↑
Hard cutInfinity-RoPE10/120067.3979.1264.4376.60
Hard cutEcho-Forcing5/120068.7579.8467.7477.32
Hard cutOurs4/120075.4284.4275.0383.14
SmoothInfinity-RoPE37/120079.0985.5677.8583.45
SmoothEcho-Forcing2/120084.9688.9385.1287.95
SmoothSWIFT3/120086.6889.9686.5389.94
SmoothMemRoPE8/120077.1282.3676.1381.72
SmoothOurs1/120088.5591.5788.2890.28

Table 6. VLM judgments of character presence, binding, and swaps.

SettingMethodPerson Presence Rate (%) ↑Prompt-to-Character Binding Accuracy (%) ↑Identity-Swap Rate (%) ↓
Hard cutInfinity-RoPE98%82%1%
Hard cutEcho-Forcing99%92%0%
Hard cutOurs99%90%0%
SmoothInfinity-RoPE93%87%0%
SmoothEcho-Forcing98%94%0%
SmoothSWIFT87%80%0%
SmoothMemRoPE86%77%1%
SmoothOurs96%93%0%

Table 7. VLM evaluation of identity preservation and instruction following.

Both rates use the same recurring-character events. Average Score is the mean of the two rates.

SettingMethodIdentity Preservation Rate (%)↑Instruction Following Rate (%)↑Average Score (%)↑
Hard cutInfinity-RoPE541333.5
Hard cutEcho-Forcing671038.5
Hard cutOurs72940.5
SmoothInfinity-RoPE46927.5
SmoothEcho-Forcing72739.5
SmoothSWIFT52327.5
SmoothMemRoPE59331
SmoothOurs79843.5

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.