acceptodds
Under review as a conference paper at ICLR 2027

Memento: Reconstruct to Remember for Consistent Long Video Generation

Abstract

Long-form video generation requires recurring subjects to remain consistent across shots, viewpoints, and scene transitions. Memory-conditioned autoregressive methods improve scalability through shot-by-shot generation. However, they primarily optimize memory conditioning or selection mechanisms, without explicitly supervising the utilization of identity-critical evidence in historical memory, allowing subject appearance to drift over time. We propose Memento, a subject-reconstruction-guided framework that treats subject preservation as an explicit identity grounding problem, introducing memory-based subject reconstruction to encourage the model to preserve and retrieve identity-critical appearance cues. Memento jointly trains next-shot generation and subject reconstruction, recovering target appearances from historical memory and subject descriptions. A dual-query memory mechanism separates these historical cues: story-conditioned queries retrieve long-range subject evidence, while shot-conditioned queries select short-range contextual references for coherent continuation. A subject-aware cinematic data pipeline aligns story, shot, and reconstruction captions through consistent subject descriptions, providing unambiguous supervision for identity grounding. Experiments demonstrate state-of-the-art subject consistency, particularly across scene transitions, together with strong cross-shot coherence and competitive visual quality.

Primary area:
generative models
Keywords:
long video generation, subject consistency, memory-conditioned generation

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_kVswOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses the challenge of subject appearance drift across shots in long-form video generation. The proposed method is built upon a memory-conditioned autoregressive generation framework and consists of three components: First, a subject-aware data pipeline utilizes Qwen3-VL for three-stage annotation to establish a consistent list of characters. Additionally, ByteTrack is employed to select the frame where a character occupies the largest portion of the screen, serving as the target for reconstruction. Second, a dual-path decoupled memory bank uses story/character descriptions and current shot descriptions to independently condition two sets of learnable queries. These queries retrieve long-range identity evidence and short-range contextual references from a shared candidate pool, forming a memory bank of fixed capacity. Third, a subject-anchored multi-task training strategy is implemented: in addition to standard next-shot denoising, the shared DiT is tasked with reconstructing target frames using only the memory bank and character text descriptions. This forces the model to retain and retrieve critical identity cues from memory. The model is based on Wan2.2 14B and was trained on approximately 20,000 film clips. On the STbench benchmark, Memento outperforms StoryDiffusion, Mask²DiT, StoryMem, and HoloCine across three subject-consistency metrics, as well as in semantic and background consistency, while ranking second in aesthetic quality. It also achieves the best subject consistency in out-of-distribution testing on VBench.

Critical strengths
  1. The research problem is precisely defined, and the motivation is clear. The authors’ argument that "consistency should be explicitly learned" is compelling.

  2. The experiments cover representative methods across three paradigms and include quantitative analysis, qualitative analysis, user studies, and ablation studies.

Critical weaknesses
  1. One of my concerns is that while the paper’s main argument revolves around "identity preservation," the three subject consistency metrics that are based on the cosine similarity of full-frame DINOv2 features and full-sequence ViCLIP features. Intuitively, these metrics evaluate whether shots featuring the same character "look similar" overall, rather than whether the individual is, in fact, the same person. There are no metrics specifically targeting facial or bodily identity.

  2. The ablation results for the three proposed modules appear somewhat inconsistent. In Table 3, the full method that incorporates all three modules ranks only second across all three subject consistency metrics, with the "story query + recon" configuration performing best. The paper does not provide a detailed explanation for this phenomenon.

  3. Regarding the comparative baselines, the paper’s baselines and literature review are mainly limited to 2025; they neither discuss nor compare against a significant number of memory-driven long-video methods that have emerged since 2026 (such as SlotMem[1], VideoMemory[2], and Context Forcing[3]). Furthermore, while the authors mention OneStory in their related work, they do not include it in their empirical comparisons.

  4. The experimental section lacks essential training details, such as the number of training steps, batch size, and total training duration. Furthermore, regarding the writing style, the manuscript consistently uses the \citet format (e.g., "text alignment Ho et al. (2020); Peebles & Xie (2023); …") throughout; this should be changed to \citep. Additionally, I suggest adjusting the layout on page 8.

[1] SlotMem: Character-Addressable Internal Memory for Narrative Long Video Generation

[2] VideoMemory: Toward Consistent Video Generation via Memory Integration

[3] Context Forcing: Consistent Autoregressive Video Generation with Long Context

Questions for authors

Please refer to weaknesses.

5 comments
AuthorsOpenReview ↗

Response to Reviewer kVsw (Part 1)

We thank the reviewer for recognizing our problem formulation and experimental coverage, and for the constructive suggestions on identity-specific evaluation, ablation analysis, recent baselines, and reproducibility.


Weakness 1 & Weakness 3.

We have expanded our quantitative evaluation by adding ArcFace-based facial identity similarity and including three additional baselines: SlotMem, VideoMemory, and Context Forcing. For each video, we select one main character and identify all shots in which this character appears alone. We crop the character's face from all valid frames in these shots and uniformly sample 20 frames from the resulting temporally ordered collection. We extract ArcFace embeddings from the sampled face crops and compute cosine similarity for every pair of distinct frames. The video's facial identity consistency score is the mean over these 190 pairs.

MethodInter-shot ↑Intra-shot ↑Inter-scene ↑ArcFace ↑Background ↑Semantic (Story / Shot) ↑Aesthetic ↑
StoryMem0.66060.8146‾\underline{0.8146}0.66920.25150.97320.2793‾\underline{0.2793} / 0.26810.4937
HoloCine0.57910.81280.65940.25920.97700.2720 / 0.2854‾\underline{0.2854}0.4568
SlotMem0.6906‾\underline{0.6906}0.76210.66900.25360.94820.2196 / 0.18660.4948
VideoMemory0.57150.77360.61750.24780.94700.2055 / 0.14760.5859
Context Forcing0.65610.78570.7174‾\underline{0.7174}0.2618‾\underline{0.2618}0.9803‾\underline{0.9803}0.2718 / 0.23490.4509
Memento (Ours)0.73380.85780.72680.28070.98050.3063 / 0.28930.4977‾\underline{0.4977}

Bold, underline‾\underline{\text{underline}}, and italics indicate the best, second-best, and third-best results, respectively. Story-level and shot-level semantic consistency are ranked separately.

ArcFace provides a face-specific identity measure that complements the original DINOv2 and ViCLIP based consistency metrics. In the updated evaluation, Memento achieves higher ArcFace similarity than StoryMem and HoloCine, alongside improvements in inter-shot, intra-shot, and inter-scene consistency. These results provide evidence of improved facial identity preservation, beyond the overall visual similarity captured by the original metrics.

The updated comparison table includes quantitative results for SlotMem, VideoMemory, and Context Forcing, extending the evaluation to the recent methods highlighted by the reviewer. We will also expand the related-work discussion to clarify their connections to and differences from Memento.

Memento achieves the highest inter-shot, intra-shot, and inter-scene consistency among the evaluated methods. It also achieves the highest ArcFace similarity (0.2807), outperforming Context Forcing (0.2618), HoloCine (0.2592), SlotMem (0.2536), StoryMem (0.2515), and VideoMemory (0.2478). These results provide consistent evidence of improvements in both overall appearance consistency and facial identity preservation. The ArcFace results are consistent with the improvements observed under the subject-consistency metrics used in our paper.

Regarding OneStory, we could not obtain publicly available official code or pretrained checkpoints and therefore could not include it in the quantitative comparison under our evaluation protocol.

AuthorsOpenReview ↗

Response to Reviewer kVsw (Part 2)


Weakness 2.

Our goal is to preserve subject identity while maintaining semantic alignment and overall video quality, rather than maximize subject consistency alone. As discussed in Sec. 4.2, all ablation variants use the same fixed memory budget, and the two retrieval paths serve complementary purposes. The Story Query + Reconstruction variant allocates the entire memory budget to story-conditioned retrieval, prioritizing subject-relevant appearance evidence. Its highest scores on all three subject-consistency metrics support the intended role of story queries in retrieving subject-related memories, which align naturally with the reconstruction objective. However, without shot-conditioned retrieval, memory selection is not explicitly guided by the contextual requirements of the upcoming shot, such as scene layout and motion-related cues. These references are important for coherent next-shot generation and are specifically targeted by the shot queries.

Our full model therefore allocates two memory frames to story-conditioned retrieval and six to shot-conditioned retrieval. Compared with Story Query + Reconstruction, this allocation improves story/shot semantic consistency from 0.2571/0.2651 to 0.3063/0.2893 and background consistency from 0.9714 to 0.9805, while retaining subject-consistency scores close to those of the identity-focused variant. These results illustrate the benefit of combining identity-oriented evidence with shot-specific contextual references.


Weakness 4.

We will expand Appendix to include the omitted training details. We train Memento for 1,200 steps with a batch size of 64 on 16 NVIDIA H100 GPUs. We will add these details to Appendix and release our training and inference code to improve reproducibility.

Regarding presentation, we will revise the citation and adjust the layout of page 8 to improve readability.

AuthorsOpenReview ↗

Awaiting Your Thoughts on Our Rebuttal

Thank you very much for your thoughtful review and constructive comments. Your insightful feedback has greatly helped us improve the overall quality of the paper. We sincerely hope that our detailed responses have adequately addressed your concerns.

If our revisions and explanations have resolved the issues you raised, we anticipate your reconsideration of the overall assessment of our paper. Your recognition would mean a great deal to us.

If you have any further questions or suggestions, we would be more than happy to continue the discussion.

We truly appreciate the time and effort you have dedicated to reviewing our submission and our response.

Best regards,

Authors

Reviewer_hU8XOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents Memento, a memory-conditioned framework for autoregressive long video generation. The authors identify a limitation of existing memory-based approaches, which is standard generation objectives do not explicitly encourage the model to preserve and utilize identity-critical information. Therefore, recurring subjects may exhibit appearance drift across shots and scene transitions. To address this issue, Memento proposes three components: (1) a subject-anchored multi-task training objective that jointly optimizes next-shot generation and memory-based subject reconstruction, (2) a dual-query memory selection mechanism that separately retrieves long-range identity information and short-range contextual information, and (3) a subject-aware data curation pipeline that provides consistent subject descriptions across shots and reconstruction targets. Experiments on STbench and VBench demonstrate improvements in subject consistency over several representative baselines, including StoryMem and HoloCine.

Critical strengths
  1. The subject reconstruction task is a conceptually appealing approach to encouraging identity preservation.
  2. The proposed method demonstrates consistent improvements over existing approaches on several evaluation metrics.
Critical weaknesses
  1. Although the intuition makes sense, I feel like the reconstruction objective is pretty unbounded as it ultimately is just a memory conditioned generation using standard flow matching loss. It does not explicitly enforce identity preservation, and its effectiveness as identity-specific supervision is insufficiently validated. I feel like this just looks like image-video joint training, instead of explicitly preserving the identity.
  2. The "Subject Consistency" metrics uses models like DINO and ViCLIP, but the similarity is computed over the features of each shot. The score could be high if the background or other attributes are maintained well, but the actual subject could still be inconsistent.
  3. The experiments provide limited analysis of how generation quality changes when the shot number or sequence length increases. The paper demonstrates a five-minute generation example, but this is mostly qualitative results.
Questions for authors

See weaknesses

3 comments
AuthorsOpenReview ↗

Response to Reviewer hU8X (Part 1)

We thank the reviewer for the constructive suggestions on reconstruction supervision, identity-specific evaluation, and long-horizon generation. We address these points with additional ablations and quantitative evaluations below.


Weakness 1.

As discussed in Section 3.3, our reconstruction task encourages identity preservation rather than merely introducing an additional generation objective. The reconstruction branch is conditioned on the subject description and historical memory, without the current shot's scene or action caption. Since reconstruction targets are sampled across different shots, backgrounds and layouts may vary, while subject identity remains consistent. This encourages the model to learn persistent identity-related appearance cues from memory. Moreover, reconstruction and next-shot generation share the same DiT, allowing the identity-relevant representations learned through reconstruction to benefit video generation.

To evaluate the contribution of reconstruction supervision, we add an ablation with both story and shot queries but without the reconstruction task. We also introduce ArcFace-based facial identity similarity as a more direct identity-consistency measure. Specificly, for each video, we select the main character and identify all shots in which this character appears alone. We crop the character's face from all valid frames in these shots and uniformly sample 20 frames from the resulting temporally ordered collection. We extract ArcFace embeddings from the sampled face crops and compute cosine similarity for every pair of distinct frames. The video's facial identity consistency score is the mean over these 190 pairs. This protocol is used for both the reconstruction ablation and the baseline comparison below. This comparison evaluates the same dual-query retrieval design with and without reconstruction supervision.

MethodInter-shot ↑Intra-shot ↑Inter-scene ↑ArcFace ↑Background ↑Semantic (Story / Shot) ↑Aesthetic ↑
Baseline + shot & story queries0.7212‾\underline{0.7212}0.7918‾\underline{0.7918}0.7204‾\underline{0.7204}0.2692‾\underline{0.2692}0.9698‾\underline{0.9698}0.2827‾\underline{0.2827} / 0.2708‾\underline{0.2708}0.4895‾\underline{0.4895}
Baseline + shot & story queries + recon. task0.73380.85780.72680.28070.98050.3063 / 0.28930.4977

Bold, and underline‾\underline{\text{underline}} indicate the best, second-best results respectively. Story-level and shot-level semantic consistency are ranked separately.

Adding reconstruction supervision improves all three subject-consistency metrics: inter-shot consistency increases from 0.7212 to 0.7338, intra-shot consistency from 0.7918 to 0.8578, and inter-scene consistency from 0.7204 to 0.7268. And ArcFace similarity also improves from 0.2692 to 0.2807, providing face-specific evidence that the improvement extends beyond overall visual similarity.


Weakness 2.

We extend the baseline comparison with ArcFace-based facial identity similarity, computed on cropped faces rather than full-frame or shot-level representations. We also include three recent baselines suggested by Reviewer kVsw: SlotMem, VideoMemory, and Context Forcing, alongside StoryMem and HoloCine.

MethodInter-shot ↑Intra-shot ↑Inter-scene ↑ArcFace ↑Background ↑Semantic (Story / Shot) ↑Aesthetic ↑
StoryMem0.66060.8146‾\underline{0.8146}0.66920.25150.97320.2793‾\underline{0.2793} / 0.26810.4937
HoloCine0.57910.81280.65940.25920.97700.2720 / 0.2854‾\underline{0.2854}0.4568
SlotMem0.6906‾\underline{0.6906}0.76210.66900.25360.94820.2196 / 0.18660.4948
VideoMemory0.57150.77360.61750.24780.94700.2055 / 0.14760.5859
Context Forcing0.65610.78570.7174‾\underline{0.7174}0.2618‾\underline{0.2618}0.9803‾\underline{0.9803}0.2718 / 0.23490.4509
Memento (Ours)0.73380.85780.72680.28070.98050.3063 / 0.28930.4977‾\underline{0.4977}

Bold, underline‾\underline{\text{underline}}, and italics indicate the best, second-best, and third-best results, respectively. Story-level and shot-level semantic consistency are ranked separately.

Memento achieves the highest inter-shot, intra-shot, and inter-scene consistency among the evaluated methods. It also achieves the highest ArcFace similarity (0.2807), outperforming Context Forcing (0.2618), HoloCine (0.2592), SlotMem (0.2536), StoryMem (0.2515), and VideoMemory (0.2478). These results provide consistent evidence of improvements in both overall appearance consistency and facial identity preservation. The ArcFace results are consistent with the improvements observed under the subject-consistency metrics used in our paper.

AuthorsOpenReview ↗

Response to Reviewer hU8X (Part 2)

Weakness 3.

We add a quantitative long-horizon evaluation by extending STbench cases to five-minute, 60-shot sequences and comparing Memento with StoryMem. At shot indices 10, 20, 30, 40, 50, and 60, we compute inter-shot consistency and ArcFace similarity relative to the first shot. This fixed-reference evaluation measures whether subjects retain their initial appearance over increasing temporal distances, rather than merely remaining similar to neighboring shots. We restrict this five-minute comparison to StoryMem, as evaluating HoloCine at this sequence length exceeds the GPU memory capacity available in our experimental setup.

Shot indexMethodInter-shot ↑ArcFace ↑
10StoryMem0.6105‾\underline{0.6105}0.2594‾\underline{0.2594}
Memento (Ours)0.64760.2749
20StoryMem0.62530.2637
Memento (Ours)0.6040‾\underline{0.6040}0.2629‾\underline{0.2629}
30StoryMem0.5173‾\underline{0.5173}0.2428‾\underline{0.2428}
Memento (Ours)0.57390.2551
40StoryMem0.5257‾\underline{0.5257}0.1930‾\underline{0.1930}
Memento (Ours)0.55480.2545
50StoryMem0.4228‾\underline{0.4228}0.1055‾\underline{0.1055}
Memento (Ours)0.60160.2627
60StoryMem0.4476‾\underline{0.4476}0.1251‾\underline{0.1251}
Memento (Ours)0.54140.2523

The results show that Memento maintains better overall subject consistency and exhibits a slower decline than StoryMem as the generation horizon increases. Although consistency decreases over longer sequences, Memento better preserves the subject's initial appearance. The ArcFace scores further support this observation through a face-specific assessment of identity consistency relative to the first shot.

AuthorsOpenReview ↗

Awaiting Your Thoughts on Our Rebuttal

Thank you very much for your thoughtful review and constructive comments. Your insightful feedback has greatly helped us improve the overall quality of the paper. We sincerely hope that our detailed responses have adequately addressed your concerns.

If our revisions and explanations have resolved the issues you raised, we anticipate your reconsideration of the overall assessment of our paper. Your recognition would mean a great deal to us.

If you have any further questions or suggestions, we would be more than happy to continue the discussion.

We truly appreciate the time and effort you have dedicated to reviewing our submission and our response.

Best regards,

Authors

Reviewer_MTGaOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper tackles the persistent issue of identity drift in shot-by-shot long video generation. The authors propose explicitly incorporating "identity-critical cues" from historical memory into the training objective via a subject reconstruction task, coupled with a dual-path retrieval mechanism at both the story and shot levels. The technical pipeline is well-motivated and interpretable. Empirically, the method achieves a cross-shot subject consistency score of 0.7338 on STbench and sets new baselines on several subject-related metrics in VBench. I lean towards acceptance, but the paper requires stronger causal evidence regarding its core mechanisms and long-horizon scalability.

Critical strengths

Precise Problem Formulation: The paper clearly distinguishes between identity anchoring and local video continuity. Notably, the cross-scene setup does not rely on the last frame of the previous shot, making it highly targeted for realistic long-form generation. Coherent Mechanism Design: The training supervision aligns naturally with the retrieval module. The reconstruction task inherently depends on historical subject information, while the dual story/shot queries effectively retrieve context at different granularities. Comprehensive Evaluation: The assessment covers intra-shot, cross-shot, and cross-scene consistency, supplemented by human preference studies and thorough ablations.

Critical weaknesses
  1. Ambiguity in Ablation Attribution: In Table 3, the variant with "Story Query + Reconstruction" surprisingly outperforms the full model on three subject consistency metrics, while the full model only wins on semantic, aesthetic, and background metrics. The authors need to clarify the ultimate optimization goal and trade-offs. Please provide Pareto curves across different 𝜆 values and story/shot Top-K ratios to justify the final configuration.
  2. Isolating the Contribution of Reconstruction Supervision: The reconstruction task is currently tightly coupled with story retrieval. Under strictly controlled memory budgets, subject descriptions, and training steps, please add baselines including "generation-only," "reconstruction-supervision-only," "a generic auxiliary task with matched parameters," and "random/mismatched reconstruction targets." We need to verify whether the gains stem specifically from identity supervision rather than just the regularization effect of an extra loss.
  3. Robustness to Upstream Annotation Noise: The pipeline uses Qwen3-VL for subject listing and ByteTrack for selecting reconstruction frames. However, in long takes or occluded scenes, these tools are prone to track ID switches, appearance shifts, and subject confusion. Please evaluate the model's sensitivity to annotation/tracking noise. Include failure rate statistics, manually verified bad cases, and a discussion on how erroneous labels propagate to the final generation.
Questions for authors
  1. Under the same identity consistency constraint, can using only story queries achieve a better efficiency-quality trade-off?
3 comments
AuthorsOpenReview ↗

Response to Reviewer MTGa ( Part 1)

We thank the reviewer for the constructive suggestions on ablation attribution, reconstruction supervision, and annotation robustness. We address these points below.


Weaknesses 1.

We would like to emphasize that the objective of Memento is not only to maximize subject consistency alone, but also to balance long-range identity preservation with local contextual coherence for consistent long-form video generation.

To better illustrate this trade-off, we analyze different Story/Shot Top-K allocations under a fixed memory budget of eight frames.

We select the 2-story / 6-shot allocation to balance subject consistency and video quality. The reconstruction-enabled single-query variants in Table 3 already represent the two endpoints of this allocation: shot-only retrieval corresponds to (K-story, K-shot) = (0, 8), and story-only retrieval corresponds to (8, 0). Together with (2, 6), these results show how allocating the fixed memory budget to the two retrieval paths affects generation. The extended comparison below adds the intermediate allocations (4, 4) and (6, 2), with the total capacity kept at eight memory frames.

K-storyK-shotInter-shot ↑Intra-shot ↑Inter-scene ↑Background ↑Semantic (Story / Shot) ↑Aesthetic ↑
080.73190.72430.70820.97230.2799‾\underline{0.2799} / 0.2679‾\underline{0.2679}0.4901
260.73380.85780.72680.98050.3063 / 0.28930.4977
440.73420.86050.72960.9772‾\underline{0.9772}0.2616 / 0.26240.4960
620.7349‾\underline{0.7349}0.8630‾\underline{0.8630}0.7320‾\underline{0.7320}0.97430.2598 / 0.25170.4965
800.73530.86490.73360.97140.2571 / 0.26510.4974‾\underline{0.4974}

Bold, underline‾\underline{\text{underline}}, and italics indicate the best, second-best, and third-best results, respectively. Story-level and shot-level semantic consistency are ranked separately.

The Story-only configuration achieves the highest scores on all three subject consistency metrics. This observation is consistent with our design motivation: story queries retrieve persistent identity evidence from historical memory, which directly supports memory-based subject reconstruction. Increasing the story allocation beyond two frames provides modest gains in subject consistency, while yielding lower semantic and background scores than (2, 6). Meanwhile, Shot-only retrieval (0, 8) underperforms (2, 6) across all reported metrics, highlighting the complementary roles of the two paths. We therefore select (2, 6) as a balanced allocation, maintaining strong subject consistency while retaining shot-specific contextual references for better semantic alignment and generation quality.

We further examine the weight λ of the reconstruction objective to assess its contribution and its trade-off with generation quality.

λInter-shot ↑Intra-shot ↑Inter-scene ↑Background ↑Semantic (Story / Shot) ↑Aesthetic ↑
0.000.72120.79180.72040.96980.2827 / 0.27080.4895
0.250.72810.84900.72560.9781‾\underline{0.9781}0.2975‾\underline{0.2975} / 0.29030.4943
0.50 (Ours)0.73380.85780.7268‾\underline{0.7268}0.98050.3063 / 0.2893‾\underline{0.2893}0.4977
0.750.73500.85900.72650.97660.2962 / 0.28100.4948‾\underline{0.4948}
1.000.7341‾\underline{0.7341}0.8581‾\underline{0.8581}0.72920.97250.2863 / 0.27470.4917

Without reconstruction supervision (λ = 0), the model obtains lower scores on all three subject-consistency metrics. Increasing λ from 0 to 0.5 improves inter-shot consistency from 0.7212 to 0.7338, intra-shot consistency from 0.7918 to 0.8578, and inter-scene consistency from 0.7204 to 0.7268. These results support the contribution of reconstruction supervision to preserving subjects.

Beyond λ = 0.5, further gains in subject consistency are small and vary across metrics, while background consistency, scene and shot level text alignment, and aesthetic quality all decline relative to λ = 0.5. For example, increasing λ to 0.75 improves inter-shot by only 0.0012, while scene-level text alignment decreases from 0.3063 to 0.2962. Among the tested weights, λ = 0.5 provides a balanced operating point, achieving the highest background consistency, scene-level text alignment, and aesthetic quality while retaining near-best subject consistency. We therefore use λ = 0.5 in the final configuration.

AuthorsOpenReview ↗

Response to Reviewer MTGa (Part 2)

Weakness 2.

The new λ experiment includes the requested generation-only baseline (λ = 0). Within the same dual-query retrieval design, adding reconstruction supervision at λ = 0.5 improves inter-shot, intra-shot, and inter-scene subject consistency by 0.0126, 0.0660, and 0.0064, respectively. The paired story-query ablations in Table 3 also provide evidence that adding reconstruction improves all three subject-consistency metrics. Together, these comparisons demonstrate the contribution of reconstruction supervision to subject consistency under both story-only and dual-query retrieval.

We further explain why the reconstruction task encourages identity preservation rather than merely acting as an additional training objective. The reconstruction branch receives only the subject description as its textual condition, together with historical memory, rather than the current shot's scene or action caption. Since reconstruction targets are sampled from different shots, backgrounds and layouts can vary, while the subject identity remains the common reference. This encourages the model to retrieve persistent subject-appearance cues from memory rather than relying solely on contextual continuity. Because reconstruction and next-shot generation share the same DiT, the identity-relevant memory utilization learned through reconstruction can also benefit video generation.

Figure 3 provides qualitative support for this mechanism. It presents generated shots alongside memory-based reconstructions, highlighted by yellow boxes. The reconstructed subjects exhibit appearances consistent with those in the generated shots across different scenes and viewpoints. This agreement complements the quantitative ablations and supports our motivation of using reconstruction supervision to promote consistent subject appearances across both tasks.


Weakness 3.

Our subject-aware data curation pipeline combines frame selection with subject-correspondence verification. ByteTrack helps maintain tracking continuity under partial occlusion by associating low-confidence detections with existing tracks. We use its bounding boxes to identify candidate frames that may contain the reconstruction subject; these tracking outputs are subsequently checked against the global subject descriptions.

During Reconstruction Captioning, Qwen3-VL compares the visual subject in each candidate frame with the corresponding description established during Story Captioning. Approximately 98.2% of these checks are judged consistent by the VLM. Only frames passing this check proceed to reconstruction-caption generation and target acceptance. If an inconsistency is detected, we discard the entire story. This filtering step reduces the risk of tracking errors propagating into reconstruction supervision.

During early pipeline development, we observed poorer subject consistency with incorrect reconstruction annotations, which motivated the subject-correspondence verification and story-level rejection described above.

If necessary, we will conduct an additional controlled mismatched-target experiment. However, as this requires modifying the data pipeline, regenerating the training data, and retraining the model, quantitative results are not yet available.


Q1. Under the same identity consistency constraint, can Story-only retrieval achieve a better efficiency–quality trade-off?

As discussed above, under the same eight-frame memory budget, our 2-story / 6-shot configuration achieves higher story/shot semantic consistency (0.3063/0.2893 vs. 0.2571/0.2651) and background consistency (0.9805 vs. 0.9714). Thus, allocating the entire budget to story retrieval provides slightly stronger subject consistency, but sacrifices contextual information useful for next-shot generation.

AuthorsOpenReview ↗

Awaiting Your Thoughts on Our Rebuttal

Thank you very much for your thoughtful review and constructive comments. Your insightful feedback has greatly helped us improve the overall quality of the paper. We sincerely hope that our detailed responses have adequately addressed your concerns.

If our revisions and explanations have resolved the issues you raised, we anticipate your reconsideration of the overall assessment of our paper. Your recognition would mean a great deal to us.

If you have any further questions or suggestions, we would be more than happy to continue the discussion.

We truly appreciate the time and effort you have dedicated to reviewing our submission and our response.

Best regards,

Authors

Comments on the paper

1 comment
Area_Chair_5UuaOpenReview ↗

Discussion Phase

Dear Reviewers, The authors have posted the rebuttal. Please review their responses and engage in the discussion.

Best, AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.