acceptodds
Under review as a conference paper at ICLR 2027

CineMem: Attention-Aligned Fine-grained Memory Retrieval for Multi-Shot Long Video Generation

Abstract

Autoregressive video generation relies on memory retrieval to model long-range dependencies. This capability becomes particularly critical for multi-shot generation, where models must reconcile substantial visual changes across shot transitions with persistent semantic continuity. Existing methods tend to perform memory retrieval at the frame level and share the same historical context across all query tokens, overlooking the heterogeneous information needs and leading to redundant retrieval and inefficient use of historical memory. To bridge this gap, we introduce CineMem, an attention-aligned fine-grained memory retrieval framework that combines online memory organization with query-adaptive retrieval for autoregressive multi-shot video generation. Specifically, guided by intrinsic memory-access patterns revealed through attention analysis, we propose an online-clustered KV cache as a memory organization strategy over all historical tokens, enabling semantic-aware memory grouping and more precise retrieval. In addition, we introduce a query-level retrieval mechanism that provides broader access to historical information under a fixed retrieval budget, thereby improving long-term memory utilization. Together, these designs enable the multi-shot video generator to preserve long-range consistency across shot transitions, particularly in revisitation scenarios, while approximating full-memory access under a constrained memory budget. To facilitate evaluation, we further introduce a suite of task-specific, VLM-assisted metrics for faithful inter-shot consistency assessment. Extensive experiments demonstrate the effectiveness of our framework.

Primary area:
generative models
Keywords:
Multi-shot video generation, Autoregressive video generation, Memory mechanism

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_zE4DOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

CineMem studies memory retrieval for autoregressive multi-shot video generation. Starting from a Wan2.1-1.3B model adapted using resampled-context training, it organizes historical KV tokens into online, head-specific semantic clusters. Each query–head pair retrieves its own relevant clusters, alongside persistent and recent context. The paper also introduces PTCS, which evaluates both preservation of required content and execution of requested shot changes. Experiments include comparisons on 100 multi-shot prompts, controlled retrieval ablations, attention-retrieval analyses, efficiency measurements, and a human study.

Critical strengths
  1. The controlled retrieval ablations provide meaningful evidence for the central design. Table 2 applies alternative retrieval strategies to the same adapted model without additional training. Clustered query/head-wise retrieval obtains PTCS 0.8007, compared with 0.7280 for frame-based shared retrieval, 0.7525 for frame-based query-wise retrieval, and 0.7442 for clustered shared retrieval. This factorial comparison supports the combination of memory organization and individualized access, rather than attributing all improvements to additional training.

  2. The paper addresses an important evaluation failure in multi-shot generation. A model can obtain high visual similarity by failing to execute the requested transition. PTCS explicitly accounts for both continuity and intended change. The human study in Appendix E.2 provides supporting evidence for this concern and for the method’s perceived inter-shot consistency. This makes the evaluation contribution useful beyond the proposed architecture.

Critical weaknesses
  1. The evaluation does not directly validate the paper’s central claim of long-range revisitation. The main quantitative benchmark evaluates 15-second videos, whereas the 30-second extension is primarily qualitative. More fundamentally, PTCS operates on the middle frames of adjacent shots (Appendix E.1). Consider an A–B–A sequence in which a character appears in the first shot, disappears in the second, and returns in the third with an incorrect appearance. Adjacent-shot evaluation does not directly compare the two occurrences of that character; a text-conditioned evaluator may therefore reward individually plausible transitions without detecting the historical identity mismatch. This is a mismatch between the emphasized contribution and its principal evaluation, rather than simply a request for longer demonstrations. The human study provides useful supporting evidence, but reports aggregate consistency ratings rather than recurrence-specific accuracy by temporal gap. Consequently, the results establish improved multi-shot generation more convincingly than improved recovery of distant historical content.

  2. The evidence does not establish the incremental necessity and practical value of fully query-wise retrieval. The argument that the union of query–head selections covers nearly all historical memory is not sufficient: even independent random selections of a fraction (p) of memory have expected union coverage (1-(1-p)^m) over (m) selections. With (p=0.3) and (m=20), this already exceeds 99.9%. This analytical null example does not imply that the proposed retrieval is random; it shows that union coverage alone cannot establish useful or necessary specialization. Table 2 provides evidence for the combined design, but bundles query-wise and head-wise retrieval rather than cleanly isolating their contributions to generation quality. The query-group variant also achieves PTCS 0.8233 versus 0.8242 for fully query-wise retrieval, without uncertainty estimates or a corresponding quality–runtime comparison. The already-cited OmniMem [4], which uses query-shared selection with head-specific access, is particularly relevant to this unresolved distinction. Moreover, Algorithm 1 performs global refinement over historical tokens while the number of clusters grows with history, so a fixed retrieval budget does not establish bounded cache-maintenance cost. These omissions leave the central fine-granularity design insufficiently justified relative to simpler alternatives.

  3. The bibliography should be expanded to cite and substantively discuss [1–3]. They address complementary aspects of multi-scale memory, compact historical conditioning, and long-horizon consistency. In particular, OneStory is closely related to the multi-shot task and adaptive use of historical information. Please explain the differences in memory representation, retrieval granularity, temporal scope, and conditioning interface.

The concern is incomplete positioning and unresolved incremental benefit, not the absence of citations in isolation. Different backbones do not automatically require direct numerical comparison.

References

[1] Zhaochong An, Menglin Jia, Haonan Qiu, et al. OneStory: Coherent Multi-Shot Video Generation with Adaptive Memory. CVPR, 2026; preprint first released in 2025.

[2] Hongjun Wang, Lin Liu, Jianguo Li, and Tao Lin. Dual-Granularity Memory for Efficient Video Generation. CVPR, 2026.

[3] Jingyu Lin, Xinyi Shang, Peng Sun, Cunjian Chen, and Zhiqiang Shen. DynaMem: Consistent Long Video Generation via Hierarchical Memory and Motion Priors. ICML, 2026.

The paper contains a promising retrieval design and useful controlled ablations. However, the main evaluation does not directly measure distant revisitation, and the necessity of fully query-wise retrieval remains unresolved relative to simpler head-adaptive or query-group alternatives. These gaps affect the central contribution and are substantial enough to prevent an acceptance recommendation in the current version.

Questions for authors
  1. Can you evaluate nonadjacent recurrence directly, using earlier visual references and explicit checks of entity identity, appearance, and scene recovery? Please stratify results by recurrence gap and include uncertainty estimates. How does PTCS score an A–B–A sequence with an identity mismatch between the two A shots?

  2. Can you separate head-wise adaptation from query-wise adaptation under matched retrieved-token budgets and matched training? In particular, how do a head-specific/query-shared variant and the query-group variant compare with the full method in quality, latency, and memory use? Please compare the coverage analysis against a random-selection null or another control that distinguishes coverage from useful retrieval.

  3. Do the reported inference timings include clustering, global refinement, cache reordering, gathering, and synchronization? Please provide an end-to-end breakdown and clarify how these costs grow with historical length.

These questions concern evidence needed to assess the current contribution, rather than optional extensions to future work.

10 comments
AuthorsOpenReview ↗

Response to Reviewer zE4D (1/9)

We sincerely appreciate the reviewer's thorough and insightful comments, which have helped us further improve our manuscript. Following your suggestions, we conducted additional experiments and revised the manuscript to strengthen its completeness and overall quality.

 

W1: The evaluation does not directly validate the paper’s central claim of long-range revisitation. The main quantitative benchmark evaluates 15-second videos, whereas the 30-second extension is primarily qualitative. More fundamentally, PTCS operates on the middle frames of adjacent shots (Appendix E.1). Consider an A–B–A sequence in which a character appears in the first shot, disappears in the second, and returns in the third with an incorrect appearance. Adjacent-shot evaluation does not directly compare the two occurrences of that character; a text-conditioned evaluator may therefore reward individually plausible transitions without detecting the historical identity mismatch. This is a mismatch between the emphasized contribution and its principal evaluation, rather than simply a request for longer demonstrations. The human study provides useful supporting evidence, but reports aggregate consistency ratings rather than recurrence-specific accuracy by temporal gap. Consequently, the results establish improved multi-shot generation more convincingly than improved recovery of distant historical content.

Q1: Can you evaluate nonadjacent recurrence directly, using earlier visual references and explicit checks of entity identity, appearance, and scene recovery? Please stratify results by recurrence gap and include uncertainty estimates. How does PTCS score an A–B–A sequence with an identity mismatch between the two A shots?

 

We thank the reviewer for this valuable suggestion to strengthen the evaluation of long-range revisitation. Following this recommendation, we conducted additional quantitative evaluations on 30-second videos and introduced a more targeted evaluation protocol for non-adjacent revisit pairs, with results stratified by temporal gaps. We also refined the human study to separately assess revisit capability and updated the main quantitative results with 95% confidence intervals. These additions are reported in Tables 1, 4, and 9 of the revised manuscript.

  • Direct evaluation of non-adjacent revisit pairs. We introduce Shot Pair Consistency to directly evaluate historical content preservation beyond adjacent shots. Specifically, we measure subject, scene, and semantic similarity between shot pairs and report their average. Semantic similarity is computed using 8 uniformly sampled frames per shot. We evaluate all shot pairs to obtain All Shot Pair Consistency and identify revisit pairs within each prompt to obtain Revisit Shot Pair Consistency. To avoid rewarding models that fail to execute required transitions, we apply a penalty coefficient of α=0.5\alpha=0.5 to pairs without successful transitions.

 

Table A. Quantitative results of Shot Pair Consistency. Updated results are included in Table 1 with the corresponding 95% confidence interval in the revised manuscriptt.

MethodAll Shot Pair Consistency Subject ↑Scene ↑Semantic ↑Avg. ↑Revisit Shot Pair Consistency Subject ↑Scene ↑Semantic ↑Avg. ↑
LongLive20.55970.57710.56550.56740.64600.66950.65500.6568
ShotStream0.71720.81680.72340.75250.73730.83460.74190.7712
Infinity-RoPE0.46920.48120.46850.47300.45960.47700.46290.4665
Self-Forcing0.48790.50850.48000.49210.48770.52870.47390.4968
MemFlow0.48290.49460.48330.48690.47710.50100.48240.4868
MultiShotMaster0.50350.65250.51210.55600.71670.86430.73560.7722
HoloCine0.52530.66030.54210.57590.73810.87760.76910.7949
CineMem (Ours)0.75060.85610.73240.77970.89510.93590.88520.9054

We apologize for the suboptimal formatting due to the character limit.

AuthorsOpenReview ↗

Response to Reviewer zE4D (2/9)

(Continued from the previous response.)

  • Direct evaluation of non-adjacent revisit pairs. As shown in Table A, CineMem achieves the highest scores across all three similarity dimensions and their averages for both all-shot and revisit pairs. In particular, its Revisit Shot Pair Consistency reaches 0.9054, substantially outperforming the second-best model, HoloCine (0.7949). We further stratify revisit pairs by temporal distance (4–8, 9–12, and 13–18 chunks). As shown in Table B, CineMem consistently achieves the best results across all distance ranges, including the longest range of 13–18 chunks (0.9123 vs. 0.8081 for HoloCine). This directly demonstrates its superior ability to recover historical visual information over extended temporal gaps.

    Importantly, the original PTCS evaluates adjacent-shot transitions and does not directly compare the two occurrences of an entity in an A–B–A sequence. Therefore, an identity mismatch between the two A shots may not be captured by PTCS alone. Our newly introduced Revisit Shot Pair Consistency explicitly compares non-adjacent revisit pairs, providing a more direct assessment of subject identity, appearance, and scene preservation.

 

Table B. Revisit Shot Pair Consistency stratified by temporal distance

Method4–8 chunks ↑9–12 chunks ↑13–18 chunks ↑Overall ↑
Causal multi-shot models
LongLive20.6404 ± 0.20600.6625 ± 0.06050.6543 ± 0.08400.6568 ± 0.0231
ShotStream0.7723 ± 0.19740.7698 ± 0.06270.7722 ± 0.06560.7712 ± 0.0179
Causal multi-prompt models
Infinity-RoPE0.4740 ± 0.00690.4694 ± 0.01110.4634 ± 0.01230.4665 ± 0.0082
Self-Forcing0.4758 ± 0.00960.5166 ± 0.03700.4840 ± 0.03600.4968 ± 0.0244
MemFlow0.4934 ± 0.00190.4898 ± 0.02440.4839 ± 0.02040.4868 ± 0.0145
Bidirectional models
MultiShotMaster0.9496 ± 0.01180.7370 ± 0.07470.7793 ± 0.08190.7722 ± 0.0271
HoloCine0.9216 ± 0.02950.7576 ± 0.07120.8081 ± 0.05880.7949 ± 0.0246
CineMem (Ours)0.9871 ± 0.00220.8845 ± 0.04600.9123 ± 0.04010.9054 ± 0.0293
AuthorsOpenReview ↗

Response to Reviewer zE4D (3/9)

(Continued from the previous response.)

 

  • Quantitative evaluation on 30-second videos. Since not all baselines support long-video generation, we select ShotStream, which formulates multi-shot generation as next-shot generation, as a representative baseline for 30-second video evaluation. As shown in Table C (Table 9 in the revised manuscript), CineMem achieves better results in transition control, text-video similarity, intra-shot subject consistency, and inter-shot similarity. More importantly, CineMem achieves higher All Shot Pair Consistency (0.7579 vs. 0.7317) and Revisit Shot Pair Consistency (0.8104 vs. 0.7329), providing further evidence of its effectiveness in long-video generation and historical content recovery.
  • Revisit-specific human evaluation. We further refine the human study by separating Revisit Capability from the general Inter-shot Consistency dimension, directly evaluating whether previously observed subjects and scenes are consistently recovered after intervening shots. As shown in Table D, CineMem achieves the highest score, which provides complementary human evaluation evidence for CineMem's long-range revisitation capability.

Finally, we report 95% confidence intervals for all major quantitative results in the revised manuscript and standard deviations for the user study, providing a more comprehensive assessment of the reliability of our results.

 

Table C. Quantitative comparison on 30-second videos. Values are reported as mean ± half-width of 95% paired bootstrap confidence intervals.

Table C-1. Video-level, intra-shot, and inter-shot metrics.

MethodTransition Control ↑Text-Video Sim. ↑Aesthetic ↑Subject Cons. ↑Background Cons. ↑Inter-shot Sim. ↑PCP ↑PTA ↑PTCS ↑
ShotStream0.9034 ± 0.03630.2093 ± 0.02450.5717 ± 0.02220.9605 ± 0.00600.9589 ± 0.00560.6580 ± 0.04020.9653 ± 0.01490.6001 ± 0.07300.6203 ± 0.0779
CineMem (Ours)0.9927 ± 0.00910.2240 ± 0.01980.5653 ± 0.01810.9617 ± 0.00850.9564 ± 0.00520.6903 ± 0.03820.8989 ± 0.04160.7536 ± 0.05970.7480 ± 0.0560

Table C-2. Shot Pair Consistency on 30-second videos.

MethodAll Subject ↑All Scene ↑All Semantic ↑All Avg. ↑Revisit Subject ↑Revisit Scene ↑Revisit Semantic ↑Revisit Avg. ↑
ShotStream0.7153 ± 0.03240.7902 ± 0.02080.6896 ± 0.04000.7317 ± 0.02830.7079 ± 0.04180.8100 ± 0.02900.6809 ± 0.04020.7329 ± 0.0385
CineMem (Ours)0.7529 ± 0.04130.8167 ± 0.02070.7042 ± 0.03330.7579 ± 0.02860.7841 ± 0.04040.8791 ± 0.02650.7680 ± 0.04360.8104 ± 0.0398

 

Table D. User study results (80 prompts). Scores are reported as mean ± standard deviation (SD) on a 1–5 scale. Higher is better.

MethodOverall Video Quality ↑Intra-shot Consistency ↑Transition Coherence ↑Revisit Capability ↑
ShotStream3.25 ± 0.443.45 ± 0.523.43 ± 0.502.53 ± 0.56
Self-Forcing2.05 ± 0.482.90 ± 0.361.40 ± 0.491.37 ± 0.49
Infinity-RoPE2.52 ± 0.653.70 ± 0.461.25 ± 0.441.27 ± 0.45
LongLive23.48 ± 0.553.67 ± 0.502.87 ± 0.603.13 ± 0.57
MemFlow2.93 ± 0.513.27 ± 0.451.27 ± 0.451.15 ± 0.36
HoloCine3.22 ± 0.883.60 ± 0.793.92 ± 0.893.62 ± 0.49
MultiShotMaster3.90 ± 0.653.82 ± 0.504.01 ± 0.634.05 ± 0.40
CineMem (Ours)4.12 ± 0.324.15 ± 0.374.55 ± 0.504.43 ± 0.50
Reviewer_yARuOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes CineMem, a fine-grained memory retrieval framework for autoregressive multi-shot video generation. It introduces an online-clustered KV cache and query-adaptive retrieval to improve access to historical information and maintain cross-shot consistency. Experiments demonstrate improved multi-shot generation performance, supported by ablation studies, attention analysis, and a VLM-assisted evaluation metric for inter-shot consistency.

Critical strengths
  1. Well-motivated memory retrieval design. The paper presents an interesting approach to organizing and retrieving historical information for multi-shot video generation. The design is motivated by an analysis of full-memory attention, which reveals cross-frame similarities among historical tokens and heterogeneous attention patterns across queries and heads. This provides a clear motivation for combining online clustering with query-adaptive retrieval.

  2. Task-specific evaluation design. The paper identifies an important limitation of conventional similarity-based metrics for multi-shot video generation: high similarity does not necessarily indicate successful shot transitions. The proposed PTCS metric considers both semantic continuity and intended changes across shots, providing a useful perspective for evaluating cross-shot consistency.

Critical weaknesses
  1. Limited evaluation. The main evaluation uses only 100 prompts, with 30 prompts included in the human study. The paper also relies heavily on its newly proposed PTCS metric to demonstrate improvements. Given the relatively small margin over MultiShotMaster (0.824 vs. 0.798), I find the current evaluation insufficient to establish a clear advantage over existing methods.

  2. Unclear necessity of fully query-wise retrieval. The query-group variant achieves nearly identical performance to query-wise retrieval (0.8233 vs. 0.8242), suggesting that independent retrieval for every query may not be necessary. I am also concerned about the reliance on feature similarity for memory clustering. Visually similar characters may be grouped together, while an object and its surrounding context may be separated. The paper does not directly examine how these cases affect generation consistency.

Questions for authors
  1. Necessity of query-wise retrieval. The query-group variant achieves nearly identical PTCS to fully query-wise retrieval (0.8233 vs. 0.8242). What are the practical advantages of performing independent retrieval for every query, particularly considering the additional computational overhead?

  2. Limitations of similarity-based clustering. How does the proposed clustering handle visually similar but distinct entities, such as multiple characters in the same scene? Also, could retrieving tokens based on feature similarity miss important contextual information, such as spatial relationships between objects and their surroundings? I would be interested in understanding how these cases affect generation consistency.

  3. Reliability of the evaluation. The main comparison is based on 100 prompts, and CineMem's PTCS improvement over MultiShotMaster is relatively modest (0.824 vs. 0.798). Does the human evaluation show a clear preference for CineMem over MultiShotMaster, particularly in challenging revisitation scenarios?

9 comments
AuthorsOpenReview ↗

Response to Reviewer yARu (1/4)

We sincerely appreciate the reviewer's time and thoughtful, constructive feedback. Following your suggestions, we have conducted additional experiments and revised the manuscript to address the concerns raised.

 

W1: Limited evaluation. The main evaluation uses only 100 prompts, with 30 prompts included in the human study. The paper also relies heavily on its newly proposed PTCS metric to demonstrate improvements. Given the relatively small margin over MultiShotMaster (0.824 vs. 0.798), I find the current evaluation insufficient to establish a clear advantage over existing methods.

Q3: Reliability of the evaluation. The main comparison is based on 100 prompts, and CineMem's PTCS improvement over MultiShotMaster is relatively modest (0.824 vs. 0.798). Does the human evaluation show a clear preference for CineMem over MultiShotMaster, particularly in challenging revisitation scenarios?

 

We thank the reviewer for the valuable suggestions regarding evaluation scale and reliability. In response, we expanded our quantitative evaluation to 150 prompts and conducted a larger user study on 80 prompts, with results reported in Tables 4 and 5 of the revised manuscript. To further reduce reliance on the VLM-based PTCS metric, we also introduce a feature-similarity-based Shot Pair Consistency evaluation, providing an independent assessment of long-range consistency and revisit capability (updated Table 1 in revised manuscript). Additionally, we provide further qualitative analyses of CineMem against MultiShotMaster in Figure 18. These additional results offer more comprehensive evidence of CineMem's effectiveness.

  • Larger-scale quantitative evaluation and user study.
    • Quantitative Results We expanded the quantitative evaluation to 150 prompts and selected two representative baselines with strong transition capabilities, ShotStream and MultiShotMaster, for comparison. As shown in Table A, CineMem achieves better results in overall video quality, transition quality, and inter-shot consistency, including the newly introduced Shot Pair Consistency metrics. In particular, CineMem achieves substantially higher Revisit Shot Pair Consistency, demonstrating its superior long-range history recovery capability.
    • User Study We also expanded the user study to 80 prompts, reporting mean opinion scores (MOS) on a 1–5 scale. As shown in Table B, CineMem achieves the highest scores across all four evaluation dimensions. Notably, it receives a revisit capability score of 4.43, compared with 4.05 for MultiShotMaster, confirming a clear human preference for CineMem in challenging revisitation scenarios.
  • Complementary evaluation beyond PTCS. Since PTCS relies on VLM-based evaluation, we introduce a complementary feature-similarity-based Shot Pair Consistency metric to provide an additional perspective on inter-shot consistency. In particular, it explicitly evaluates revisit shot pairs to assess long-range historical information recovery. To avoid overestimating models that fail to execute required transitions, we apply a penalty coefficient of α=0.5\alpha=0.5 to shot pairs without successful transitions. As shown in the Table A-2, CineMem consistently outperforms the baselines, particularly in Revisit Shot Pair Consistency, further supporting its long-term consistency advantages.

 

 

Table A. Larger-scale quantitative evaluation (150 prompts). We report mean scores in these tables. The corresponding 95% confidence intervals are provided in the revised manuscript.

Table A-1. General generation quality and PTCS metrics.

MethodTransition ↑Text ↑Aesthetic ↑Intra-sub ↑Intra-bg ↑Inter-sim ↑PCP ↑PTA ↑PTCS ↑
ShotStream0.92750.21980.57510.96370.96160.73930.95870.65340.6800
MultiShotMaster0.97890.22720.52550.93570.93290.62150.91900.75530.7772
CineMem (Ours)0.99810.23080.57870.96830.96490.74690.94060.79910.8078

Table A-2. Shot Pair Consistency metrics.

MethodA-sub ↑A-sce ↑A-sem ↑A-avg ↑R-sub ↑R-sce ↑R-sem ↑R-avg ↑
ShotStream0.71290.80980.70380.74220.71430.81540.69720.7423
MultiShotMaster0.47680.63370.47940.53000.59710.79800.60170.6656
CineMem (Ours)0.73300.84550.71200.76350.87010.91720.84940.8789

A = All Shot Pair Consistency; R = Revisit Shot Pair Consistency. sub = Subject; sce = Scene; sem = Semantic; avg = Average.

AuthorsOpenReview ↗

Response to Reviewer yARu (2/4)

(Continued from the previous response.)

  • Further analysis against MultiShotMaster. Notably, MultiShotMaster is a bidirectional model, whereas CineMem operates under an autoregressive generation paradigm, where future information is inaccessible during generation. Despite this causal constraint, CineMem achieves better quantitative and human evaluation results. To further understand this advantage, we provide additional qualitative comparisons in Figure 18 (revised manuscript). MultiShotMaster exhibits noticeable first-frame artifacts, visual inconsistencies, and less reliable transition control, which may contribute to its lower evaluation scores.

 

Table B. User study results (80 prompts). Scores are reported as mean ± standard deviation (SD) on a 1–5 scale. Higher is better.

MethodOverall Video Quality ↑Intra-shot Consistency ↑Transition Coherence ↑Revisit Capability ↑
ShotStream3.25 ± 0.443.45 ± 0.523.43 ± 0.502.53 ± 0.56
Self-Forcing2.05 ± 0.482.90 ± 0.361.40 ± 0.491.37 ± 0.49
Infinity-RoPE2.52 ± 0.653.70 ± 0.461.25 ± 0.441.27 ± 0.45
LongLive23.48 ± 0.553.67 ± 0.502.87 ± 0.603.13 ± 0.57
MemFlow2.93 ± 0.513.27 ± 0.451.27 ± 0.451.15 ± 0.36
HoloCine3.22 ± 0.883.60 ± 0.793.92 ± 0.893.62 ± 0.49
MultiShotMaster3.90 ± 0.653.82 ± 0.504.01 ± 0.634.05 ± 0.40
CineMem (Ours)4.12 ± 0.324.15 ± 0.374.55 ± 0.504.43 ± 0.50
AuthorsOpenReview ↗

Response to Reviewer yARu (3/4)

W2: Unclear necessity of fully query-wise retrieval. The query-group variant achieves nearly identical performance to query-wise retrieval (0.8233 vs. 0.8242), suggesting that independent retrieval for every query may not be necessary. ...... Q1: Necessity of query-wise retrieval. The query-group variant achieves nearly identical PTCS to fully query-wise retrieval (0.8233 vs. 0.8242). What are the practical advantages of performing independent retrieval for every query, particularly considering the additional computational overhead?

 

We thank the reviewer for raising this important question about the necessity of query-wise retrieval. This prompted us to further clarify the implementation of the query-group variant and evaluate its actual efficiency and retrieval quality.

  • Clarification of the query-group variant. Unlike conventional content-agnostic query grouping based on predefined positional partitions, our proposed query-group variant adopts content-aware grouping of query tokens within the current chunk. Specifically, it recursively partitions queries according to their spatial and semantic similarity using an adaptive tree structure, allowing queries with similar content preferences to share retrieved memory. This variant serves as an approximation of CineMem's query-wise retrieval strategy. We apologize that its implementation was not sufficiently clarified in the original manuscript.

  • Efficiency and generation quality comparison. We conducted additional efficiency experiments to compare the two strategies (Tables C and D; detailed analyses are provided in Table 1 and Appendix D of the revised manuscript). As shown in Table C, the additional content-aware grouping operations introduce considerable overhead: the query-group variant increases end-to-end runtime by 3.4%, with retrieval selection/query preparation time increasing by 78.0%. Thus, query grouping does not yield practical efficiency gains. Meanwhile, as shown in Table D, CineMem achieves better results in Revisit Shot Pair Consistency (0.9054 vs. 0.8601) and All Shot Pair Consistency (0.7797 vs. 0.7592), while also reducing end-to-end runtime from 9.87 to 9.55 minutes. These results demonstrate that fully query-wise retrieval provides a better efficiency-quality trade-off than the query-group approximation.

  • Direct evaluation of retrieval quality. To further investigate the necessity of independent retrieval, we evaluate the retrieval quality of both strategies against the top 30% most attended historical tokens identified by the full-memory model. As shown in Table E (visualization in Figure 11 of the revised manuscript), query-wise retrieval achieves higher precision (62.58% vs. 54.58%) and recall (68.07% vs. 52.31%) than query grouping. This indicates that sharing retrieved memory among grouped queries compromises the ability to retrieve query-specific relevant historical information, further validating the importance of fine-grained query-wise retrieval.

 

 

Table C. Runtime comparison between query-wise and query-group retrieval

MethodEnd-to-end Runtime (21 chunks) ↓Retrieval Selection / Query Preparation ↓Memory Gathering ↓
CineMem (Query-wise)9.55 min0.23 min0.82 min
Query-group Variant9.87 min0.41 min0.98 min
Relative Change+3.4%+78.0%+19.5%

   

Table D. Efficiency and generation quality comparison. The results are obtained using the trained version.

MethodPTCS ↑Revisit Pair Sim ↑All Pair Sim ↑End-to-end Runtime ↓Peak GPU Memory ↓
CineMem (Query-wise)0.82420.90540.77979.55 min30.47 GiB
Query-group Variant0.82330.86010.75929.87 min30.48 GiB

   

Table E. Retrieval quality comparison

MethodRetrieval Precision ↑Retrieval Recall ↑
CineMem (Query-wise)62.58%68.07%
Query-group Variant54.58%52.31%
Reviewer_TbpbOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces CineMem, a fine-grained memory retrieval framework for autoregressive multi-shot video generation. Motivated by heterogeneous historical attention patterns across query tokens and attention heads, the method combines an online-clustered KV cache with query- and head-wise retrieval. Historical tokens are grouped by feature similarity rather than frame membership, allowing different queries to access relevant historical content under a constrained per-query attention budget. The authors additionally introduce a prompt-conditioned evaluation metric, PTCS, to jointly assess transition execution and semantic preservation. Experiments demonstrate improvements over coarse-grained retrieval strategies and competitive performance against existing multi-shot video generation models.

Critical strengths

The method is well motivated by attention analyses and addresses the important challenge of preserving visual content across shot changes.

The plug-in ablations help isolate the effects of memory organization and retrieval design, with further support from efficiency analysis and human evaluation.

Critical weaknesses

PTCS evaluates the middle frames of adjacent shots using the global and shot-level prompts. However, when an object or character disappears for several shots and subsequently reappears, this protocol does not directly compare its earlier and returning appearances, potentially overlooking changes in identity or visual details. Although Table 7 groups results by revisit distance, it still relies on PTCS. A direct evaluation of non-adjacent revisit pairs would provide stronger evidence of long-term identity preservation.

Although CineMem reduces theoretical computational cost compared with full-history attention, its retrieval overhead is substantially higher than that of frame-level retrieval, yielding only a marginal reduction in total FLOPs relative to the latter. While the paper provides runtime and memory measurements for a 42-chunk rollout, further quantitative comparisons and scaling analyses on longer videos are needed to establish the practical efficiency benefits of the proposed method.

The novelty of CineMem's clustered KV cache and query-wise, head-wise retrieval relative to existing memory retrieval methods, particularly OmniMem and CausalCine, is not sufficiently established. Although the paper acknowledges that CausalCine's weights are unavailable, the current comparisons do not adequately demonstrate the advantages of CineMem over closely related retrieval strategies. A clearer method-level comparison and stronger controlled experiments against reproducible alternatives are needed to substantiate its technical novelty.

Questions for authors

Can the authors provide direct evaluations of non-adjacent revisit pairs to verify whether object identities and visual details are preserved when objects reappear after several shots, beyond the PTCS results in Table 7?

Can the authors provide quantitative end-to-end latency and memory comparisons against frame-level retrieval, together with scaling results for longer rollouts, to clarify whether the additional retrieval overhead is justified?

Can the authors clarify the key technical differences between CineMem and OmniMem/CausalCine and provide controlled comparisons against reproducible alternative retrieval strategies to better establish the contribution of the proposed design?

8 comments
AuthorsOpenReview ↗

Response to Reviewer Tbpb (1/3)

We sincerely appreciate your thoughtful and constructive comments on our work. Your feedback has been particularly helpful in identifying opportunities to strengthen our evaluation, further clarify the efficiency analysis, and better articulate the technical contributions and novelty of CineMem. In response to your suggestions, we have revised the manuscript and conducted additional experiments to address the concerns you raised.

 

W1: PTCS evaluates the middle frames of adjacent shots using the global and shot-level prompts. However, when an object or character disappears for several shots and subsequently reappears, this protocol does not directly compare its earlier and returning appearances, potentially overlooking changes in identity or visual details. Although Table 7 groups results by revisit distance, it still relies on PTCS. A direct evaluation of non-adjacent revisit pairs would provide stronger evidence of long-term identity preservation. Q1: Can the authors provide direct evaluations of non-adjacent revisit pairs to verify whether object identities and visual details are preserved when objects reappear after several shots, beyond the PTCS results in Table 7?

 

We thank the reviewer for this valuable suggestion, which motivated us to introduce a more targeted evaluation of long-range consistency based on shot pairs. Following the reviewer's recommendation, we conducted additional evaluations of All Shot Pair Consistency and Revisit Shot Pair Consistency, providing direct evidence of identity preservation beyond adjacent shots. The main results are summarized in Table A (also incorporated into the revised Table 1), with details added to Section 4.2 and Appendix E.1.

  • Evaluation metrics. We extend pairwise similarity evaluation from adjacent shots to all shot pairs, measuring subject, scene, and semantic similarity and reporting their average. Semantic similarity is computed using 8 uniformly sampled frames per shot. We report the average across all shot pairs as All Shot Pair Consistency. Additionally, we identify revisit pairs for each prompt, where previously introduced content reappears after intervening shots, and compute their average similarity as Revisit Shot Pair Consistency.
  • Transition-aware penalty. To avoid favoring models that achieve high similarity simply by failing to execute transitions, we introduce a penalty coefficient α\alpha. For any shot pair without the required transition, its raw similarity score is multiplied by α\alpha. For non-adjacent pairs, the penalty applies when none of the intervening shots executes a transition. We set α=0.5\alpha=0.5.
  • Results. CineMem achieves 0.9054 in Revisit Shot Pair Consistency, substantially outperforming the second-best model, HoloCine (0.7949). Notably, HoloCine also achieves a relatively high transition-control score, suggesting that CineMem's advantage stems from stronger revisit consistency rather than missed transitions by competing models.

 

Table A. Quantitative results of Shot Pair Consistency. Updated results are included in Table 1 with the corresponding 95% confidence interval in the revised manuscriptt.

MethodAll Shot Pair Consistency Subject ↑Scene ↑Semantic ↑Avg. ↑Revisit Shot Pair Consistency Subject ↑Scene ↑Semantic ↑Avg. ↑
LongLive20.55970.57710.56550.56740.64600.66950.65500.6568
ShotStream0.71720.81680.72340.75250.73730.83460.74190.7712
Infinity-RoPE0.46920.48120.46850.47300.45960.47700.46290.4665
Self-Forcing0.48790.50850.48000.49210.48770.52870.47390.4968
MemFlow0.48290.49460.48330.48690.47710.50100.48240.4868
MultiShotMaster0.50350.65250.51210.55600.71670.86430.73560.7722
HoloCine0.52530.66030.54210.57590.73810.87760.76910.7949
CineMem (Ours)0.75060.85610.73240.77970.89510.93590.88520.9054
AuthorsOpenReview ↗

Response to Reviewer Tbpb (2/3)

W2: Although CineMem reduces theoretical computational cost compared with full-history attention, its retrieval overhead is substantially higher than that of frame-level retrieval, yielding only a marginal reduction in total FLOPs relative to the latter. While the paper provides runtime and memory measurements for a 42-chunk rollout, further quantitative comparisons and scaling analyses on longer videos are needed to establish the practical efficiency benefits of the proposed method.

Q2: Can the authors provide quantitative end-to-end latency and memory comparisons against frame-level retrieval, together with scaling results for longer rollouts, to clarify whether the additional retrieval overhead is justified?

We thank the reviewer for suggesting a more comprehensive efficiency analysis. In the original Figure 11 (Figure 13 in the revised manuscript), we reported the runtime and memory consumption over a 42-chunk rollout (corresponding to a 30-second video) on H200 GPU, showing that CineMem incurs only a modest efficiency overhead compared with frame-level retrieval. To provide a more detailed assessment, we conducted additional experiments on 4 A800 GPUs (instead of H200 due to hardware availability constraints, for which we apologize), including quantitative latency and memory comparisons (Table B) and scaling analyses over a 60-chunk rollout (Tables C and D). Additional visualizations are provided in Figures 12, 14, and 15 of Appendix D in the revised manuscript.

  • End-to-end latency and memory comparison.
    • Baseline We compare CineMem against a frame-level retrieval baseline with shared queries, corresponding to the frame-level retrieval strategy discussed in our paper and analogous to the retrieval design in CausalCine. We reproduce this baseline by plugging it into our trained full-memory model, with a matched retrieved-token budget to ensure a fair comparison.
    • Results As shown in Table B, frame-level retrieval is slightly faster, as expected from its coarser retrieval granularity. However, CineMem incurs only 3.13% additional denoising time (9.55 vs. 9.26 min) and 1.13% additional peak memory (30.47 vs. 30.13 GiB), while improving PTCS by 0.0727 (0.8007 vs. 0.7280) and Revisit Shot Pair Consistency by 0.0356 (0.8900 vs. 0.8544). These results demonstrate that CineMem achieves substantial quality improvements with marginal additional overhead in this setting.
  • Scaling analysis on 60-chunk rollouts. We further evaluate per-chunk denoising time and peak GPU memory throughout a 60-chunk rollout. As shown in Tables C and D, memory consumption scales similarly for both methods, with only a 2.49% difference at chunk 60 (72.04 vs. 70.29 GiB). This is expected because both methods retain the full historical KV cache. Regarding denoising time, CineMem exhibits a steeper growth trend. Nevertheless, the overhead should be considered alongside CineMem's substantial consistency improvements. Moreover, the retrieval operations offer further optimization opportunities through specialized implementations, such as Triton kernels. Together with the marginal overhead observed in the 30-second generation setting, these results demonstrate a favorable efficiency-quality trade-off for CineMem. We have incorporated the additional quantitative results and visualizations into Appendix D of the revised manuscript.

Table B. End-to-end latency and memory comparison. CineMem results are obtained using its plug-in version.

MethodDenoising Time (min) ↓Peak Memory (GiB) ↓PTCS ↑Revisit Shot Pair Consistency ↑
CineMem (Clustering Retrieval, Query-wise)9.5530.470.80070.8900
Frame-level Retrieval (Query-shared)9.2630.130.72800.8544

Table C. Per-chunk denoising time in seconds scaling on 60-chunk rollouts

MethodChunk 10Chunk 20Chunk 30Chunk 40Chunk 50Chunk 60
CineMem (Ours)28.7730.4332.0633.7135.0737.94
Frame-level Retrieval28.7229.0829.2129.3829.5430.51

Table D. Peak GPU memory in GiB scaling on 60-chunk rollouts

MethodChunk 10Chunk 20Chunk 30Chunk 40Chunk 50Chunk 60
CineMem (Ours)19.1129.3339.8050.3761.1572.04
Frame-level Retrieval18.9929.1239.3049.6359.9670.29
AuthorsOpenReview ↗

Response to Reviewer Tbpb (3/3)

W3: The novelty of CineMem's clustered KV cache and query-wise, head-wise retrieval relative to existing memory retrieval methods, particularly OmniMem and CausalCine, is not sufficiently established. Although the paper acknowledges that CausalCine's weights are unavailable, the current comparisons do not adequately demonstrate the advantages of CineMem over closely related retrieval strategies. A clearer method-level comparison and stronger controlled experiments against reproducible alternatives are needed to substantiate its technical novelty.

Q3: Can the authors clarify the key technical differences between CineMem and OmniMem/CausalCine and provide controlled comparisons against reproducible alternative retrieval strategies to better establish the contribution of the proposed design?

 

We thank the reviewer for the helpful suggestion to better clarify the technical novelty of CineMem. In the original manuscript, we included an ablation comparing frame-level memory grouping with shared-query retrieval (denoted as F + Shared), which is equivalent to the retrieval mechanism of CausalCine. We apologize that this connection was not sufficiently explicit and have clarified the labeling in the revised manuscript. To further establish CineMem's technical contributions, we provide additional method-level and experimental comparisons. In revised manuscript, a visual illustration of the key differences is added to Figure 6, while quantitative and qualitative comparisons are provided in Table 2 and Figure 22, respectively.

  • Key technical differences (Figure 6 and revised Section 2) CineMem differs from CausalCine and OmniMem in two key aspects: KV cache organization and retrieval granularity.
    • CausalCine and OmniMem. CausalCine organizes the KV cache at the frame level and shares the retrieved frames across all queries and attention heads. OmniMem further divides frames into smaller blocks and retrieves at the block level, while grouping queries according to predefined positional partitions and performing head-wise retrieval. Despite their different granularities, both methods rely on content-agnostic grouping strategies: memory tokens are partitioned according to predefined frame, block, or positional boundaries, without explicitly considering their feature similarity or semantic relationships.
    • CineMem (Ours). In contrast, CineMem adopts content-aware clustering to organize memory tokens, grouping tokens with similar features and strong correlations in the attention feature space. This provides a finer-grained and more adaptive memory organization than frame-level grouping, while avoiding the excessive number of memory candidates that can arise from uniform block partitioning. Moreover, CineMem performs query-wise and head-wise retrieval, providing each query-head with the historical tokens it attends to most during attention computation. By tailoring memory retrieval to the specific attention preferences of each query-head, CineMem achieves truly attention-aligned historical information retrieval.
  • Controlled comparisons (updated Table 2 and Figure 22). To isolate the effects of retrieval design, we reproduce CausalCine and OmniMem through plug-in implementations on the same trained full-memory model, ensuring a controlled comparison without changing the underlying model. As shown in Table F, CineMem outperforms both alternatives in PTCS and Revisit Shot Pair Consistency, demonstrating improved transition execution and long-range consistency.

 

Table E. Method-level comparison

MethodKV Cache OrganizationQuery Retrieval Granularity
CausalCineFixed, content-agnostic frame-level groupingShared across all queries and heads
OmniMemFixed, content-agnostic block-level groupingPredefined query groups; head-wise retrieval
CineMem (Ours)Content-aware feature-based clusteringQuery-wise and head-wise retrieval

 

Table F. Quantitative comparison of retrieval strategies. CineMem results are obtained using its plug-in version.

MethodPTCS ↑Revisit Shot Pair Consistency ↑
CineMem (Ours)0.80070.8900
CausalCine (Reproduced)0.72800.8544
OmniMem (Reproduced)0.69310.7745

 

We sincerely thank you for your time and thoughtful feedback. Your comments have been invaluable in improving our work and clarifying its contributions. We hope our responses have addressed your concerns, and we would be happy to answer any further questions during the rebuttal period.

Best regards, Authors

Comments on the paper

2 comments
AuthorsOpenReview ↗

General Response to All Reviewers

We sincerely thank all reviewers for their time and thoughtful feedback. We appreciate their recognition of CineMem's well-motivated memory retrieval design and clear attention-based motivation (Tbpb, yARu), meaningful controlled retrieval ablations (Tbpb, zE4D), and task-specific evaluation design that jointly considers semantic continuity and shot transition execution (yARu, zE4D). We are also encouraged by the recognition of the importance of addressing long-range consistency in autoregressive multi-shot video generation (Tbpb, yARu, zE4D).

Scope of revision. Following the reviewers' suggestions, we have conducted additional experiments and revised the manuscript in three main aspects:

  • Comprehensive evaluation (Section 4.2, Appendix E).
  • Efficiency and scalability analysis (Appendix D).
  • Validation of query-wise retrieval (Appendices B and F).

We have also expanded the related-work discussion to better clarify the technical contributions of CineMem (Section 2). All newly added or revised content is highlighted in blue.

Hi Reviewers,

The authors have posted their rebuttal along with new results. Please review their responses, engage with the authors to discuss any remaining concerns.

Best, Area Chair

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.