- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
KVMEM introduces a learned compressed-memory system for autoregressive video generation. A KV compression adapter combines local spatial summaries with learned global residual corrections, and is trained using attention reconstruction and generation-level DMD supervision. Compressed entries are stored in a fixed-capacity hierarchical bank, with content- and time-aware merging across temporal bins. The intended benefit is broader historical coverage without expanding the active attention budget. Evaluation includes minute-long single-prompt and multi-scene generation, component and memory-allocation ablations, throughput measurements, and a user study.
Critical strengths
-
The method offers a coherent approach to an important memory trade-off. Learning spatial summaries while progressively merging older history separates spatial detail from temporal coverage. The pre-RoPE representation and explicit integration with the recent context provide a reasonably concrete design for bounded-context generation.
-
The component results indicate that both learned compression and temporal organization matter. In Table 3(a), imaging quality falls from 72.16 to 66.74 when the adapter is replaced and to 65.49 without the hierarchy. The reported reduction in abrupt changes is also practically relevant. These results make the method promising, although their interpretation depends on resolving the configuration and training-control issues below.
Critical weaknesses
-
The reported experimental configurations contradict the central equal-budget comparison. Table 1 defines TB as the context token budget excluding the current chunk and reports 23.4k for KVMEM. However, Table 4 specifies 36 memory entries with 390 tokens each and nine recent frames with 1,560 tokens each. These sum to (36\times390+9\times1560=28{,}080) historical tokens, which is 20% greater than 23,400. The throughput discussion likewise describes nine compressed-memory frame-equivalents plus nine preceding frames, consistent with the larger budget. This discrepancy directly affects comparisons against methods reported at 23.4k. Additionally, Table 3(e) uses 64 latent queries as the default, whereas Table 4 specifies 390 queries per head. Since these queries determine the global summaries used by the adapter, the difference is relevant to both the architecture and its ablations. Until the actual configurations are reconciled, the reported quality gains cannot reliably substantiate the claimed equal-budget advantage.
-
The reconstruction objective does not establish attention fidelity in the context where compressed memory is actually used. Equation 5 compares attention over original versus compressed historical KV states in isolation, whereas Equation 2 jointly normalizes attention over compressed memory, recent context, and the current chunk. Matching the former does not ensure matching the latter. For example, compressing four identical memory tokens into one identical token leaves memory-only attention output unchanged and can yield zero reconstruction error. Yet, with one local token having the same logit, the total memory attention mass changes from (4/5) to (1/2). If the local and memory values differ, the actual attention output changes. This is an analytical limitation of the surrogate objective, not evidence that the trained model necessarily fails. DMD co-adaptation could compensate, but the paper does not isolate this effect or directly measure joint-attention fidelity. The concern becomes more consequential for repeatedly merged memories because the historical merge cascade is detached. The existing whole-adapter ablation does not establish whether gains come from reconstruction supervision, DMD co-adaptation, additional capacity, or the merge policy.
-
[1–3] are absent from the submitted bibliography and relevant to multi-scale memory, adaptive context compression, and hierarchical long-video memory. Echo-Infinity [4] is an especially important additional omission because it learns an evolving compressed history jointly with the video generator. Please distinguish the proposed local-plus-global KV adapter and explicit merge hierarchy from learned recurrent memory consolidation, and identify which claimed benefits are supported by controlled comparisons.
The method is promising, and the reported component gains deserve further investigation. However, the unresolved token-budget and adapter-configuration contradictions undermine the main comparison. In addition, the claimed attention fidelity is not established for the joint attention operation used during generation. These are central evidential concerns, rather than presentation issues alone.
References
[1] Zhaochong An, Menglin Jia, Haonan Qiu, et al. OneStory: Coherent Multi-Shot Video Generation with Adaptive Memory. CVPR, 2026; preprint first released in 2025.
[2] Hongjun Wang, Lin Liu, Jianguo Li, and Tao Lin. Dual-Granularity Memory for Efficient Video Generation. CVPR, 2026.
[3] Jingyu Lin, Xinyi Shang, Peng Sun, Cunjian Chen, and Zhiqiang Shen. DynaMem: Consistent Long Video Generation via Hierarchical Memory and Motion Priors. ICML, 2026.
[4] Yuxuan Bian, Zeyue Xue, Songchun Zhang, et al. Echo-Infinity: Learning Evolving Memory for Real-Time Infinite Video Generation. arXiv:2606.04527, 2026.
Questions for authors
-
Which exact token budget and latent-query count produced each main table? Please reconcile 23.4k versus 28.08k historical tokens and 64 versus 390 queries. If the reported configurations differ, please provide corrected comparisons at an identical actual attention budget.
-
Can you measure reconstruction error using actual future queries and the complete attention context, including recent and current tokens? Does the method preserve both the memory attention output and its normalization relative to local context, particularly after repeated temporal merges?
-
Were all memory-policy and pooling controls trained with matched generator initialization, data, teacher, optimization steps, and generator adapters? Please separate reconstruction-only, DMD-only, and combined supervision. Also specify the evaluation sample counts, random-seed protocol, numerical LPIPS threshold, and participant/trial counts for the user study.
These clarifications and controls are necessary to assess the reported efficiency and compression claims.