- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes ReCollage, a self-supervised history memory compression method for long video generation, aiming to address the difficulty of existing methods in allocating limited memory budgets according to content importance. The authors use learnable queries to select important information from historical VAE tokens and train the token selector using the reconstruction loss of a video diffusion model through a differentiable Collage-and-Scatter mechanism.
Critical strengths
-
History memory compression in long video generation is an important problem, with clear motivation and practical value.
-
The Collage-and-Scatter design is quite interesting. It learns content-adaptive token selection through self-supervised reconstruction while preserving the original VAE tokens and their positional information.
-
The experimental performance is good, showing significant improvements in historical content recovery and long-term consistency.
Critical weaknesses
-
The computational overhead of the method appears relatively high. The context encoder contains multiple Transformer blocks and needs to process historical tokens and learnable queries, but the paper lacks quantitative statistics on additional inference latency, memory consumption, and computational complexity.
-
The baseline settings in some experiments are problematic, and comparisons with more state-of-the-art baseline methods are needed. For example, Table 2 only directly compares against Uniform and Random, lacking comparisons with other advanced memory selection or retrieval methods.
-
The experimental scenarios are relatively limited and are mostly synthetic. Some tests simulate long-term memory tasks by repeating static frames, artificially constructing distractor segments, or synthesizing camera trajectories, lacking validation in more complex and natural long video generation scenarios.
Questions for authors
-
ReCollage mainly learns token selection through historical reconstruction, but information important for reconstruction is not necessarily what future generation requires, correct?
-
For longer video histories, how do the computational cost and memory consumption of the context encoder change? Could the authors provide corresponding efficiency comparisons?
-
Could the authors include more complex real-world long video scenarios and comparisons with other advanced memory management methods to further validate the generalization ability of the proposed method?