- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper discusses an interesting finding: the history cache in autoregressive video generation can considerably reduce the impact of new text input controls.
The authors define minimal sufficient visual memory (MSVM) as the smallest amount of history needed to preserve the future video generation distribution. They also introduce D-IBVM, a framework that uses memory ablation and causal interventions to identify important memory units.
However, the authors find that new text prompts have very little impact when the history cache is present. Therefore, several planned experiments cannot be conducted. Instead, the paper focuses on analyzing the history cache itself. The experiments show that some old frames, especially the first cached frame, can have a large impact on generation, even when their actual visual content is not important.
Critical strengths
The paper studies an interesting and important problem in autoregressive video generation: the relationship between history memory and new input controls. The finding that cached history can significantly reduce the effect of new prompts is interesting and may be useful for future research.
The authors perform several controlled experiments, including memory ablation and frame replacement, to better understand the role of different cached frames.
Critical weaknesses
Personally, the paper is hard to follow and read generally. The abstract is almost a full page, I was overwhelmed with many numbers before getting to the main body. In terms of writing style, the paper has an “We tried to do this, but it did not work; instead, we did that” structure.
I also suspect there are AI-generated contents without verification, e.g. did the author really check Fig 2?
For the math/theory part, it is very unclear and non-rigorous. E.g. In Def 1, what is the clear definition of ? The previous one is the full decision space? For Proposition 1: Is the deficit really convex? Not concave?
Paragraphs 3 and 4 (MSVM definitions, DIVM, and Algorithm 1) contain formalizations that did not produce any results, and it is in section 5 that the reader finally learns about what actually was tested.
The work is inconsistent with its own framework. The title, the abstract and the Sections 3-4 discuss MSVM, DIVM and the "clamped decision" criterion. Afterwards, the work states that this mode is "currently unrunnable" for both backbones. Proposition 1, Algorithm 1 and experiments E2-E4 yield no result. About half of the work is devoted to defining things. The main contribution is measuring attenuation and finding attention sinks.
Technically, the paper does not clearly distinguish visual consistency from controllability. More clarification that needs to be made is the meaning of prompt-switch attenuation. A less significant change made in the produced continuation does not have to mean that it has less controllability because sometimes it may be good to keep the chosen scene.
Questions for authors
Please refer to the weakness section.