- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes a method to select which frames should be stored in the memory bank for autoregressive video generation. After the base method retrieves relevant candidate observations, the proposed method update the memory bank by filtering out the frames which are more redundant given the current memory bank and new candidate observations. The redundancy is decided based on the view coverage, which is approximated by the closeness in viewing position and direction and visual resemblance. The results show that it can improve long-horizon generation quality by only maintaining a small memory bank.
Critical strengths
- The idea is well-motivated, and the paper is clearly written.
- The results show that the proposed method can improve the baselines in terms of the long horizon generation quality and memory footprint.
Critical weaknesses
I don't have major concerns over the paper, but I want to know how efficient the proposed method is. Since this method requires an extra step after the memory bank is retrieved, then how much overhead does it create? Is it more straightforward to move this view-diversity design to the retrieval stage?
Questions for authors
- Can the authors compare the proposed method with [1]? Since [1] is focus on retrieving the missing information, does it mean if the retrieval process itself is coverage-aware, a separate selection process after retrieval is not necessary?
- Can the authors report the computation overhead like extra time over the baselines?
[1] Joo, Minseok, et al. "Retrieve What's Missing: Coverage-Maximizing Retrieval for Consistent Long Video Generation."
4 comments
Dear Reviewers,
This is a friendly reminder that the rebuttal discussion period is approaching its deadline.
Please take some time to review the authors' rebuttal and actively participate in the discussion. If you have any remaining questions or concerns, please raise them and share your feedback with the authors and other reviewers.
As the discussion period is about to close, we kindly ask you to complete your discussion and provide your updated assessment before the deadline.
Thank you for your time and effort.
Best regards,
Area Chair
Reply to the rebuttal
Thank the authors for providing the rebuttal. I will keep my score as Weak Accept since I don't have other major concerns. While I tend to believe that the overall scope and technical contribution remain moderate, I think the paper has its merits as a lightweight pipeline / component.
Thank you for the discussion
Dear Reviewer,
Thank you for reading our rebuttal and for your continued support. We appreciate your questions on overhead and coverage-aware retrieval; they helped us clarify how KEEPSAKE fits into a full memory pipeline, and we will include both discussions in the revised paper. We are glad the lightweight, plug-in nature of the method came through, as it is central to how we hope it will be used.
Thank you again for your time and constructive feedback.
Overhead And Comparison With CovRAG
Thank you for the positive assessment, and for the kind words about the motivation and clarity of the paper. Both of your questions get at how KEEPSAKE fits into a full memory pipeline, and answering them helped us sharpen that part of the paper.
Q1. Comparison with coverage-aware retrieval (CovRAG)
Thank you for pointing us to CovRAG; it is closely related, and whether coverage-aware retrieval makes retention unnecessary is exactly the right question to ask. One clarification first: KEEPSAKE does not run after retrieval. It decides which frames stay in the memory bank, once per memory update; the host's retrieval rule then reads from that bank unchanged.
This is why the two are complementary rather than alternatives. CovRAG decides what to read for each new view, but the archive it reads from still keeps every generated frame: storage keeps growing, every lookup searches the growing archive, and drifted frames remain retrieval candidates. KEEPSAKE decides what to keep, which addresses all three, and a coverage-aware retriever can run directly on its bounded bank with only 32 candidates to score. Its code is not yet public, so we could not run the combination, but we will discuss it in the revised related work.
Q2. Computation overhead
KEEPSAKE's own cost is small. It runs once per memory update, building the substitution graph over the bank and new frames, scoring them, and releasing evicted entries; on every host it takes under 1% of generation time. Its net effect is a saving, because every later lookup searches 32 frames instead of the full, growing history:
The size of the end-to-end saving depends on how much of each host's runtime scales with memory. On WorldMem, generation dominates and retrieval was already cheap, so the gain is small. On VMem, 3D reconstruction scales with memory size, so a one-minute video drops from over 8 hours to under 2; stored frame data also falls by 96% and peak GPU allocation by about 70%.
MemCam and WorldMem timings use the same setup as the paper (Appendix A.2): one NVIDIA H100 PCIe GPU, 16 CPUs, and 64 GiB of host memory per generation job. VMem timings are recorded on one NVIDIA RTX PRO 6000 Blackwell GPU.