- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies KV cache management for camera-controlled video world models. It uses a small learned scorer to retain historical blocks, and then compresses the history through salient tokens and pooled summaries. The scorer is trained offline with future attention and poses, while only observed information is used online and the generator weights remain frozen. The authors evaluate revisit consistency, speed, and GPU memory on LingBot-World-Fast and Matrix-Game 2.0 using camera loop trajectories.
Critical strengths
On LingBot, same-place increases from 1.77 to 2.29, and throughput improves by about 37%. On Matrix, consistency, speed, and GPU memory are also improved together. The same-place gains over stronger baselines such as WorldKV and Relax Forcing are also supported by per-clip statistical analysis.
The method only trains a small scorer, without fine-tuning the generator or relying on an external history bank. Packed summaries and gists are both counted in the cache budget, and the inference cost also includes the scoring and compression overhead.
Critical weaknesses
The No packing variant keeps the full spatial tokens. Appendix B also says that the three variants use the same archive members and gist rules, changing only the L1 representation, but their average KV counts are all 15.1k. The paper does not explain clearly how different representations give the same token count, so the compression gains in this comparison are still not easy to interpret.
The revisit-pair selection is also not fully clear. The initial context is always retained, but the paper does not say how many evaluated returns concern other previously visited views. This makes the evidence for retaining non-initial history harder to judge.
Questions for authors
In Table 2(b), the compression methods are different, but the average KV counts are all 15.1k. How is this number counted?
How are the eight revisit pairs per clip selected? Do they also include views first visited after the protected initial frames?
Appendix E gives r and k_min for LingBot. What values are used for Matrix-Game 2.0 in the archive-capacity rule?
3 comments
Thank you again for your feedback. We would appreciate it if you could let us know whether our response has addressed your concerns. If there are any remaining questions or concerns, we would be happy to discuss them further and provide additional clarification.
Thanks for the clarifications and additional experiments. The revised ablation resolves my question about KV counts, and the new results show that the method can help on revisits to non-initial views. I will keep my rating.
Response to Reviewer ubPC
We sincerely thank you for your careful reading of our paper and your constructive comments. We have conducted additional experiments and checked the relevant experimental settings and evaluation details. We hope that the results and clarifications below address your questions.
W1 / Q1: KV counts in Table 2(b)
Thank you for pointing out the issue with the KV counts. KV denotes the mean number of tokens actually resident in the cache during evaluation. It includes protected initial and recent context, selected original tokens, pooled summaries, and coarse gists. Full-resolution chunks contribute all spatial tokens; packed chunks contribute selected tokens plus summaries; gist-only chunks contribute their gist tokens.
The revised Table 2(b) reports the 60-clip L1-only controls below, also presented in Table 6, Appendix F.3. Retention settings are fixed and L2 gists are disabled; the packed variants retain L1 summaries. KV counts are measured from the actual resident cache.
Both packed variants use fewer resident KV tokens than the full-token baseline. Table 2(b) separately lists the 100-clip full-method reference with L1 packing and L2 gists: SP 2.29 and mean resident KV 15.1k tokens.
W2 / Q2: Revisit-pair selection and non-initial history
For each clip, we first identify all candidate pairs with a pose distance of at most 0.15 and a separation of at least 30 decoded RGB frames. We uniformly select eight pairs from the candidates ordered along the timeline, without filtering based on the protected initial context, and use the same frame pairs for all methods. Across the 100-clip LingBot evaluation, 741 of the 800 selected pairs (92.625%) have their first-visit frame after the initial chunk, which covers RGB frames 0–8. Separately, 592 of the 800 pairs (74%) have a revisit pose whose distance from the initial pose exceeds 0.15.
The sampling procedure and these statistics are provided in Appendix A. We additionally evaluate two trajectory types targeting non-initial history (Table 5(a,b), Appendix F.1). Double lookback contains two departure-and-return cycles to intermediate viewpoints first observed after the initial conditioning frames and uses the original frozen predictor. Long departure-and-return uses a trajectory-specific predictor checkpoint, with attention settings of 40k tokens for Window and 47k for WorldRetain. SP is evaluated with Qwen3-VL-32B.
WorldRetain achieves higher SP with fewer resident tokens and faster generation in both configurations. Figure 6 shows the intermediate reference viewpoint and its later revisit.
Q3: Matrix-Game 2.0 Configuration
Both backbones initially select half of the candidate history, rounded up, with a preliminary minimum of two historical chunks when available. Matrix-Game 2.0 protects two initial frames and the most recent chunk. The selected history then undergoes spatial compression and a final check against the 47,000-token ceiling used by both backbones. KV is measured from the resulting resident cache. These settings are specified in Appendix E.