- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper proposes StreamCacheVGGT, a training-free cache-management framework for streaming visual geometry transformers. Cross-Layer Consistency-Enhanced Scoring refines FFN-residual importance scores using rank stability across layers. Uncertainty-Gated Deferred Compaction then combines immediate retention, merging, and eviction with a short verification pool inside a fixed cache budget. The paper reports reconstruction, pose, depth, and long-sequence results, together with throughput and memory measurements.
Critical strengths
- The motivation is practical. Maintaining useful geometric context under a bounded streaming KV cache is important for long-sequence reconstruction.
- The method combines inexpensive importance proxies with a concrete attempt to avoid irreversible decisions on uncertain tokens, while remaining compatible with FlashAttention.
- The evaluation covers several indoor, outdoor, depth, and long-sequence benchmarks. Providing controlled same-backbone results and acknowledging the accuracy-throughput trade-off and dynamic-scene limitations are useful.
Critical weaknesses
- My biggest concern is that the contribution of deferred verification is not isolated. Table 7 compares eviction, CLCES, and CLCES+UGDC, but UGDC introduces both immediate token merging and the deferred branch. Therefore, its improvements cannot be attributed specifically to uncertainty-gated verification. A CLCES+immediate-merge variant without deferral, followed by a matched-budget deferred variant, is necessary. Random or recency-based deferral with the same reserved capacity would also test whether cross-layer uncertainty is informative.
- The figures in the paper are not informative. The figures are mostly just boxes with numbers, making them hard to read and get meaningful information. They should be at least polished for publication.
- The deferred branch is underspecified despite being central to the method. Appendix C gives H=2 and a recurrence threshold, but does not define the descriptor, deferred capacity, pool-overflow policy, motion-residual threshold, or dynamic-token TTL. It is also unclear whether deferred tokens participate in attention and how their layer-specific keys and values are preserved or reconstructed when a descriptor is promoted. These details determine both the online algorithm and whether its claimed memory accounting is valid.
- Several result tables and their explanations need to be reconciled. Table 7 shows that CLCES alone worsens 7-Scenes Accuracy from 0.0259 to 0.0306, whereas Table 11 reports an improvement from approximately 0.026 to 0.021 at the stated default lambda and window. Appendix E says the complete method is best on continuous KITTI, but Table 10 gives lower errors for CLCES alone at both 300 and 500 frames. Table 8 also shows substantial long-sequence completeness regressions, for example 0.479 versus 0.057 for InfiniteVGGT on Classroom. These results may use different protocols, but the distinctions and the resulting limits on the claims need to be explicit.
- The assumptions behind scoring and merging need stronger empirical support. Rank dispersion is a heuristic uncertainty measure, rather than a demonstrated predictor of harmful eviction. Similarly, Appendix B's minimum-variance interpretation assumes that merged keys observe one underlying feature and that FFN scores are inverse noise variances, neither of which is established. Cosine-nearest key fusion also does not generally preserve softmax attention or token multiplicity. A comparison with simple merging, measurements of attention or geometry error, and stress tests under motion and viewpoint changes would clarify when these approximations are safe.
Questions for authors
Please see above weaknesses.