- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper presents ResKV-Tree, a hierarchical KV cache representation for training-free streaming video understanding. The paper focus on the query-agnostic streaming video understanding setting, where future questions are unknown when video frames arrive. ResKV-Tree addresses limitations in existing methods by recursively organizing historical KV pairs into a hierarchical "residual tree" structure.
The the number of roots reaches a predefined budget, two roots are merged. The KV-NMS module removes redundant KV pairs by combining a saliency score based on a proxy query and a value space similarity score. Among the remaining KV pairs, the salient ones are promoted to a new parent, and others remain in the child nodes. At test time, the pipeline does a best-first search over the tree using the true user query.
The experiments involve three LLM backbones, three offline video benchmarks, and two streaming video benchmarks, and demonstrate that ResKV-Tree consistently improves over the baselines.
Critical strengths
- The proposed residual hierarchy is well-motivated. It alleviates the tension between compression and preserving fine-grained evidence with an intuitive design. The tree search step allows retrieval at different levels of granularity.
- The empirical results are strong and reasonably comprehensive. ResKV-Tree consistently improves performance across three backbones and both streaming and offline benchmarks. The improvement on OVO-Bench are particularly large.
Critical weaknesses
- There is not sufficient evidence for the claimed efficiency advantages. Efficiency is an important motivation for the proposed method. The paper mainly reports end-to-end TTFT and peak GPU memory after 1024 frames. Isolating the latency for tree construction, maintenance, and search costs and reporting them separately can be helpful.
- Hyperparameter selection process is unclear. The proposed pipeline introduced many hyperparameters and it is unclear how their values are selected; in particular, whether they were selected on a held-out validation set and then kept fixed across all reported benchmarks and backbones.
Questions for authors
- I found the irreversible KV pruning design in KV-NMS a bit unintuitive. Have you compared the current KV-NMS design against a variant that does not permanently discard redundant KV pairs, but instead retains or demotes them to deeper residual nodes?
- How do the computational and memory costs scale with video length?