- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes LoomKV, an online low-rank KV cache compression method for Video LLMs. Unlike previous methods such as xKV, which require the full KV cache before compression, LoomKV constructs the compressed KV cache incrementally during prefill. The method combines cross-layer sharing with incremental SVD and Gram Matrix Refresh to update the low-rank representation without storing the full historical KV cache. Experiments show that LoomKV significantly reduces memory usage and supports longer video inputs while maintaining competitive accuracy under 8x compression. The paper also provides ablation studies on cross-layer sharing and refresh frequency. However, the method introduces additional prefill and decoding latency, and its algorithmic novelty is relatively limited compared with existing low-rank compression methods.
Critical strengths
-
The method uses incremental SVD and Gram Matrix Refresh, with solid mathematical support. Simple and reasonable design.
-
Good memory reduction without storing full KV cache. Also maintains good accuracy under high compression ratio.
Critical weaknesses
-
Limited novelty. The method mainly combines cross-layer low-rank compression from xKV with existing incremental SVD techniques. The main algorithmic contribution is relatively limited.
-
Efficiency concerns. The method introduces higher prefill and decoding latency (1.51x prefill time and around 30% decoding overhead). The paper also lacks real serving throughput experiments, so it is unclear whether the memory savings can bring practical benefits.
Questions for authors
No
Response to Reviewer LUdH
Thank you for the review.
W1: limited novelty (xKV plus incremental SVD). The algebra is classical, and we say so (Sec. 2, Lemma 1); the contribution is not a new factorization. xKV horizontally concatenates every layer's completed cache and factorizes it. In the regime that motivates compression, that completed cache is exactly what cannot be held. Our contribution is removing that construction dependency: we build the cross-layer low-rank cache incrementally during chunked prefill, keeping a compact sufficient statistic (A, Q, G) and a periodic refresh that Lemma 1 makes exact for the represented history.
The empirical claim is specific to video caches. The frame×space×channel grid one would expect a video cache to reward does not pay for itself: three-mode Tucker scores 47.97 vs 59.12 for cross-layer, −11.15 [−15.64, −6.73], p<1e-3. Cross-layer redundancy is the structure worth keeping. Offline xKV is the ceiling we measure against, not a baseline. On one card, our gap to it on Qwen2-VL is −2.71 [−6.09, +0.66]. Under the in-process 24 GB allocator cap, offline xKV cannot form its own input at 128 frames, while LoomKV runs to 224.
W2: efficiency (1.51× prefill, ~30% slower decode) and no serving throughput. Both costs are stated in the paper.