- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes GALP-KV, a training-free method for physical KV cache compression in vision-language models (VLMs). The authors argue that existing KV compression methods primarily treat visual tokens as one-dimensional sequences, potentially overlooking their spatial structure and degrading spatial grounding performance. GALP-KV operates after multimodal prefill, identifies visual KV positions, maps them to the model-specific spatial grid, and preserves all non-visual states. It combines attention-based importance scores with spatial priors and introduces a budget-gated selection strategy. Under moderate compression, it uses a shared layout-preserving retention set, while aggressive compression employs layer- and KV-head-specific selection. Experiments on Qwen2.5-VL and LLaVA-1.5-7B demonstrate approximately 49–56% physical KV memory reduction while largely preserving QA performance. Spatial coverage analysis and component ablations indicate that the gated per-head selection, rather than the geometric prior itself, drives the main improvements in grounding performance. However, the method does not consistently outperform the strongest baseline, NACL, on grounding and captioning metrics, and the current implementation does not achieve robust end-to-end acceleration.
Critical strengths
- The paper addresses the memory overhead of visual KV states in vision-language models through a training-free approach that physically removes selected cache entries after multimodal prefill. Experiments on Qwen2.5-VL and LLaVA-1.5-7B demonstrate substantial KV memory reduction (approximately 49–56%) while largely preserving QA performance. This provides practical evidence that visual KV caches contain considerable redundancy and can be compressed without retraining.
- The paper evaluates QA, captioning, and spatial grounding across different compression budgets, supported by component ablations and spatial coverage metrics. An interesting finding is that spatial grounding is more sensitive to visual KV compression than QA, and that budget-gated head-specific selection contributes more to grounding performance than the explicit geometric prior. These analyses provide useful insights into the trade-offs involved in multimodal KV cache compression.
Critical weaknesses
- Limited novelty and unclear contribution of the proposed spatial mechanisms. The method primarily combines existing attention-based KV selection with spatial anchors and budget-dependent selection policies. More importantly, the ablation results show that the geometric prior provides almost no improvement and can even slightly degrade grounding performance. The reported gains are mainly attributed to gated per-head selection, but this attribution is not entirely clear. For example, at a retention ratio of 0.435, the method should use the shared stable route rather than per-head selection, yet the corresponding ablation improvement is still attributed to the gated per-head mechanism. This raises questions about which components are actually responsible for the improvements and whether explicit spatial modeling provides a substantial contribution.
- Limited improvements over strong baselines and insufficient practical acceleration. Although GALP-KV achieves substantial physical KV cache reduction, it does not consistently outperform NACL, which achieves better referring mIoU and captioning performance at the main evaluated budgets. Furthermore, the implementation does not reduce peak GPU memory usage or demonstrate robust end-to-end inference acceleration. These limitations weaken the practical significance of the proposed method, particularly given its additional selection complexity and relatively modest improvements over existing approaches.
Questions for authors
- The ablation results suggest that the geometric prior provides little or no improvement, while the authors attribute the main grounding gains to budget-gated per-head selection. However, at the retention ratio of 0.435, the method uses the shared stable route rather than the per-head proxy route, yet the improvement is still attributed to gated per-head selection. Could the authors clarify which specific components account for this improvement and provide a more direct comparison isolating the contribution of the gating mechanism?
- NACL achieves better referring mIoU and captioning performance at the main evaluated compression budgets, while GALP-KV does not demonstrate a reduction in peak GPU memory or substantial end-to-end inference acceleration. Could the authors clarify the main practical advantages of GALP-KV over NACL and provide evidence that its additional selection complexity offers meaningful benefits in realistic inference settings?
Response to Reviewer Co1F: Gate-Isolation Evidence, Causal Attribution, and Practical Advantages over NACL
We deeply appreciate your careful reading and for pinpointing the mechanistic attribution in Table 12 (formerly Table 8). We have corrected the attribution and added direct gate-isolation experiments in the revised PDF:
(1) Streamlining the Geometry Prior: Following dual-budget sensitivity sweeps confirming negligible non-monotonic fluctuations (≤0.0010 mIoU; Tables 7 & 8), we removed the keyword-based geometric prior (ϕi weight =0) in our revised default policy, GALP-KV-Lite (parser-free default policy keeping all remaining spatial components unchanged, while GALP-KV-Full retains ϕi as an optional structural regularizer; Abstract, Sections 3.4, 4.4, 5, and Appendix B.3), keeping all reported metrics within 0.0006 mIoU (Table 12).
(2) Novelty of GALP-KV: This work is not a simple recombination of 1D heuristics: under aggressive compression (r=0.05), standard 1D attention selectors collapse to 0.1809 mIoU, whereas our model-aligned 2D grid mapping, layout anchors, and budget-gated routing preserve 0.3655 mIoU (+0.1846 gain; Appendix B.2). Formulating position-consistent post-prefill physical KV compression over model-aligned 2D grids and uncovering the budget-dependent trade-off between shared layout coverage/reusability and per-head spatial specialization constitute our core contributions.
(3) Clarifying Table 12 attribution & isolating the budget-gated mechanism (Question 1): At r=0.435>τ=0.37, GALP-KV activates the shared
Stableroute rather than per-head selection. We relabeled the final rows in Table 12 asFull GALP-KV (stable)andFull GALP-KV (proxy)and rewrote Section 4.4 and Appendix B.6:Stablepolicy—model-aligned 2D grid neighborhood pooling and guarded task-aware spatial replacement (+0.0047 over pure attention 0.3838), with per-head selection inactive.Proxybranch, gaining +0.0241 (0.3693→0.3934) over+Geometryand +0.0100 (0.3834→0.3934; Appendix B.2) over sharedStablerouting.Directly isolating the gate across both budget endpoints (Section 4.4 & Appendix B.2, Table 6):
forced-proxy,stable, andgatedon identical diagnostic splits confirmsgatedis bit-identical tostable:forced-proxy(per-head)stable(shared)gated(reported)Forcing per-head routing at r=0.435 brings ≤0.0052 single-round gain while doubling the maximum blind radius (0.0942→0.2006) and disrupting prefix reuse (transfer drops from 0.886 to 0.523).
gatedmatchesproxy(0.3655 mIoU, 171 regressions), whereas forcingstablecollapses to 0.0839 (762 regressions)—the gate rescues +0.2816 mIoU. Shared attention-top-k without layout-aware routing reaches only 0.1809, confirming a +0.1846 gain from layout-aware per-head routing. Thus, the gate routes optimally at both extremes.(4) Practical advantages over NACL & runtime efficiency (Question 2):
output_attentions=Truewith lightweight w=32 window slicing (Q[:,−w:,:]KT) and immediate buffer release eliminates compression memory spikes—cutting decode-stage peak VRAM by 18.4% at batch size 8 (14.2 GB→11.6 GB) and selection latency to 34 ms (Section 4.5 & Appendix G.3).