- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies query-agnostic compression of visual KV prefixes for reuse across questions. It compares quantization, pruning, retention of deterministically spaced evicted tokens, spatially merged tokens, and moment summaries under approximately matched memory budgets. Experiments cover LLaVA-1.5-7B and Qwen2.5-VL-7B on four VQA benchmarks. With bf16 retained cores, sampled tokens have higher mean scores than merged tokens in all eight model–task cells and non-negative margins over moment summaries. The paper also derives an attention-mass bound and discusses outlier dilution, dispersion instability, and conditional RoPE cancellation as possible explanations. Additional experiments show that quantization results depend strongly on grouping and metadata accounting.
Critical strengths
-
Relevant problem formulation. Compressing an image prefix before future questions are available is a meaningful serving constraint. Separating token selection from the representation of evicted information is useful, and the paper appropriately identifies original-token retention as a baseline that more elaborate representations should justify outperforming.
-
Useful negative results and disclosures. The study reports failed summary configurations, paired uncertainty estimates, the scope restriction to bf16 retained cores, and the strong effect of quantization grouping. These disclosures provide useful evidence within the tested setting and avoid presenting every experiment as a success.
Critical weaknesses
-
The central comparison does not yet establish a general disadvantage of merging or summaries. The merged-token baseline directly averages post-RoPE K/V within fixed spatial tiles, without counts or dispersion correction (Section 3). This can change both positional structure and softmax mass. Even for identical keys, replacing (n) entries with one changes their unnormalized mass from (n\exp(x)) to (\exp(x)) unless multiplicity is compensated. The paper discusses RoPE cancellation, but discussion of a failure mechanism is not a demonstration that a position-aware, count-corrected merging baseline still loses to sampling. Moment summaries address some mass effects, but do not resolve the representativeness of the plain merging comparison. The conditional RoPE lemma and cross-model performance differences also do not isolate RoPE as the cause of the observed collapse. Internal inconsistencies further weaken confidence in the comparisons. In Table 21, Qwen–TextVQA is 0.880 for “K row / V groups” and 0.000 for “K/channel / V row,” whereas the text attributes the rescue to K grouping. A labeling error could explain this, but the configuration-to-result mapping needs correction. Likewise, the 6.2% metadata claim for K per-channel / V per-token-row appears inconsistent with Appendix C.1’s 32-token grouping formula: for (n=576) and (d=128), it gives 15.625%. Finally, the 17-token sampling attachment uses approximately 5.5% more bytes than the summary attachment. The whole-cache gap is small, but this favors sampling, contrary to Appendix C.3’s stated bias direction.
-
The contribution rests on empirical conclusions whose strength and scope remain limited. Only one of eight sampling-versus-moment-summary comparisons has a confidence interval excluding zero; the quantized-core robustness criterion is not met. The study does include text-rich tasks such as TextVQA, ChartQA, and DocVQA, but remains limited to two model families, single-image short-answer VQA, greedy decoding, and fixed source-order subsets of 150 examples, with some comparisons using 48. It does not adequately separate task requirements such as localized retrieval versus distributed aggregation, or test answer quality across multiple real questions sharing an image. The theory does not close this gap. Proposition 1 bounds single-query attention-output differences, not task-level gains. Proposition 2 concerns value-only permutation-invariant moments and does not establish impossibility for joint K–V summaries. The assertion that sampled retention lies outside Equation (2)’s estimator class also needs justification, since its retained numerator and mass can be written in that form. Moreover, the headline 6.5–8.3× latency reduction includes prefix reuse; compression’s incremental speedup over an already cached full prefix is approximately 1.24–1.38×. Compression construction cost and finite-reuse amortization are needed to assess the practical contribution.
Questions for authors
-
Configuration and accounting clarification. Can you reconcile Tables 20–22 with the claim that K grouping drives the rescue? Please identify the exact grouping of each column and whether the issue is limited to labels or affects the reported contrasts. Can you also reconcile the metadata fractions with Appendix C.1, including the claimed 6.2% overhead for K per-channel / V per-token-row?
-
Merging controls. Have you evaluated count/logit-mass compensation and a principled position-aware merging strategy under the same effective byte budget? If so, please provide the results. If not, what evidence distinguishes an inherent limitation of merging from limitations of the fixed-tile post-RoPE averaging baseline? Targeted ablations of multiplicity correction and positional handling would materially affect my assessment.
-
Statistical support and scope. What is the strongest conclusion supported by the paired intervals, particularly given that only one bf16 sampling-versus-moment-summary interval excludes zero and the quantized-core criterion is not met? Do available results show whether the ordering depends on localized detail retrieval versus distributed aggregation, or on answering multiple real questions from one compressed image prefix? I acknowledge the existing OCR/document benchmarks; my concern is generalization across information structure and task requirements.
-
Theory and deployment interpretation. Why does sampled retention fall outside Equation (2)’s class when its extra tokens supply a numerator and denominator contribution in exactly that form? Which parts of the outlier argument apply to the statistics actually stored by your implementation, including joint K–V information? Please also clarify the one-time compression cost and the number of prefix reuses required for an end-to-end benefit.
If the rebuttal convincingly clarifies the inconsistencies and provides credible evidence supporting the central empirical contribution, I would reconsider my assessment and consider raising my score.