acceptodds
Under review as a conference paper at ICLR 2027

MEKV: Controlling Concentration in Query-Agnostic Visual KV Compression

Abstract

Large vision-language models require compressing visual KV caches for efficient multi-query serving. When compression happens before user queries are known, a natural approach is to retain tokens according to a relevance signal—yet we show that high-quality signals can degrade accuracy when over-concentrated, starving unseen questions of needed evidence. We develop a concentration theory along the maximum-entropy temperature path, proving that a surrogate accuracy objective is single-peaked with the optimum given by a moment-matching condition. This yields MEKV, a simple, training-free method that separates signal quality from concentration control via a single temperature parameter. The theory further predicts that the right strategy is model-specific: models whose optimum requires strong tempering need concentration control first, while those near the raw signal benefit more from fine-grained token selection. On full validation across three benchmarks, MEKV improves over coverage baselines by up to 17 pp, with the temperature transferring across tasks without re-tuning. The prediction is borne out by a strategy reversal: same-signal per-head top- falls far behind MEKV on Qwen yet excels on InternVL, and placing per-head selection inside calibrated quotas repairs the failure while preserving the gains.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Vision-Language Models, KV Cache Compression, Efficient Inference, Visual Token Compression, Multimodal Learning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_Q9A7OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies query-agnostic visual KV-cache compression, where an image cache must be compressed before downstream questions are available. Synthetic Question Attention (SQA) supplies spatial relevance scores, and MEKV interpolates between spatial coverage and relevance-based allocation using a temperature-controlled exponential family. Tokens are sampled uniformly within the resulting within the resulting regional quotas. The paper establishes concavity of an oracle-attention KL surrogate along this family and uses concentration diagnostics to interpret differences between Qwen and InternVL. Experiments report improvements over coverage baselines on document, chart, and scene-text VQA, together with a calibrated per-head selection extension.

Critical strengths

1. The empirical observation about excessive concentration is useful. Informative proxy-attention scores can produce worse retention decisions than spatial coverage when used too aggressively. The Qwen results provide a concrete example of this failure, motivating calibration as a relevant consideration for reusable visual caches.

2. Some improvements are supported by substantial evaluation. The fixed-temperature InternVL experiments cover full validation sets across three benchmarks and report paired bootstrap intervals. The calibrated per-head extension also includes a DocVQA evaluation explicitly disjoint from development data. These results provide evidence that balancing relevance and coverage coverage can improve particular compression policies.

Critical weaknesses

1. The theoretical results do not establish the central claims about task accuracy or architecture dependent strategy selection. Theorem 1 establishes strict concavity of A(λ)=−KL(a∣pλ)A(\lambda)=-\mathrm{KL}(a|p_\lambda), which follows directly from the exponential-family construction. However, aa is an attention distribution obtained from real questions, and the paper provides no bound or consistency argument connecting this surrogate to ANLS or VQA accuracy after discrete, capacity-constrained token retention. Matching an attention moment therefore does not establish that the resulting cache preserves the evidence required to answer unseen questions. The extension to the "architecture reversal" is particularly under-supported. Proposition 1 decomposes allocation concentration, not prediction error. Its nonnegative off-path term does not determine whether a policy's additional structure improves accuracy. Likewise, a local surrogate expansion along the temperature family does not establish an accuracy ordering between MEKV and per-head top-kk, which generally lies outside that family. The quantitative diagnostic is also inaccurate in important reported settings. At 20% retention, the moment-matching estimates are approximately 0.613 versus a measured grid optimum of 0.25 for Qwen, and 0.996 versus 0.42 for InternVL, as shown in Appendix C. These are discrepancies of roughly a factor of 2.4. Corollary 2 additionally assumes that tightening the budget increases the oracle moment, but the operational oracle in Appendix C.1 is defined from real-question attention without specifying how it becomes budget-dependent. Figures 4 - 6 consequently provide descriptive observations rather than strong validation of the proposed theoretical explanation. In particular, failing to detect a statistically significant secondary peak does not establish unimodality. This gap matters because the explanatory framework is presented as a principal contribution beyond temperature tuning.

2. The experiments do not sufficiently isolate the proposed contribution or establish its advantage under a consistent eavluation protocol. The headline gains of up to 17 percentage points are against coverage baselines. The stronger same-signal comparison gives a substantially different picture: on InternVL DocVQA, MEKV scores 75.56/56.40, while SnapKV-style selection scores 77.89/64.28 at 20%/10% retention. SnapKV-style selection also outperforms MEKV on InternVL textVQA, shown in Table 13. Against optimized linear signal-coverage mixing, the reported temperature advantage is only 1.13/2.03 points on Qwen and is statistically unresolved on InternVL, shown in Appendix H. Those comparisons use the best settings from sweeps and do not establish held-out superiority of the exponential parameterization. Evaluation coverage and splits are fragmented. Table 13 explicitly uses development evaluation for Qwen DocVQA but full validation for InternVL and the TextVQA comparisons. Appendix B does not establish that the full DocVQA validation results exclude temperature-selection images. Appendix I provides a disjoint evaluation for the hybrid extension, but this does not resolve the corresponding question for the base method. Furthermore, there is no consolidated comparison containing uncompressed performance and all stated baselines across the principal model–benchmark settings. Without uncompressed scores, the accuracy cost associated with the reported cache savings remains difficult to assess. The hybrid extension also changes two factors simultaneously: it introduces regional quotas and replaces global deterministic top-kk with tempered sampling within regions. Consequently, Table 2 does not isolate how much improvement comes from regional allocation, sampling, or their interaction. This is consequential because the experiment is used to support the paper’s causal interpretation of concentration control. Finally, the third-architecture evidence has a major scope limitation. In Gemma’s primary “full8” arm, the layers being compressed account for only 4.76% of total visual KV capacity, while the remaining layers retain their visual KV (Appendix B.4). The nominal 20%/10% retention settings therefore correspond to approximately 3.8%/4.3% reductions in total visual KV capacity under that accounting. This is a much narrower compression setting than the headline retention percentages suggest.

Questions for authors

1. Evaluation clarification. Where are the DocVQA development examples drawn from, does "DocVQA-300" count images or questions, and which reported evaluations exclude those images? Please identify the splits and score aggregation conventions for the base method, baselines, and hybrid extension.

2. Evidence needed to assess the empirical contribution. Can you provide a common, image-disjoint comparison of base MEKV, calibrated per-head selection, tuned linear mixing, the stated compression baselines, and the uncompressed model? Please specify the baseline adaption parameters. For example, the filtering layer is an explicit parameter of the original FastV method, but its value is not specified in Appendix B.3.

3. Clarification of the theoretical claims. What justifies using oracle-attention FL as a surrogate for downstream accuracy? How is the budget-dependent oracle a(r)a(r) in Corollary 2 constructed, and which result supports predicting the relative accuracy of an off-path top-kk policy?

4. Evidence needed for the causal interpretation. What happens when regional quotas and the within-region selection rule are varied independently? In particular, do calibrated quotas still repair Qwen when selection within each region remains deterministic top-kk, and how much improvement is obtained from tempered sampling without regional quotas?

3 comments

Thank you for the thorough review; it drove the main structural, experimental, and scoping revisions in the updated PDF.

Q1. Splits and aggregation (§5, Appendix B.3). DocVQA validation (1,286 images) is split into three disjoint subsets: DocVQA-300 (300 images / 1,799 questions), used solely to select fixed parameters; 200 images from earlier method checks, excluded from both selection and testing; and strictly held-out DocVQA-786. ChartQA and TextVQA use full validation sets, fully held out. All main Qwen results are now on held-out splits (Table 2). Excluding DocVQA-300 from InternVL full validation (DocVQA-986) leaves every conclusion unchanged: Base MEKV − Stratified is +5.2 [4.1, 6.3] / +7.6 [6.3, 8.9] pp at 20% / 10%; SnapKV-style − Base MEKV is +2.6 [1.3, 3.9] / +8.6 [7.1, 10.2] pp. All scores are image-mean ×100 (questions averaged within each image, then across images).

Q2. Consolidated comparison and baseline parameters (Table 2; Appendices B.3, G, H). Image-mean ×100, 20% / 10%:

Model & SplitUncomp.StratifiedBase MEKVSnapKV / Top-kkCalib. Per-Head
Qwen DocVQA-78695.4683.97 / 61.3288.21 / 76.0077.66† / 64.71†89.38 / 78.58
Qwen ChartQA86.2655.16 / 37.0371.04 / 54.9561.37† / 44.02†70.28 / 57.85
Qwen TextVQA—66.25 / 50.9877.08 / 68.0075.37 / 66.82—
InternVL DocVQA (full / 786‡)91.8270.40 / 48.5275.56 / 56.4077.89 (78.24†‡) / 64.28 (65.26†‡)79.93‡ / 66.46‡
InternVL ChartQA85.5873.41 / 57.1979.13 / 69.1779.92 / 71.1579.84† / 73.52†
InternVL TextVQA—66.69 / 53.1074.15 / 68.9475.62 / 72.84—

† pooled 3-seed per-head harness matched to Calib. Per-Head; ‡ held-out DocVQA-786.

  • The real-query oracle nearly matches uncompressed serving even at 10% budget (InternVL 91.00 vs. 91.82 DocVQA; 84.28 vs. 85.58 ChartQA), so the remaining gap reflects query-agnosticism, not cache capacity. With a 32×3232\times 32 partition at frozen τ=2\tau = 2, Base MEKV closes its gap to per-head top-kk on InternVL at 20% (+1.5 pp) and reverses it at 10% (+4.1 pp; Table 4).
  • FastV-style (K=2K = 2): full instruction-free image prefill, all visual KV kept in layers 0–1, layers ≥2\ge 2 pruned by early-layer visual attention to the byte budget, so retained entries keep full-image context and the comparison isolates the retention rule at equal memory. PyramidKV-style: layer budgets decrease linearly from 1.5k1.5k at the lowest to 0.5k0.5k at the highest KV layer (mean kk) under the same SQA signal. Linear mixing is scoped as a best-of-sweep check on DocVQA-300 (Appendix H).

Q3. Scope of theoretical claims (§4, §5.2, Appendix C.5). We agree and have narrowed the claims. We do not claim a bound from −KL(a∥p(λ))-\mathrm{KL}(a \Vert p(\lambda)) to ANLS; the surrogate is an analytical model, and λ^\hat{\lambda} is a regime diagnostic and search initializer (it over-concentrates at generous budgets in direct no-search runs and converges at 5%; Table 7). Corollary 2 is now stated as conditional; since our oracle has no budget parameter, the budget shift in Figure 5 is reported as an empirical regularity. Proposition 1 decomposes concentration, not error, and does not rank off-path policies; direct evidence for the reversal comes from the ablations in §5.3. The absence of a significant secondary peak is described as consistent with single-peakedness, not proof of it.

Q4. Quotas and within-region selection varied independently (Table 3; Appendix I.1). On held-out DocVQA-786 and full ChartQA (full table in our response to Reviewer zzSY): on Qwen, calibrated quotas with deterministic within-region top-kk repair the failure in all four settings (+4.2 / +5.0 pp DocVQA, +4.2 / +8.5 pp ChartQA, all significant); tempering without quotas adds +6.7 to +11.8 pp; adding quotas atop tempering adds a further significant +2.1 to +4.4 pp in three of four settings. On InternVL, quotas alone bring no significant change, while tempering helps (+0.9 to +4.4 pp), consistent with its near-raw regime.

Gemma scope. We agree with your accounting: full8 compresses only the 8 full-attention layers (4.76% of visual KV, a 3.8–4.3% whole-cache reduction). The revision frames it as a full-attention-layer probe in a hybrid architecture, alongside the all48 arm (Table 10).

Reviewer_Q9A7OpenReview ↗

Official Comment of Submission54483 by Reviewer Q9A7

Thank you for the detailed responses. The revisions and additional experiments have resolved nearly all of my concerns, and I have increased my rating. My remaining concern is the lack of a comparison between MEKV and linear signal-coverage mixing on the same held-out images, with both methods' parameters selected on the development set and then frozen. This would clarify whether the exponential-family parameterization offers an advantage beyond the general benefit of balancing relevance and coverage, since the current best-of-sweep development results do not establish held-out superiority.

Dear Reviewer Q9A7,

Thank you for your careful re-evaluation and for raising your rating. Your detailed feedback led to substantial improvements in our revision.

Regarding your remaining point, we have launched the held-out comparison with linear signal-coverage mixing on DocVQA-786 and full ChartQA, with α\alpha selected per budget solely on DocVQA-300 and frozen across benchmarks, under the matched 3-seed protocol. The multi-seed runs are still in progress, and we will include the full comparison in Appendix H of the final manuscript.

Best regards,

The Authors

Reviewer_zzSYOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies query-agnostic visual KV-cache compression for serving multiple questions about the same image. It identifies over-concentration as a failure mode of relevance-based retention and proposes MEKV, which uses temperature-controlled regional allocation based on synthetic-question attention. The paper provides theoretical analysis of a KL-based surrogate objective and evaluates concentration control across three architectures and three benchmarks.

Critical strengths
  1. Interesting problem analysis. The finding that an informative relevance signal can hurt performance through over-concentration is useful. The reversal between Qwen and InternVL provides an informative example of architecture-dependent compression behavior.
  2. Simple method with transfer evidence. MEKV controls regional allocation through a single temperature. Transferring the development-selected temperature across benchmarks, together with same-signal comparisons and paired statistical tests, strengthens the empirical evaluation.
Critical weaknesses
  1. The theoretical connection to actual accuracy remains limited. Single-peakedness is established for the surrogate objective −KL(a∥pλ)-\mathrm{KL}(a\|p_\lambda), rather than downstream question-answering accuracy. The moment-matching estimator requires real-query oracle statistics and differs substantially from the measured optimum in several settings. Thus, the theory provides a useful interpretation but does not reliably determine the deployment temperature without empirical tuning.
  2. The practical advantage is conditional. SQA preprocessing costs approximately 18–25 seconds per image, but the number of subsequent queries needed to justify this cost is not established. Moreover, basic MEKV trails same-signal SnapKV-style selection on InternVL. The combined variant is promising, but changes both regional allocation and within-region sampling, making the contribution of each change less clear.
Questions for authors
  1. How well does the surrogate objective predict actual accuracy across temperatures, and can moment matching replace temperature search without substantial performance loss?
  2. Including SQA construction, what is the total latency and memory trade-off as the number of questions per image increases?
  3. Can the combined variant separately ablate calibrated regional quotas and tempered within-region sampling?
2 comments

Thank you for the constructive review. All points are addressed with new analyses in the revised PDF.

Q1. Surrogate prediction and moment matching without search (§4.1, §5.2, Appendix C.5). Across all eight empirical curves, accuracy peaks inside the temperature path with no significant secondary maximum (Table 10), though Theorem 1 proves concavity for −KL(a∥p(λ))-\mathrm{KL}(a \Vert p(\lambda)) rather than ANLS—now stated explicitly. Used directly without search on DocVQA-300, τ=1/λ^\tau = 1/\hat{\lambda} scores 74.1 / 54.4 vs. best-tested 75.3 / 55.6 on InternVL (−1.2 / −1.3 pp, and +3.5 / +8.8 pp over coverage), whereas on Qwen it over-concentrates at 20% / 10% (82.7 / 67.0 vs. 87.6 / 73.5), narrowing to −0.8 pp vs. frozen τ=4\tau = 4 at 5% budget (Table 7). Thus λ^\hat{\lambda} does not fully replace search. It serves as a closed-form regime diagnostic, computed once per model on a small development split and stable across resampled SQA sets (λ^\hat{\lambda} varies by <0.007<0.007; Table 22), and as a search initializer.

Q2. Total latency and memory trade-off vs. query count (§6, Appendix F). Per-image cost over QQ queries is a one-time offline SQA pass (25.0 s Qwen, 17.8 s InternVL—shared identically by the same-signal SnapKV- and PyramidKV-style baselines, with negligible quota-solve time) plus image prefill and QQ online queries (1.42–1.46 s/query Qwen, 1.02–1.07 s/query InternVL; Table 13). Online latency matches stratified eviction within ±0.4% and uncompressed serving within tens of ms, so the offline SQA pass does not amortize through latency. Instead, MEKV targets memory-bound multi-document serving, cutting peak visual KV memory by 77–89% (4.3×–9.0× more resident caches; Table 14) with higher accuracy at a fixed budget.

Q3. Separating regional quotas from tempered sampling (§5.3, Table 3; Appendix I.1). 2×22\times 2 factorial ablation on strictly held-out DocVQA-786 and full ChartQA (image-mean ×100, 20% / 10% budget):

Model & DatasetA: Per-head top-kkB: Quotas + top-kkC: Tempered (no quotas)D: Quotas + tempered
Qwen DocVQA-78677.66 / 64.7181.88 / 69.6889.33 / 76.5089.38 / 78.58
Qwen ChartQA61.37 / 44.0265.57 / 52.5368.07 / 53.4370.28 / 57.85
InternVL DocVQA-78678.24 / 65.2677.11 / 64.5382.10 / 69.6979.93 / 66.46
InternVL ChartQA79.75 / 71.6479.46 / 70.9080.62 / 74.5679.84 / 73.52
  • Qwen. Each concentration mechanism repairs the collapse independently: quotas alone (B − A) add +4.2 to +8.5 pp (all significant), and token tempering alone (C − A) adds +6.7 to +11.8 pp. Adding quotas atop tempering (D − C) adds a further significant +2.1 to +4.4 pp in three of four Qwen settings (DocVQA 10%, ChartQA 20% / 10%).
  • InternVL. 16×1616\times 16 quotas alone bring no gain (B vs. A) and slightly constrain D below C, while token tempering helps (+0.9 to +4.4 pp), consistent with InternVL's near-raw calibration regime.

Base MEKV vs. SnapKV-style on InternVL (§5.1, §5.3, Tables 2, 4). On held-out benchmarks, Base MEKV beats per-head top-kk on Qwen by +9.7 to +11.3 pp (Table 2). On InternVL, the gap reflects spatial granularity: refining Base MEKV to 32×3232\times 32 at frozen τ=2\tau = 2 matches per-head top-kk at 20% (+1.5 pp) and significantly exceeds it at 10% (+4.1 pp, 95% CI [+2.1, +6.0]; Table 4).

AuthorsOpenReview ↗

Thank You & Looking Forward to Further Discussion

Dear Reviewer zzSY,

Thank you again for your constructive feedback, which helped sharpen both our experiments and framing. In our response and the revised PDF, we have addressed each of your points—covering the surrogate and no-search moment matching, the latency–memory trade-off across query counts, the 2×22\times 2 ablation separating regional quotas from tempered sampling, and the InternVL granularity comparison with SnapKV-style selection.

If any part of the new results or revisions remains unclear, please let us know and we will be glad to clarify before the discussion window closes.

Best regards,

The Authors

Reviewer_qtLzOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies query-agnostic visual KV-cache compression, where a compressed cache must support future questions that are unavailable during compression. Its central observation is that an informative importance signal can still lead to poor retention decisions when the budget is excessively concentrated on a few high-scoring regions. The proposed training-free method, MEKV, uses temperature-controlled allocation to balance reliance on Synthetic Question Attention (SQA) against spatial coverage. The paper develops an exponential-family formulation of retention distributions, proves strict concavity of a KL-based surrogate objective, and derives a moment-matching condition for its optimum. It further distinguishes signal-aligned concentration from off-path allocation structure, and investigates the conditions under which concentration calibration versus finer-grained selection should be prioritized. Experiments analyze three architectures and evaluate document and text-centric QA benchmarks.

Critical strengths
  1. Useful distinction between signal quality and allocation concentration. A positively informative importance signal does not necessarily justify concentrating retention on its top-scoring regions, especially when future queries are unknown.
  2. Interpretable theoretical framework. The temperature path, moment-matching condition, and concentration decomposition coherently characterize retention policies. The proofs identify when the surrogate optimum is interior or at an endpoint, and the formulation accommodates regimes where sharpening beyond the raw signal is beneficial.
  3. Simple, potentially useful base method. MEKV is training-free and introduces a single temperature for its regional allocator. Cross-task temperature transfer and compatibility with finer-grained selection are practically appealing.
Critical weaknesses
  1. Behavior near the degenerate-signal regime is insufficiently characterized. The strict-concavity result assumes a non-constant log-ratio ℓ = log s − log u. When the SQA distribution approaches the coverage distribution, ℓ becomes nearly constant and the curvature, −Varₚλ[ℓ], approaches zero; at exact equality, all temperatures produce the same allocation. The paper does not sufficiently discuss how informative temperature calibration remains in this regime or how finite-sample SQA noise affects the moment-matching diagnostic.
  2. The connection between the theoretical surrogate and downstream QA performance remains incomplete. Theorem 1 establishes strict concavity of −KL(a‖pλ), not unimodality of ANLS or task accuracy. The paper appropriately labels this as a surrogate, and the empirical secondary-peak checks provide supporting evidence, but they do not establish a general correspondence between the surrogate optimum and the metric-optimal temperature.
  3. The practical cost of constructing the query-agnostic cache remains insufficiently contextualized. Appendix F reports one-time SQA costs (25.0 s/image Qwen, 17.8 s InternVL) and correctly frames the contribution as quality under a fixed budget, not latency. However, comparable overhead to stratified eviction does not show when the pipeline is advantageous.
  4. The empirical concentration findings depend on a spatial partition whose sensitivity is not established. SQA aggregation, concentration measurement, and integer quota allocation all depend on the binning scheme.
Questions for authors
  1. As SQA approaches the coverage distribution, how stable are the moment-matching estimate, realized allocations, and downstream accuracy across independently generated proxy-question sets, and is a coverage fallback appropriate in this regime?
  2. What downstream scores does the moment-matched temperature achieve without a subsequent performance-based grid search, and how much accuracy is lost relative to the best tested temperature? 3.As the number of downstream queries increases, how does total per-image cost, including SQA construction, compare with coverage-based and uncompressed serving?
  3. How sensitive are the selected temperature and architecture-dependent strategy ordering to grid resolution and offset, and do the main conclusions hold under alternative feasible partitions?
2 comments

Thank you for the precise questions; each led to new analyses now included in the revised PDF.

Q1. Degenerate-signal regime and proxy-question noise (§4.1, §5.3, Appendix I.2).

  • Built-in fallback. As s→us \to u, ℓ→0\ell \to 0 and the surrogate curvature −Var[ℓ]→0-\mathrm{Var}[\ell] \to 0, so every λ\lambda yields p(λ)=up(\lambda) = u: MEKV collapses to stratified coverage by construction, exactly when there is no signal to calibrate. Our settings lie far from this limit (SQA–oracle correlation ρ\rho = 0.66–0.78).
  • Proxy resampling. Regenerating the proxy questions three times with sampled decoding (T=0.7T = 0.7) changes frozen-τ\tau MEKV by only 0.4–1.5 pp, and every paired 95% CI against the greedy set includes zero (Table 22). λ^\hat{\lambda} stays within 0.606–0.612 (Qwen) and 0.994–0.995 (InternVL), so the regime diagnosis is unchanged.

Q2. Moment-matched temperature without search (§5.2, Appendix C.5). Direct 3-seed inference at τ=1/λ^\tau = 1/\hat{\lambda} on DocVQA-300 (ANLS ×100):

PolicyQwen 20%Qwen 10%InternVL 20%InternVL 10%
τ=1/λ^\tau = 1/\hat{\lambda} (no search)82.767.074.154.4
Best tested τ∗\tau^\ast87.673.575.355.6
Stratified coverage84.765.170.645.6

On InternVL, λ^\hat{\lambda} lands within 1.2–1.3 pp of τ∗\tau^\ast and +3.5 / +8.8 pp above coverage; on Qwen it over-concentrates (−4.9 / −6.5 pp at 20% / 10%, narrowing to −0.8 pp vs. frozen τ=4\tau = 4 at 5% budget; Table 7). We therefore present λ^\hat{\lambda} as a closed-form regime diagnostic and search initializer, not a search replacement.

Q3. Total cost vs. number of queries (§6, Appendix F). Relative to both coverage and uncompressed serving, MEKV's only extra cost is a one-time offline SQA pass (25.0 s/image Qwen, 17.8 s InternVL), which the same-signal SnapKV- and PyramidKV-style baselines also pay; its quota solve is negligible. Online per-query latency (~1.0–1.5 s/query; Table 13) matches stratified eviction (within ±0.4% across 24 configurations) and is no slower than uncompressed serving. The SQA pass therefore does not amortize through latency. What it buys is accuracy and capacity at a fixed memory budget: peak visual KV memory drops by 77–89%, fitting 4.3×–9.0× more document caches. We state this trade-off explicitly in §6.

Q4. Grid resolution and offset (§5.3, Table 4; Appendix I.3). Base MEKV minus same-signal per-head top-kk (pp), DocVQA-300, frozen τ\tau:

PartitionQwen 20%Qwen 10%InternVL 20%InternVL 10%
8×88\times 8+15.4+13.0−4.8−9.5
16×1616\times 16 (standard)+16.0+14.5−2.5−4.4
16×1616\times 16 (half-cell shifted)+14.0+12.4−3.6−6.0
32×3232\times 32+10.2+15.6+1.5+4.1

On Qwen, MEKV also beats matched-grid coverage in every partition (+2.2 to +19.6 pp), and τ∗\tau^\ast stays interior ([1.41, 5.66]). On InternVL, the comparison improves monotonically with resolution, reaching parity at 20% and a significant lead at 10% under 32×3232\times 32 (95% CI [+2.1, +6.0]). Allocation granularity is the binding constraint on InternVL, and finer spatial bins can supply it.

Surrogate–accuracy link. We agree that Theorem 1 concerns −KL(a∥p(λ))-\mathrm{KL}(a \Vert p(\lambda)), not ANLS. The revision states this explicitly, describes the secondary-peak checks as consistent with single-peakedness rather than proof of it, and grounds the accuracy claims in the held-out and factorial experiments.

AuthorsOpenReview ↗

Thank You & Looking Forward to Further Discussion

Dear Reviewer qtLz,

Thank you again for your thoughtful and constructive review. Your questions have been genuinely helpful in improving our paper. In our response and the revised PDF, we have addressed each of your points—covering degenerate signals and proxy-question noise, moment matching without search and the surrogate–accuracy scoping, total cost versus query count, and grid resolution and offset.

Please let us know if anything would benefit from further clarification; we will respond promptly before the discussion period closes.

Best regards,

The Authors

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response and Summary of Manuscript Revision

We thank all reviewers for their constructive feedback and have uploaded a revised PDF. Below is a summary of how the shared concerns are resolved in the revision:

Unified held-out evaluation (§5, Table 2; Appendices B.3, G). We document the exact disjoint splits (strictly held-out DocVQA-786, dev-excluded DocVQA-986, and fully held-out ChartQA and TextVQA), add uncompressed and real-query oracle references, and specify all baseline adaptations (including post-prefill FastV at K=2K=2). With a single frozen τ\tau per model, Base MEKV beats spatial coverage in every held-out setting (+4.2 to +17.9 pp on Qwen; +5.2 to +15.8 pp on InternVL). 2×22\times 2 factorial ablation (§5.3, Table 3; Appendix I.1). Varying regional quotas and within-region tempered sampling independently shows that each concentration-control mechanism repairs Qwen on its own (+4.2 to +8.5 pp from quotas alone; +6.7 to +11.8 pp from tempering alone), while combining them improves over per-head top-kk by up to +13.9 pp on held-out benchmarks and preserves InternVL’s gains. Proxy-question and spatial-grid robustness (§5.3, Table 4; Appendices I.2–I.3). Regenerating SQA questions three times (T=0.7T=0.7) changes MEKV by only 0.4–1.5 pp (all non-significant) and λ^\hat{\lambda} by <0.007<0.007. Across 8×88\times 8 to 32×3232\times 32 and half-cell-shifted grids, Qwen consistently beats per-head top-kk (+10.2 to +16.0 pp), while 32×3232\times 32 binning brings InternVL Base MEKV to parity with per-head top-kk at 20% (+1.5 pp) and significantly past it at 10% (+4.1 pp). Diagnostic framing and serving trade-offs (§3–§6; Appendices C.5, F). We reframe Theorem 1 and λ^\hat{\lambda} as an analytical surrogate and regime diagnostic (adding direct no-search τ=1/λ^\tau=1/\hat{\lambda} results), state Corollary 2 conditionally, clarify Gemma full8 as a full-attention layer probe (4.76% of visual KV), and report per-query latency alongside 4.3×–9.0×4.3\times\text{–}9.0\times resident cache capacity gains.

Point-by-point answers and compact tables tailored to each review are provided in the individual responses below.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.