acceptodds
Under review as a conference paper at ICLR 2027

SelKV: Selective KV Cache Merging with Per-Token Merge-or-Drop and Attention Compensation

Abstract

Large Language Models (LLMs) rely on a key-value (KV) cache whose memory footprint grows linearly with context length, creating a major inference bottleneck. Token merging methods reduce this cost by consolidating evicted tokens into retained cache entries rather than discarding them, but existing approaches merge every evicted token uniformly, regardless of whether its content is compatible with its merge target, and correct the resulting softmax under-weighting of merged positions, when they correct it at all, using raw merge counts that become unstable under aggressive compression. We propose SelKV, a training-free framework with two components: a soft cosine gate that modulates merge intensity per token based on value-vector similarity, continuously interpolating between full merge and drop, and an attention-ratio compensation mechanism that corrects softmax imbalance using a bounded, prefill-derived logit bias in place of unstable count-based correction. Across three models spanning MHA and GQA architectures on 16 LongBench datasets, both components are shown to be statistically significant, and SelKV substantially outperforms existing merging baselines, with the margin widening further under aggressive compression. Against strong eviction baselines, SelKV matches performance overall while exceeding full-cache quality on complex multi-document QA, and on the RULER benchmark it retains 95% of full-cache quality at contexts up to 64k tokens. At 100k tokens, SelKV delivers a 3.3x decoding speedup over the full cache.

Primary area:
optimization
Keywords:
KV cache compression, token merging, attention compensation, large language models, inference efficiency

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 4 reviews, above 27% of the others.
Reviewer_q2CcOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

SelKV is a KV cache compression method that requires no training and compresses once after prefill. It targets two problems in existing token merging methods: every evicted token receives the same merge or drop decision regardless of compatibility with its merge target, and a merged entry representing several tokens still gets roughly the attention of one token, a problem that count-based corrections worsen under aggressive compression. SelKV answers these with a soft cosine gate on value vectors that interpolates between full merge and drop, and with a decode-time logit bias derived from prefill attention ratios, bounded by the redistributed attention mass.

The method is evaluated on 16 LongBench datasets across three MHA and GQA models at 25% retention, and on RULER up to 64k tokens. As I read it, the contributions are the two mechanisms and a bootstrap-validated account of when merging helps: blind merging incurs a significant loss that the full method recovers, SelKV outperforms prior merging baselines, and it reaches parity, rather than superiority, with strong eviction. The paper also reports a 3.3x decoding speedup over the full cache at 100k tokens.

Critical strengths
  • The central claims rest on paired bootstrap tests over ablations where all configurations share the same token selection and code path, and the eviction comparison is reported as not significant rather than buried.

  • The paper shows that blind merging is a measured failure mode, not an assumption: under identical selection, eviction alone beats unconditional merging by 0.69 points (p below 0.001), and the gate plus compensation recovers essentially all of that loss. The staged ablation separates what each component contributes.

Critical weaknesses

Weakness 1: Merging is justified against a dichotomy that no longer holds, and the paper's own statistics show it reaches at best parity with simpler eviction

  • The paper motivates merging by contrasting it with eviction: eviction destroys information irreversibly, while merging retains it. I had trouble accepting this framing. Sparse attention with an indexer over the full cache, as in DeepSeek-V3.2, selects tokens for each query and discards nothing, and the full cache can be offloaded instead of compressed. Recent production architectures (e.g., Deepseek-v4 HCA and CSA) learn compression during training, and their attention operates on the compressed entries directly. With these alternatives in place, the choice between destroying tokens and merging them is not exhaustive. The under-weighting of merged positions that the paper corrects is a cost of applying merging after training, not an unavoidable property of compression. I looked for a discussion or a baseline covering either alternative and could not find one.

  • The paper's own results remove the reason to adopt merging even within the framing it chose. The paired bootstrap comparison against eviction gives +0.12 points with p = 0.20, and the paper correctly does not claim superiority. Parity is also the best case on offer: at 31.5k tokens SelKV ties SnapKV exactly at 48.46, at 10% retention SnapKV leads (Table 10, Appendix K), PyramidKV leads on the MHA model, and SnapKV leads on code tasks. The one setting where merging could hold an advantage, QA where the evidence is spread across documents, is described by the authors as suggestive rather than conclusive. What the paper establishes is that merging can be made no worse than eviction, at the cost of a second prefill pass, routing, and a logit bias during decoding. The design space is thin as well: the selection rule is standard and contributes almost nothing (Appendix H), the choice of gate signal barely moves the average (Table 6), and the compensation refines KeepKV's logit bias. I do not see the case for preferring this machinery over the simpler baseline it matches.

Weakness 2: The aggressive compression claim conflicts with Table 10, several reported numbers have no table behind them, and RULER is reported without an eviction baseline

  • Several reported numbers do not reconcile with the paper's own tables. Section 4.4 states that at 10% retention SelKV drops only 1.75%, from 39.81 to 38.06. Table 10 reports 37.56 for SelKV at 10%. The 38.06 in that sentence is the SnapKV number. With the table value the drop is 2.25%, and SnapKV is ahead. The same section cites 10% scores for KVMerger and WeightedKV and points to Appendix K, but the tables in Appendix K contain neither method. Table 2 and Table 6 report different scores for the same configurations: 39.01 against 38.96 with no gate and no compensation, and 39.57 against 39.59 with the value cosine gate alone. Appendix M states that decoding is greedy and deterministic, so identical configurations should give identical scores. The gaps for individual tasks quoted in Section 4.4, such as +4.89 on Qasper against KVMerger, appear in no table.

  • The omitted comparisons are the ones the abstract relies on. RULER is reported against the full cache only, so the claim of retaining 95% quality at 32k and 64k tokens has no eviction comparison point. D2O is the only prior method that partitions tokens into drop and merge sets, exactly the design choice the gate is positioned against, and it is not evaluated. The main LongBench table runs two of the three models on inputs of 3.5k tokens, short for a long context compression paper. The speedup at 100k tokens in the abstract comes with no quality measurement at that length. The limitations section discloses each of these gaps, but the abstract still carries the corresponding numbers as contributions.

Questions for authors

Question 1: Section 4.4 reports SelKV at 38.06 at 10% retention, but Table 10 reports 37.56, and 38.06 is SnapKV's score. Which number is correct, and if Table 10 is, how should the abstract's claim that margins widen under aggressive compression be read when SnapKV leads at the most aggressive setting? Please also point to where the cited 10% scores for KVMerger (32.98) and WeightedKV (33.71) appear, since the tables in Appendix K contain neither method.

Question 2: Table 2 and Table 6 report different scores for the same configurations: 39.01 against 38.96 with no gate and no compensation, and 39.57 against 39.59 with the value cosine gate alone. Appendix M states that decoding is greedy and deterministic. What exactly differs between the two experiments?

Question 3: D2O is excluded on the ground that its official implementation supports only LLaMA-2, yet Appendix E reports your own results on LLaMA-2-7B. What prevents evaluating D2O on LLaMA-2-7B, the one model where both methods are available? It is the comparison most directly matched to the gate's claim, since D2O is the only prior method that partitions tokens into drop and merge sets.

Question 4: RULER is reported against the full cache only, so the 95% retention headline at 32k and 64k has no eviction comparison point. Can SnapKV be added to Table 4 at these two lengths? Appendix M puts a full evaluation on 16 datasets at 1 to 2 GPU hours, so the cost appears modest.

Question 5: The related work stops around the middle of 2025 and omits the directions that most directly bear on the paper's framing: DeepSeek-V3.2's sparse attention, where a lightning indexer selects from the full cache and discards nothing, and DeepSeek-V4's CSA, which learns compression during training. Why are these not discussed, and under what serving conditions is this form of lossy compression after prefill preferable to either?

Question 6: The aggregate gap over eviction is +0.12 and not significant, and the paper locates merging's remaining advantage on tasks where evidence is spread across documents. On which individual tasks or prompts does SelKV beat the eviction only configuration of Table 2, and does the win concentrate where evidence is distributed?

4 comments

We recognize this review raises fundamental questions about the paper's positioning, and we address them directly.

CW1: Merging motivation and DeepSeek-V3.2/V4

The reviewer raises an important point about the evolving landscape. We agree the drop-vs-merge dichotomy is not exhaustive and will revise the introduction.

DeepSeek-V3.2's sparse attention uses a full-cache indexer that selects tokens per query without discarding anything. This is powerful but has different requirements: it maintains the full KV cache in memory (or on secondary storage with offloading) and adds a retrieval step per query. SelKV targets the setting where the cache must physically fit in GPU memory, e.g., serving multiple concurrent requests on commodity hardware where offloading is too slow.

DeepSeek-V4's CSA/HCA learns compression during training, producing superior compressed representations, but requires training-time integration and architecture-specific design. SelKV is training-free and applies post-hoc to any existing model. CSA/HCA is for frontier model design; SelKV is for compressing the cache of already-deployed models that cannot be retrained.

We will add a "Positioning" subsection to the introduction distinguishing three compression paradigms:

  1. Training-time (CSA, HCA): best quality, requires architecture redesign
  2. Full-cache with sparse retrieval (DeepSeek-V3.2): no information loss, requires full cache in memory
  3. Post-prefill compression (SelKV, SnapKV, H2O): training-free, reduces physical cache size, applicable to any model

SelKV contributes to paradigm 3, which remains relevant for serving existing models under memory constraints.

CW2: Merging vs eviction, why prefer the machinery?

We accept the reviewer's assessment that SelKV achieves parity with eviction, not superiority, in aggregate. The case for merging rests on three points:

  1. Safety: Without the gate, merging loses 0.65 points (p<0.001). The gate makes merging safe. Among post-prefill compression strategies, merging is the only one that preserves evicted information within the compressed cache (as opposed to sparse retrieval or offloading, which maintain a full copy); if it can be made reliable, it opens a path to better future methods.

  2. Per-task advantages: On all three multi-document QA tasks, SelKV outperforms SnapKV: musique +0.66, hotpotqa +0.19, 2wikimqa +0.05. These are the tasks where compressed information matters, since the answer requires synthesizing evidence from multiple locations and eviction permanently loses some of it. SelKV wins on 8 of 16 tasks (2 ties, 6 losses; see Q6 below for the full table).

  3. Merging-method comparison: SelKV outperforms all prior merging methods by large margins (KVMerger +1.70, WeightedKV +1.39, LOOK-M +1.19, all p<0.001) and the eviction baseline H2O by +1.59. For practitioners who choose merging for the reasons above, SelKV is the best option.

Regarding the "thin design space" concern: we agree the token selection rule is standard (Appendix H), but the gate signal choice matters more than the aggregate suggests. Value cosine outperforms both key cosine (+0.29 without compensation) and an EMS-style key×value gate (+0.75 with compensation, p<0.001), a meaningful difference supported by the ablation: the value-only gate empirically outperforms alternatives that incorporate key information. The compensation mechanism is also fundamentally different from KeepKV's merge-count bias: attention-ratio compensation is bounded by the actual redistributed attention mass, while merge-count compensation uses raw counts that become unstable under aggressive compression (KeepKV scores 38.90 vs SelKV's 39.90).

We acknowledge this is a narrower contribution than the original framing suggested, and will revise the abstract and introduction accordingly.

Q1: Section 4.4 numerical error

The reviewer is correct. Table 10 reports SelKV=37.56 and SnapKV=38.06 at 10% retention. The Section 4.4 text incorrectly cites 38.06 for SelKV, which comes from a different experiment. SnapKV leads at 10% retention.

The abstract's claim that "margins widen under aggressive compression" refers to the comparison against other merging methods (KVMerger=32.98, WeightedKV=33.71), not against eviction. We will clarify this and correct the text.

Q2: Table 2 vs Table 6 discrepancy

Tables 2 and 6 came from separate experimental runs, both using the code's default hyperparameters rather than the values specified in Section 4.1. This affected Tables 2 and 6 (LLaMA-3.1 ablation); Tables 1 and 4 used correct settings and have been re-verified. We re-ran all LLaMA-3.1 configurations at 25% from a single checkpoint with the correct settings (see the unified 2×2 factorial in our response to Reviewer WE9x Q6). All baseline methods were also re-run. All numbers in the revision will come from this unified run.

Q3: D2O comparison

We ran D2O on all three models, including LLaMA-2-7B as the reviewer requested. We implemented D2O's core algorithm (key-cosine nearest-neighbor matching, mean-similarity threshold, similarity-weighted merging) within our framework for a controlled comparison (same token selection, same retention budget, same evaluation code):

MethodLLaMA-2-7BLLaMA-3.1-8BGemma-2-9B
SelKV31.7839.9042.31
SnapKV (eviction)32.7739.7742.16
D2O31.8939.4842.13

On LLaMA-3.1, SelKV leads D2O by +0.42 (paired bootstrap, one-sided p<0.001, 95% CI [+0.20, +0.63]), winning 13/16 tasks. On Gemma-2, SelKV leads by +0.18. On LLaMA-2 (MHA), D2O slightly leads SelKV (31.89 vs 31.78), though both trail SnapKV (32.77). This is consistent with the paper's finding that merging is weaker under MHA where union-based selection does not apply.

See Reviewer fg8h Q1 for per-task breakdown and implementation details.

Q4: RULER with SnapKV

We added SnapKV and 5 other methods to RULER (LLaMA-3.1-8B, 25% retention):

Method4k32k64k
Full Cache92.184.481.2
SnapKV (eviction)80.782.278.2
SelKV (gate+comp)79.980.577.0
Merge-all (no gate)79.675.475.7
H2O40.677.975.8
LOOK-M28.170.672.0

SnapKV leads SelKV by 0.8 at 4k, 1.7 at 32k, and 1.2 at 64k, consistent with the aggregate LongBench finding. RULER's tasks are predominantly exact-match retrieval where token identity matters, the regime where eviction has a natural advantage. We present this transparently. SelKV outperforms all other merging methods; compensation drives the largest gain at 32k (+5.2: gate-only 75.3→80.5) while the gate contributes at 64k (+1.4).

Q5: DeepSeek-V3.2/V4 in related work

See CW1 above. We will add these to the related work with a discussion of when post-prefill compression remains the right choice.

Q6: Per-task analysis of merging vs eviction

Using SnapKV as the eviction baseline (same token selection, eviction-only), full per-task breakdown on LLaMA-3.1-8B (25% retention):

TaskCategorySnapKVSelKVΔ
trecFew-shot65.5066.50+1.00
triviaqaFew-shot89.2490.00+0.76
musiqueMulti-doc QA18.8919.55+0.66
multifieldqa_enSingle-doc QA49.8850.27+0.39
hotpotqaMulti-doc QA39.7539.94+0.19
narrativeqaSingle-doc QA21.3821.56+0.18
samsumFew-shot42.1042.19+0.09
2wikimqaMulti-doc QA35.8635.91+0.05
passage_retrieval_enSynthetic34.0034.00+0.00
multi_newsSummarization23.9123.91+0.00
passage_countSynthetic4.084.03−0.05
gov_reportSummarization28.0928.00−0.09
repobench-pCode58.4858.31−0.17
qmsumSummarization21.5621.38−0.18
qasperSingle-doc QA38.9238.62−0.30
lccCode64.6964.17−0.52
AVERAGE39.7739.90+0.13

SelKV wins on 8 tasks, ties on 2, SnapKV wins on 6. Note: SnapKV uses the same importance scoring (Eq. 1) as SelKV but evicts rather than merges, so this is a direct eviction-vs-merging comparison. Per-task deltas are indicative (all methods were re-run from a single checkpoint; we do not have per-task significance tests). The pattern aligns with the design:

  • SelKV wins on all 3 multi-document QA tasks (musique +0.66, hotpotqa +0.19, 2wikimqa +0.05), where answers require synthesizing evidence from multiple locations and merging preserves information that eviction discards.
  • SnapKV wins on code (lcc −0.52) and retrieval (qasper −0.30), where tokens carry unique structural identity that averaging corrupts.

Merging helps when compatible information is distributed, and hurts when token identity is critical. The gate limits the worst per-task delta to −0.52 (lcc).

I recognize the authors for the additional experiments and detailed response.

But the new results do not quite restore the case for the method. The revised positioning puts SelKV in the same setting as SnapKV, and the simpler eviction baseline matches or beats SelKV. Since the contribution reads as weaker after the rebuttal, I will keep my rating at 1. After all, I thank the author's honest disclosure.

We appreciate the reviewer's acknowledgement and fair reading of the results. We want to clarify one point about framing, since the rebuttal may have put too much weight on the SelKV-vs-SnapKV comparison.

We agree that merging does not beat eviction in aggregate, and that is not the claim we are making. What we show is that blind merging loses 0.65 points vs eviction (p<0.001), that prior merging methods all operate in this regime, and that the gate and compensation recover the loss entirely. The ablations show why: value-space compatibility turns out to be the right merging criterion (+0.75 over EMS-style key×value, p<0.001), and attention-ratio compensation replaces KeepKV's count-based correction which becomes harmful at high compression. These findings apply to any merging pipeline, not just ours.

The per-task picture also matters: SelKV wins 8/16 tasks (all three multi-doc QA), SnapKV wins 6 (code, retrieval). Merging and eviction fail on different tasks for different reasons, and knowing when each applies is useful even if the averages tie.

We respect that the reviewer may still consider this insufficient for acceptance. We have revised the abstract and claims accordingly.

Reviewer_WE9xOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper addresses a practical challenge in long-context language model inference: reducing KV cache memory consumption while preserving useful contextual information. Conventional token merging can introduce representation distortion by indiscriminately combining incompatible tokens and may disrupt attention allocation after compression. To address these issues, SelKV introduces a training-free framework combining value-similarity-based selective merging with attention-ratio compensation. The paper highlights the importance of representation compatibility and attention redistribution in KV cache compression, offering useful insights into improving the reliability of token merging. The evaluation investigates compression quality, component effectiveness, robustness under different compression levels, and inference efficiency, providing empirical support for the proposed approach.

Critical strengths

The paper addresses a practically relevant problem in memory-efficient long-context inference. Its training-free design makes the approach potentially applicable to existing language models without additional optimization or retraining. The proposed combination of selective merging and attention compensation offers a concrete attempt to reduce representation distortion and preserve information during cache compression. The controlled ablation experiments provide statistical evidence that indiscriminate merging can degrade performance and that the proposed mechanisms can mitigate this degradation. Comparisons across different model architectures and compression levels offer useful observations about the method's applicability and limitations. The paper also includes sensitivity analyses, long-context evaluations, and inference efficiency measurements, extending the investigation beyond prediction quality to practical deployment considerations. These results provide meaningful empirical support for improving the reliability of KV cache merging.

Critical weaknesses

The paper's main weaknesses concern an incomplete specification of its attention-based routing mechanism, insufficient justification for value-only similarity as a merging criterion, and experimental evidence that does not fully establish the novelty and necessity of continuous gating. Sections 3.2 and 3.4 appear to require different portions of the attention matrix, but Section 3.1 describes computing attention weights only for the final observation-window queries. It is therefore unclear how the routing stage obtains attention weights for arbitrary evicted tokens or whether additional computations are included in the reported complexity. Furthermore, value-vector similarity alone does not necessarily establish functional compatibility between tokens, since their keys may respond differently to future queries. Although the proposed soft gate improves upon indiscriminate merging, the experiments do not compare it against simple binary similarity-based gating or sufficiently isolate its advantages over closely related methods such as D2O and EMS, which have already explored selective merging and similarity-weighted aggregation. Consequently, it remains unclear whether continuous gating provides a substantive methodological advance or primarily reformulates established techniques. Finally, the long-context evaluation lacks strong eviction baselines and corresponding component ablations, limiting the evidence that the proposed mechanisms remain competitive and effective in challenging long-context retrieval settings. These limitations weaken confidence in both the methodological distinctiveness and the generality of the reported improvements.

Questions for authors

(1) The definition and computation of attention weights are insufficiently clarified across different stages of the proposed method. Section 3.2 aggregates attention from observation-window queries to cached tokens, whereas Section 3.4 requires attention from evicted tokens to retained tokens. Do these operations use the same attention tensor (A^{(\ell)}_{h,i,j})? How are the corresponding query and key positions defined, and how are the attention weights required by each stage obtained?

(2) The merge-target routing mechanism is not fully specified and appears inconsistent with the available prefill attention information. Section 3.1 computes attention weights only for the final 32 query positions, while Equation (3) requires attention weights from arbitrary evicted tokens to candidate merge targets. How are these additional weights obtained, and are the associated computational costs included in the reported complexity and latency? Furthermore, how does the method handle routing buckets without retained tokens or candidate targets that are inaccessible under causal attention constraints?

(3) The theoretical justification for using value cosine similarity alone to determine merging compatibility remains insufficient. Similar value vectors do not necessarily imply similar contributions to future attention outputs, particularly when their corresponding keys respond differently to future queries. Since SelKV modifies retained values while leaving their keys unchanged, why is value similarity sufficient to determine merging compatibility and merge intensity? Can the authors provide a stronger theoretical or empirical justification for this design choice?

(4) The existing experiments do not adequately establish the necessity of continuous gating. Although the ablations demonstrate improvements over indiscriminate merging and compare different similarity signals, they do not isolate the benefit of continuous merge intensity. Can the authors compare the proposed soft cosine gate against binary similarity-based gates with fixed or validation-selected thresholds under identical selection, routing, merging, and compensation settings? This would clarify whether continuous modulation provides additional benefits beyond filtering incompatible tokens.

(5) The novelty of the proposed selective merging mechanism relative to closely related methods remains insufficiently established. D2O already explores similarity-based merge-or-drop decisions and similarity-weighted merging, while EMS incorporates both key and value similarities into merging decisions. What substantive methodological advantage does SelKV provide beyond these existing mechanisms? Can the authors include controlled comparisons against D2O-style weighting and EMS-style merging under identical token selection, routing, and cache budgets to demonstrate the independent contribution of the proposed soft gate?

(6) The experimental design does not fully isolate the individual contributions and potential interaction of the two proposed components. Table 2 evaluates merge-all, gate-only, and gate-plus-compensation configurations but omits merge-all with compensation alone. Without a complete factorial comparison, it is difficult to determine whether the gains arise from independent improvements or genuine complementarity between gating and compensation. Can the authors provide a complete 2×2 ablation with statistical confidence intervals and clarify how much of the improvement over existing merging baselines is attributable specifically to each proposed component?

(7) The long-context experiments do not adequately establish the competitiveness or robustness of SelKV relative to existing compression methods. The RULER evaluation compares SelKV only against the full cache, without strong eviction baselines or component-level ablations. Can the authors provide comparisons with SnapKV, eviction-only, merge-all, and gate-only configurations at 4k, 32k, and 64k tokens under identical retention budgets? These results would help determine whether the proposed mechanisms remain effective as context length increases and whether their benefits extend beyond comparisons with the uncompressed cache.

(8) Several inconsistencies in the reported experimental results raise concerns about the reliability of the quantitative analysis. Section 4.4 reports a score of 38.06 for SelKV at 10% retention, whereas Appendix K, Table 10 reports 37.56, with 38.06 corresponding to SnapKV. Consequently, the reported performance decrease from 25% retention should be 2.25 rather than 1.75 points. Additionally, the gate-only results differ across Table 2, Appendix D, and Appendix F. Can the authors verify these values against the original experimental records and explain whether the discrepancies arise from reporting errors or differences in experimental configurations?

4 comments

We address each question below.

Q1 & Q2: Attention computation across stages

Thank you for identifying this imprecision. Section 3.1 describes a memory-efficient two-pass approach (SDPA prefill + observation-window re-pass). In our implementation, we use a single eager forward pass over the full sequence, which produces the complete attention matrix A^(ℓ)_{h,i,j} (shape H × n × n per layer). Different stages use different portions of this tensor:

  • Importance scoring (§3.2): Uses observation-window attention, the last W=32 rows of the attention matrix: s_j = Σ_{i∈W} A_{i,j} · ‖v_j‖. This is equivalent to the two-pass approach described in Section 3.1.
  • Bucket-attention routing (§3.4, Eq. 3): Uses the full attention matrix. For each evicted token j, the routing looks up A_{h,j,i} for all retained tokens i in the same bucket and selects argmax. This requires j to index the query dimension, which is why the full matrix is needed, the observation window alone cannot provide attention from arbitrary evicted positions.
  • Merging weights and compensation (§3.5–3.6, Eqs. 5–6): Cumulative attention a_{h,i} is summed over all queries (not just the observation window), and R_i = (a_{kept,i} + Σ_{j→i} g_j · a_{disc,j}) / a_{kept,i}. Since R_i ≥ 1, the mask α · log(R_i) ≥ 0, boosting attention to merged slots proportionally to the discarded attention routed to them.

Computational cost: The eager prefill adds ~33ms over SDPA-only at 4096 tokens (175ms vs 142ms on H100). This cost is already reflected in our reported latency (see Reviewer 8ode Q1). We will correct Section 3.1 to distinguish the observation-window attention used for scoring from the full attention used for routing and merging.

Edge cases: Since top-k selection is global (not per-bucket), some buckets may contain no retained token. When an evicted token has no same-bucket target, it falls back to global-attention routing: it picks the retained token it attended to most across all buckets.

We will add a "Computation Pipeline" box to Section 3 in the revision.

Q3: Value cosine similarity justification

Value cosine similarity answers the specific question the gate must decide: "will averaging v_evicted and v_target produce a faithful representation?"

The reviewer raises a valid concern: similar values with dissimilar keys could mean the merged entry is retrieved by the wrong queries. Attention-weighted merging (Eq. 5) mitigates this by weighting each evicted token's contribution by its prefill attention, so low-attention tokens barely affect the merged value regardless of gate value.

Empirically, the gate-space ablation is decisive. We compare all gate variants with compensation enabled (LLaMA-3.1-8B, 16 datasets, 25% retention):

Gate spaceAvgΔ vs value
Value cosine (ours)39.90--
Key × Value (EMS-style)39.15-0.75
No gate (merge all)39.51-0.39

Without compensation, the ordering is consistent: value cosine (39.69) > key cosine (39.40) > no gate (39.12).

Value cosine outperforms key×value gating on 12/16 tasks (paired bootstrap, one-sided p<0.001, 95% CI [+0.26, +1.39]). The largest losses for the key×value gate are on trec (-4.50) and triviaqa (-1.96), tasks where the key-similarity factor incorrectly suppresses merges that the value gate alone would permit. Adding key similarity introduces a stricter filter that discards useful merges.

Q4: Binary vs continuous gating

We ran binary threshold gates on LLaMA-3.1-8B (16 datasets, 25% retention, no compensation):

Gate typeThresholdAvg
No gate (merge all)n/a39.12
Binary (hard)0.139.48
Binary (hard)0.339.55
Binary (hard)0.539.51
Soft cosine (ours)n/a39.69

All gating strategies improve over no gate (+0.36 to +0.57). The soft cosine gate leads the best binary threshold (0.3) by +0.14. Beyond the accuracy gain, the soft gate has two practical advantages: (1) it requires no threshold selection, eliminating a hyperparameter that would need tuning per model and compression ratio, and (2) its continuous modulation preserves partial information from moderately compatible tokens that a binary gate would either fully merge or fully drop. 35% of evicted tokens receive gate values in the partial-merge range (0.1–0.8), which a binary gate cannot represent.

Q5: Comparison with D2O and EMS

D2O: We reimplemented D2O's core algorithm within our framework for a controlled comparison (see Reviewer fg8h Q1 for full results and why we reimplemented). SelKV outperforms D2O by +0.42 on LLaMA-3.1 (paired bootstrap, one-sided p<0.001, 95% CI [+0.20, +0.63]), winning on 13 of 16 tasks, and by +0.18 on Gemma-2. On LLaMA-2 (MHA), D2O slightly leads (31.89 vs 31.78), consistent with the paper's finding that merging is weaker under MHA.

EMS (Efficient Memory Sieve) incorporates both key and value similarities into its merge/drop decisions. To test whether this joint criterion improves SelKV, we implemented an EMS-style key×value gate within our framework: gate = cosine_sim(k_evict, k_target) × cosine_sim(v_evict, v_target), keeping all other components identical (same selection, routing, compensation). The value-only gate outperforms the key×value gate by +0.75 (39.90 vs 39.15, one-sided p<0.001), winning 12/16 tasks (see Q3 above for the full table). The empirical result is clear: the key-similarity factor introduces a stricter filter that discards useful merges.

EMS also differs architecturally: it operates within each attention head independently and uses per-step compression during decoding. SelKV's architecture-aware selection (union for GQA) and one-shot post-prefill compression target a different operating point. We will add a discussion of both methods to the related work section.

Q6: Complete 2×2 factorial ablation

Full factorial from a unified re-run of all configurations from a single code checkpoint (LLaMA-3.1-8B, 16 datasets, 25% retention):

No Compensation+ Compensation
No Gate (merge all)39.1239.51
+ Gate39.6939.90

The contributions are complementary but partially overlapping: the gate provides +0.57 without compensation (39.12→39.69) and +0.39 with it (39.51→39.90). Compensation provides +0.39 without the gate (39.12→39.51) and +0.21 with it (39.69→39.90). The sub-additivity (0.57+0.39=0.96 vs actual 0.78) indicates the two mechanisms address overlapping failure modes, but each contributes independently.

Paired bootstrap tests (10k iterations, hierarchical resampling of datasets and examples within each dataset; one-sided p-values testing H_a: Δ > 0):

EffectΔ95% CIp (one-sided)
Gate (no comp)+0.57[+0.21, +0.99]0.001***
Gate (+ comp)+0.39[−0.00, +0.81]0.026*
Comp (no gate)+0.39[−0.02, +0.87]0.035*
Comp (+ gate)+0.21[+0.03, +0.47]0.003**
Full SelKV vs merge-all+0.78[+0.30, +1.43]<0.001***

Both effects are significant at α=0.05 in both settings, with the full SelKV vs merge-all gap highly significant (+0.78, p<0.001).

Q7: RULER with eviction baselines and ablations

All requested configurations on RULER at 4k/32k/64k (LLaMA-3.1-8B, 25% retention):

Method4k32k64k
Full Cache92.184.481.2
SnapKV (eviction)80.782.278.2
PyramidKV (eviction)78.282.679.9
SelKV (gate+comp)79.980.577.0
Gate only79.975.377.1
Merge-all (no gate)79.675.475.7
H2O40.677.975.8
LOOK-M28.170.672.0

Eviction methods (SnapKV, PyramidKV) lead SelKV by 0.8–2.9 points, consistent with RULER's retrieval-heavy task mix favoring exact token preservation. SelKV outperforms all other merging methods on RULER: LOOK-M collapses at 4k (28.1), H2O at 4k (40.6), while SelKV holds at 79.9. Compensation drives the largest gain at 32k: gate-only (75.3) ≈ merge-all (75.4), but adding compensation yields 80.5 (+5.2). The gate contributes at 64k (+1.4: 75.7→77.1). The gap between SelKV and SnapKV ranges from 0.8 (4k) to 1.7 (32k).

Q8: Numerical inconsistencies

We apologize for these errors.

Error 1, Section 4.4 SelKV at 10%: The text cites 38.06, which comes from a different run (ours_r10_recent16, using the correct m_r=16) than Table 10's 37.56 (original run). SnapKV also scores 38.06 in the original run, so the coincidence is real but creates confusion. The core issue: Section 4.4 and Table 10 cite different SelKV runs. In neither case does SelKV lead SnapKV at 10% (at best they tie). The "margins widen under aggressive compression" claim is incorrect for the SelKV-vs-SnapKV comparison at 10%. We will correct this and use consistent numbers from a single run in the revision.

SelKV still outperforms KVMerger (32.98) and WeightedKV (33.71) at 10%, so the advantage over other merging methods does widen under aggressive compression.

Error 2, Table 2 vs Table 6: These came from separate experimental runs, both using the code's default hyperparameters rather than the values specified in Section 4.1. This affected Tables 2 and 6; Tables 1 and 4 used correct settings and have been re-verified. We re-ran all LLaMA-3.1 configurations at 25% and all baselines from a single code checkpoint with correct settings (see our response to Reviewer fg8h Q2 and the 2×2 factorial above in Q6).

Error 3, KVMerger/WeightedKV at 10% not in tables: The numbers (32.98 and 33.71) are correct and exist in our result files but were inadvertently omitted from Tables 10/11. We will add them in the revision.

We have verified all reported numbers against the original result.json files and will provide a complete mapping from every table entry to its source experiment directory in a new appendix.

Reviewer_WE9xOpenReview ↗

Thanks for the detailed responses.

Q1–Q2: Clarification accepted, but the full attention matrix must be described in §3, with its memory cost and the fallback routing rule stated in the method. Q3: Still empirical only; the theoretical concern about unchanged keys is not addressed. Q4–Q5: +0.14 over the best binary gate, without a CI, is too small to establish the necessity of continuous gating or the advantage over D2O/EMS. Q6: Useful, but two of the four CIs include zero, so the two components are not cleanly separable. Q7: Helpful, though SelKV trailing SnapKV (and PyramidKV at 32k/64k) should be reflected in the claims. Q8: Errors acknowledged; all tables need to be regenerated from a single verified run.

The added results improve transparency but do not strengthen the case for the method. Score remains 2 (weak rejection).

Reviewer_fg8hOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposed a method for consolidating evicted tokens into retained cache entries. Unlike existing approaches merging every evicted token uniformly, this paper proposed SelKV, a training-free, one-shot post-prefill KV-cache compression framework with two mechanisms: (1) a soft cosine gate that modulates merge intensity per token based on value-vector similarity, continuously interpolating between full merge and drop, and (2) an attention-ratio compensation mechanism that corrects softmax imbalance using a bounded, prefill-derived logit bias in place of unstable count-based correction. Experimental results show SelKV is on-par with SOTA methods.

Critical strengths
  • Thorough experimental details are provided.
  • Anonymized code with fixed seed and config table is provided, which supports the reproducibility of the paper.
  • Ablation studies on the gate and compensation etc are conducted.
Critical weaknesses
  • The novelty is incremental. This paper provides a different angle and method for consolidating evicted tokens from existing methods, but it is unclear what are the benefits, how it is better than state-of-the-art methods and moves the field forward.
  • This paper didn’t show why evicted tokens are important to study, and the paper's own numbers argue against it. For example:
    • Eviction beats blind merging significantly: If evicted-token content were important, merging it shouldn't destroy value relative to dropping it.
    • If the retained information mattered, careful merging should significantly beat eviction. It doesn't.
  • There is no clear superiority over SOTA methods.
  • KeepKV and WeightedKV are not evaluated on Gemma-2. D2O is not evaluated - the stated reason (official implementation supports only LLaMA-2) does not explain its absence on LongChat-7B-v1.5-32k, which is LLaMA-2-based.
Questions for authors
  • Given LongChat-7B-v1.5-32k is LLaMA-2-based and D2O's official implementation supports LLaMA-2, why is D2O absent even there? 
  • Why Table 2 and Appendix D results don’t match, for example no gate result is different from the appendix D one (39.01 vs 38.96)?
4 comments

We appreciate the reviewer's recognition of the experimental details and reproducibility efforts.

CW1: Novelty / "why study evicted tokens?"

We agree that blind merging is worse than eviction, and that is one of the paper's findings (merge-all = 39.12 vs SnapKV = 39.77, p<0.001). The question is whether this means evicted tokens should always be discarded.

We argue no. With the cosine gate, SelKV outperforms eviction on 8 of 16 tasks (2 ties, 6 losses), including all three multi-document QA tasks where answers require synthesizing evidence from multiple locations (musique +0.66, hotpotqa +0.19, 2wikimqa +0.05). It also wins on few-shot tasks (trec +1.00, triviaqa +0.76) and single-document QA (multifieldqa_en +0.39, narrativeqa +0.18).

Eviction wins on 6 tasks, primarily code (lcc −0.52) where token identity is critical and averaging corrupts syntax. In aggregate, SelKV scores 39.90 vs SnapKV's 39.77 (+0.13), achieving parity with eviction while preserving information from compressed tokens. The aggregate gap is not statistically significant (p ≈ 0.2); the contribution is that merging matches eviction rather than beating it.

The reason blind merging fails is that it merges incompatible tokens indiscriminately, creating corrupted averages. The gate checks value-vector similarity per token and drops incompatible tokens (gate ≈ 0) while merging compatible ones (gate ≈ 1). This transforms merging from a risky strategy (−0.65 vs eviction) into a reliable one.

Additionally, SelKV outperforms all prior merging methods: KVMerger +1.70, WeightedKV +1.39, LOOK-M +1.19 (all p<0.001), and the eviction baseline H2O by +1.59. For practitioners who adopt merging for its information-preserving properties, SelKV is the best available option.

CW2: Missing baselines on Gemma-2

We provide comparisons on both models (16 LongBench datasets, 25% retention):

MethodLLaMA-3.1-8BGemma-2-9B
SelKV39.9042.31
SnapKV (eviction)39.7742.16
LOOK-M38.7141.18
D2O39.4842.13
KeepKV (count-based comp)38.9042.09
WeightedKV38.5140.91
H2O38.3141.20
KVMerger38.2040.70

On LLaMA-3.1, SelKV outperforms KVMerger by +1.70, WeightedKV by +1.39, H2O by +1.59, LOOK-M by +1.19, and KeepKV by +1.00. KVMerger was already reported on Gemma-2 (Table 3), where SelKV leads by +1.61 (p<0.001). Full per-task breakdown (LLaMA-3.1, 25% retention):

TaskSelKVKVMergerΔ
trec66.5059.50+7.00
qasper38.6233.11+5.51
multifieldqa_en50.2745.52+4.75
musique19.5516.65+2.90
hotpotqa39.9438.33+1.61
2wikimqa35.9134.44+1.47
repobench-p58.3156.99+1.32
triviaqa90.0089.04+0.96
samsum42.1941.27+0.92
narrativeqa21.5620.80+0.76
lcc64.1763.48+0.69
gov_report28.0027.54+0.46
passage_retrieval_en34.0034.00+0.00
passage_count4.034.03+0.00
qmsum21.3821.61−0.23
multi_news23.9124.89−0.98
AVERAGE39.9038.20+1.70

SelKV wins on 12 tasks (2 ties, 2 losses). The largest gains are on information-retrieval tasks (trec, qasper, multifieldqa_en, musique) where KVMerger's indiscriminate merging corrupts value representations.

Q1: D2O comparison

The reviewer correctly notes that D2O should be evaluated. We reimplemented D2O's core merging algorithm (key-cosine nearest-neighbor matching, mean-similarity threshold, similarity-weighted merging) within our evaluation framework. We chose reimplementation over the official code to isolate the merging strategy as the only variable: same token selection, same retention budget, same evaluation pipeline. Using the official D2O code would conflate differences in selection mechanism, hyperparameters, and evaluation setup with the merging strategy itself:

MethodLLaMA-2-7BLLaMA-3.1-8BGemma-2-9B
SelKV31.7839.9042.31
SnapKV (eviction)32.7739.7742.16
D2O31.8939.4842.13

On LLaMA-3.1, SelKV outperforms D2O by +0.42, winning on 13 of 16 tasks (paired bootstrap, one-sided p<0.001, 95% CI [+0.20, +0.63]). The largest gaps are on trec (+1.50), musique (+0.75), and samsum (+0.70). On Gemma-2, SelKV leads D2O by +0.18.

On LLaMA-2 (MHA architecture), D2O slightly outperforms SelKV (31.89 vs 31.78), though both trail SnapKV (32.77). This is consistent with the paper's finding that merging shows weaker results under MHA, where each head operates independently and the union-based selection strategy does not apply.

D2O differs from SelKV in two key respects: (1) D2O uses key-cosine similarity for its merge/drop decision, while SelKV uses value-cosine similarity (complementary to attention-based routing); (2) D2O lacks a compensation mechanism for attention redistribution. SelKV's advantage is most pronounced on multi-document QA (musique +0.75, 2wikimqa +0.55) and few-shot tasks (trec +1.50, triviaqa +0.68), where the gate and compensation together preserve distributed evidence more effectively.

Q2: Table 2 vs Appendix D number discrepancy

Good catch. The discrepancy arises from two issues:

  1. Tables 2 and 6 come from separate experimental runs.
  2. Both runs used the code's default hyperparameters rather than the values specified in Section 4.1 (including the head-reduction rule and other settings). We discovered this mismatch while preparing the rebuttal.

The hyperparameter mismatch affected Tables 2 and 6 (LLaMA-3.1 ablation numbers). Tables 1 and 4 (cross-model comparisons) used the correct settings and have been re-verified. Tables 10–12 (retention-ratio sweeps) used the original defaults and will be footnoted in the revision.

We have now re-run all LLaMA-3.1 configurations at 25% retention with the correct hyperparameters (W=32, K=5, B=32, m_r=16, union for GQA) from a single code checkpoint. The unified numbers are: no-gate = 39.12, gate-only = 39.69, comp-only = 39.51, gate+comp = 39.90 (see our 2×2 factorial in the response to Reviewer WE9x Q6). All baseline methods (SnapKV, KVMerger, H2O, LOOK-M, WeightedKV) were also re-run in this unified setting; small shifts (e.g., SnapKV 39.79→39.77) reflect the corrected hyperparameters. The relative ordering and effect sizes are consistent with the originals. All numbers in the revision will come from this single corrected run, with a table mapping each entry to its source experiment directory.

Thank the authors for the thorough response and the newly added experiments. Just, I still have concerns regarding the contribution of the proposed method. This paper provides a different angle and method for consolidating evicted tokens from existing methods, but it is unclear what are the theoretical or practical benefits consolidating evicted tokens, as the performance is on par and there is no cost advantage. I will keep my rating at 2.

We thank the reviewer for engaging with the new experiments and for the specific feedback.

On the question of what benefit consolidation provides: the starting observation is that blind merging loses to eviction by 0.65 points (p<0.001), and all prior merging methods sit in that deficit. SelKV closes the gap entirely, so practitioners who want to preserve information from compressed tokens (rather than discard them) can now do so without paying a quality penalty. The practical benefit is not that SelKV beats eviction on average, but that merging is no longer a losing strategy.

Where the benefit shows up concretely is at the task level. SelKV wins 8 of 16 tasks, including all three multi-document QA benchmarks (musique +0.66, hotpotqa +0.19, 2wikimqa +0.05), where synthesizing evidence across locations is precisely the scenario where retaining evicted-token information should help. Eviction wins on 6 tasks (code, retrieval) where token identity matters more than coverage.

Regarding cost: SelKV adds 28ms per layer at 25% retention (Table 7), which is 1.3% of the full-cache baseline. The overhead comes from the gate and compensation, which are both elementwise operations on the already-computed attention. The cost is comparable, not prohibitive.

We have revised the paper's claims to reflect this framing accurately. We understand the reviewer may still view the contribution as incremental, and we appreciate the careful reading.

Reviewer_8odeOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

SelKV is a training-free KV-cache compression method that selectively merges or drops each evicted token. It uses value-vector cosine similarity as a soft merge gate and introduces attention-ratio compensation to avoid the instability of raw merge-count correction. Across LongBench, RULER, and latency experiments on three models, SelKV substantially improves over blind merging methods and reaches statistical parity with strong eviction.

Critical strengths
  • The central experiment is unusually well controlled: token selection and the code path are held fixed while the gate and compensation are varied, and paired bootstrap confidence intervals are reported. This directly supports the paper’s main causal claim.

  • The claims are appropriately scoped. The authors explicitly show that SelKV reaches parity with strong eviction rather than claiming superiority, while demonstrating clear and statistically significant gains over existing merging approaches.

Critical weaknesses
  • SelKV does not establish whether merging offers an advantage over eviction in the regime where long-context preservation matters most. The strongest long-context quality experiment, RULER at 32k–64k, compares only against the full cache and omits strong eviction baselines.

  • Broader models, longer decoding, and quality measurement at the latency endpoint are needed for a stronger deployment claim. The 3.3× speed result is measured at 100k tokens, while generation quality is validated only up to 64k and for generations of at most 500 tokens. Is quality preserved at the 100k-token operating point used for the latency result?

Questions for authors
  • How much overhead is introduced during prefill and compression? The decode-time speedup is reported, but SelKV also computes observation-window attention, routing, value cosine similarities, merging, and compensation statistics. Please report end-to-end latency or separately report prefill/compression overhead, especially for shorter generations where preprocessing cost may be important.

  • The results suggest that SelKV is most useful for distributed-evidence QA but weaker for code and the tested MHA model. Can the authors clarify the conditions under which SelKV should be preferred over eviction?

  • Why should we use merging base methods if eviction methods are better?How does SelKV compare with strong eviction baselines on RULER at 32k and 64k? Please report or discuss comparisons against SnapKV, PyramidKV or another strong eviction baseline under the same retention ratio and implementation conditions.

2 comments

We appreciate the reviewer's careful evaluation.

1: RULER lacks eviction baselines

Agreed, the RULER evaluation should include eviction baselines. We ran SnapKV under identical conditions (25% retention, same model, same RULER tasks):

Method4k32k64k
Full Cache92.184.481.2
SnapKV (eviction)80.782.278.2
PyramidKV (eviction)78.282.679.9
SelKV (gate+comp)79.980.577.0
Gate only79.975.377.1
Merge-all (no gate)79.675.475.7
H2O40.677.975.8
LOOK-M28.170.672.0

Eviction methods (SnapKV, PyramidKV) lead SelKV on RULER by 0.8–2.9 points, consistent with our LongBench finding that SelKV achieves parity with eviction in aggregate, not superiority. RULER's tasks are predominantly needle-retrieval, where exact token identity matters more than information aggregation.

That said, SelKV clearly outperforms all other merging methods: LOOK-M collapses at 4k (28.1) and H2O at 4k (40.6). Compensation drives the largest gain: at 32k, gate-only (75.3) ≈ merge-all (75.4), but adding compensation yields +5.2 (75.3→80.5). The gate contributes at 64k (+1.4: merge-all 75.7→gate-only 77.1). The components' contributions vary by context length rather than accumulating uniformly.

Note: RULER 4k scores differ from the original submission (Full Cache 92.4→92.1, SelKV 80.5→79.9) due to the unified re-run with Section 4.1 hyperparameters. H2O (40.6) and LOOK-M (28.1) collapsing at 4k but recovering at longer lengths reflects their reliance on accumulated attention statistics, which are sparse at 4k.

The case for SelKV on RULER rests on its advantage over other merging methods, not over eviction. We will add this nuance to the revision.

2: Quality at 100k tokens; broader models

We did run SelKV at 100k tokens for the latency experiment (3.3× decode speedup). Our initial RULER evaluation covered 32k and 64k, where SelKV retains ≈95% of full-cache quality (80.5/84.4 at 32k, 77.0/81.2 at 64k). We will extend to 128k in the revision.

Regarding broader models: we evaluate on 3 architectures (MHA via LongChat-7B, GQA via LLaMA-3.1-8B and Gemma-2-9B) spanning 7B–9B parameters. Extending to 70B would strengthen the paper but is beyond our current compute budget.

Q1: Prefill/compression overhead

End-to-end timing breakdown (LLaMA-3.1-8B-Instruct, H100 80GB, sequence length 4096, 25% retention):

PhaseSelKVSnapKVH2OFull Cache
Prefill (ms)175174174142
Compress (ms)27411N/A
Decode (128 tokens, ms)1949196319301982
Total (ms)2151214121142123

All compression methods use eager attention during prefill to obtain the attention matrix needed for token selection and routing, adding ~33ms over Full Cache's SDPA-only pass (175ms vs 142ms). SelKV's compression step (27ms) is 23ms more than SnapKV (4ms) due to the cosine gate and bucket-attention routing, but amortized over 128 decode tokens this is <0.2ms per generated token. Decode is faster after compression (1949ms vs 1982ms) because the reduced cache (25% retention = 1024 tokens) requires less memory bandwidth per step. The net overhead of SelKV over Full Cache is 28ms total (1.3%), which vanishes for longer generations. For comparison, LOOK-M's merging step costs 105ms (4× SelKV), so SelKV's bucket-based routing is considerably cheaper than pairwise merging.

Q2: When to prefer SelKV over eviction

SelKV should be preferred when:

  1. Information preservation matters: On distributed-evidence QA tasks (hotpotqa, 2wikimqa, musique), where answers are scattered across the context, SelKV consistently outperforms eviction-only methods. Eviction permanently discards these tokens; selective merging preserves compatible information.

  2. Aggressive compression is needed: At 25% retention, merging recovers information that eviction discards. At 10% retention, SnapKV leads (38.06 vs 37.56), and at 50% the gap narrows (40.34 vs 40.29).

  3. Safe defaults are desired: Without the cosine gate, blind merging causes severe per-task failures (trec −3.50, gov_report −1.38, triviaqa −1.20 vs eviction). The gate limits the worst per-task delta to −0.52 (lcc), making merging a reliable option.

SelKV should NOT be preferred when token identity is critical (code generation) or under MHA architectures where the method shows weaker results.

Q3: RULER comparison with eviction baselines

See CW1 above.

Thank you for the additional experiments and clarifications. The long-context eviction comparisons and timing breakdown address part of my concerns about evaluation completeness and computational overhead. Although the practical advantage over strong eviction and quality at the 100k-token latency setting remain insufficiently established, I find the controlled analysis of selective merging and compensation informative. With the clarified scope and limitations, I am raising my rating from 2 (Weak Reject) to 3 (Weak Accept).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.