- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes DisenKV, a training-free KV-cache eviction method designed to address potential biases arising from shared directional structures among observation queries.
The authors observe that recent observation queries exhibit substantial directional similarity. When multiple queries contain similar shared components, conventional attention-based importance estimation may repeatedly reinforce certain historical tokens while underrepresenting distinctive evidence expressed by fewer queries. The authors refer to this phenomenon as common-demand bias.
To address this issue, DisenKV introduces a leave-one-out (LOO) query residualization mechanism that estimates a shared direction from the remaining observation queries and removes its projection from each query. The resulting query-specific residuals provide the primary eviction-risk evidence, while the original queries preserve broadly relevant contextual information and contribute complementary evidence through asymmetric late fusion.
The method is evaluated across four model families ranging from 3.8B to 24B parameters, covering 16 LongBench tasks and additional RULER evaluations. At 20% KV-cache retention, DisenKV consistently improves over DefensiveKV across all four evaluated models. The authors further investigate tighter retention budgets, layer-wise allocation, observation-window sensitivity, component ablations, cross-method transferability, controlled counterfactual interventions, and inference efficiency.
Overall, this paper presents an interesting perspective on KV-cache retention by examining the dependence structure of the observation evidence itself. The proposed approach is lightweight and empirically effective. However, the relationship between shared query directions and semantically redundant evidence, as well as the causal connection between changes in retention scoring and downstream accuracy improvements, requires further clarification.
Critical strengths
This paper proposes DisenKV, a training-free KV-cache eviction method designed to address potential biases arising from shared directional structures among observation queries.
The authors observe that recent observation queries exhibit substantial directional similarity. When multiple queries contain similar shared components, conventional attention-based importance estimation may repeatedly reinforce certain historical tokens while underrepresenting distinctive evidence expressed by fewer queries. The authors refer to this phenomenon as common-demand bias.
To address this issue, DisenKV introduces a leave-one-out (LOO) query residualization mechanism that estimates a shared direction from the remaining observation queries and removes its projection from each query. The resulting query-specific residuals provide the primary eviction-risk evidence, while the original queries preserve broadly relevant contextual information and contribute complementary evidence through asymmetric late fusion.
The method is evaluated across four model families ranging from 3.8B to 24B parameters, covering 16 LongBench tasks and additional RULER evaluations. At 20% KV-cache retention, DisenKV consistently improves over DefensiveKV across all four evaluated models. The authors further investigate tighter retention budgets, layer-wise allocation, observation-window sensitivity, component ablations, cross-method transferability, controlled counterfactual interventions, and inference efficiency.
Overall, this paper presents an interesting perspective on KV-cache retention by examining the dependence structure of the observation evidence itself. The proposed approach is lightweight and empirically effective. However, the relationship between shared query directions and semantically redundant evidence, as well as the causal connection between changes in retention scoring and downstream accuracy improvements, requires further clarification.
Critical weaknesses
W1. The semantic relationship between shared query structure and common-demand bias remains insufficiently established.
The central motivation of this paper is based on the observation that a substantial fraction of observation-query energy aligns with a shared direction estimated from neighboring queries.
However, strong geometric alignment does not necessarily imply semantically redundant or undesirable retention evidence. Shared directional components may also arise from representation anisotropy, biases introduced by query projections, or genuinely useful contextual information repeatedly requested by the task.
Although the authors appropriately acknowledge that geometric alignment is not equivalent to semantic relevance, the paper does not sufficiently establish which shared components actually contribute to incorrect token eviction.
I would encourage the authors to provide additional null controls and investigate the relationship between shared-component strength, retention errors, and downstream generation failures.
W2. The counterfactual experiments do not fully establish a causal connection between scoring bias and downstream performance.
The controlled interventions in Appendix E.6 demonstrate that increasing shared-component strength changes retention-score dominance under raw scoring, while residualization suppresses this response.
This supports the claim that query residualization changes the sensitivity of importance estimation to shared query structure.
However, the interventions are primarily evaluated through retention scores and ranking statistics. They are not propagated through actual KV-cache eviction and subsequent generation to directly evaluate downstream accuracy.
Consequently, the current results do not establish to what extent suppressing common-demand dominance directly reduces the probability of evicting task-relevant tokens.
A stronger evaluation would apply the same interventions to the complete eviction and generation pipeline while measuring relevant-token survival and downstream accuracy.
W3. Comparisons with simpler common-component removal methods are still insufficient.
The paper includes full-window mean and norm-matched raw-query controls, which are useful. However, it remains unclear whether the proposed LOO residualization mechanism provides a substantial advantage over more general representation-centering or decorrelation techniques.
For example, direct query mean-centering, PCA-based dominant-direction removal, and suitable random-direction projection controls would help establish whether the observed improvements specifically arise from the proposed LOO shared-direction estimation or from a more general query debiasing effect.
This distinction is important for assessing the algorithmic novelty of the method, since projection-based removal of common representation components has been extensively studied in prior work.
W4. Generalization across task types and more diverse inference settings requires further validation.
Although LongBench evaluations cover four model families, experiments involving more aggressive retention budgets, RULER, and most mechanism ablations are primarily conducted on Llama-3.1-8B.
Moreover, the performance regression on the RULER CWE task indicates that suppressing shared query evidence can be harmful when repeated or distributed information is genuinely important for the downstream task.
This suggests that query residualization may introduce a tradeoff between preserving distinctive evidence and retaining repeated task-relevant information.
A more systematic analysis of different task categories, particularly those requiring frequency aggregation or distributed evidence, would strengthen the generality of the proposed interpretation.
W5. Statistical robustness and the independent contribution of layer-wise allocation require additional analysis.
Several reported component-level improvements are relatively small. For example, the difference between LOO estimation and the full-window mean is approximately 0.41 points, while the contribution of late correction is approximately 0.32 points on the reported LongBench average.
Without sufficient uncertainty estimates or repeated experiments, it is difficult to determine the robustness of these relatively small differences.
Furthermore, although Layer-DisenKV and Layer-DefensiveKV share the same global allocation rule and total cache budget, changes in importance scores can still alter the resulting allocation across layers.
A comparison with fixed per-layer budgets would help distinguish improvements arising directly from query evidence construction from those associated with changes in layer-wise capacity distribution.
Questions for authors
Q1. How can the authors distinguish semantically redundant shared query components from genuinely useful shared task demand?
Could the authors quantify the relationship between shared-component energy and actual token-eviction errors?
In particular, it would be valuable to analyze whether examples with stronger shared query structure exhibit higher rates of incorrect eviction under raw scoring, and whether residualization preferentially preserves gold-answer-supporting tokens.
Q2. Can the authors provide end-to-end counterfactual interventions connecting scoring bias to downstream performance?
Appendix E.6 demonstrates that manipulating shared-component strength changes retention-score behavior.
Could the authors extend these interventions to actual KV-cache eviction and subsequent generation, reporting gold-token survival rates, attention-output perturbations, and final task accuracy?
Such experiments would help establish whether suppressing common-demand dominance causally improves downstream performance rather than merely modifying scoring statistics.
Q3. Why is LOO projection preferable to simpler query-centering or dominant-direction removal methods?
Although the comparison with full-window mean estimation supports the usefulness of LOO, its advantage over alternative common-component removal techniques remains unclear.
Could the authors compare LOO residualization against direct query mean-centering, PCA-based dominant-component removal, and appropriate random-direction controls under identical scoring and retention budgets?
These experiments would help clarify which aspects of the proposed evidence disentanglement are essential.
Q4. How can the method avoid suppressing useful repeated evidence in tasks requiring distributed information aggregation?
The performance decrease on the RULER CWE task suggests that shared demand can sometimes carry genuinely important task information.
Could the authors further investigate the differences between CWE, multi-document QA, and retrieval-oriented tasks in terms of query geometry and relevant-token survival?
Would it be possible to adapt the residualization strength or raw-residual fusion coefficient based on the structure of the observation queries?
Q5. Can the authors provide additional evidence of statistical robustness and cross-model generalization?
It would be helpful to report results across multiple random seeds and reproduce the key norm-matched, LOO, and counterfactual experiments on at least one additional model family.
For the layer-wise variant, a fixed per-layer budget comparison would also help isolate improvements from query evidence construction versus changes in budget redistribution.
6 comments
Response to BRkb (2/5) — End-to-End Shared-Component Intervention
Q2. Does intervening on the shared component affect actual KV eviction and generation?
(1) Controlled interventions establish an effect on retention scores. We extended the intervention to actual KV eviction and answer generation on 200 HotpotQA examples.
(2) Both methods have lower mean F1 point estimates at the higher shared-component strength. Both 95% paired confidence intervals span zero.
In Appendix E.6, we increased the shared-component multiplier from to while holding query norms fixed. The retention-risk gap between high- and low-support tokens increased from 0.50 to 3.26 under raw-query scoring and decreased from 0.73 to 0.19 under residual scoring. Residualization reduced the response slope on all 16 LongBench tasks.
For the end-to-end experiment, we used Llama-3.1-8B at 20% KV retention and increased from to only in the primary risk-scoring branch while holding query norms fixed. We then performed KV eviction and generated answers. At , both methods reproduced their corresponding default retained indices and answers on five validation examples.
| Method | Supporting-passage KV-slot retention | Answer F1 | F1 change (95% paired CI) |
|---|---|---|---|
| Matched raw-only | 20.09% → 20.15% | 55.75 → 51.10 | −4.65 [−9.73, 0.38] |
| DisenKV | 19.84% → 19.85% | 57.42 → 54.66 | −2.76 [−6.40, 0.76] |
Across examples, the Spearman correlations between changes in supporting-passage retention and F1 were −0.075 for matched raw-only (95% CI [−0.220, 0.065]) and −0.057 for DisenKV (95% CI [−0.179, 0.066]). This experiment measures evidence retention and answer F1 after eviction. Intermediate attention-output changes remain unquantified.
The controlled scoring result establishes that residualization changes sensitivity to shared demand. The end-to-end results leave open whether the methods differ in their F1 response and how much of that response can be attributed to aggregate evidence retention.
Response to BRkb (3/5) — LOO, Mean-Centering, PCA, and Random Projection
Q3. How does LOO compare with mean-centering, PCA, and random projection?
(1) Single-direction LOO achieved the highest macro-average in this comparison. It scored 48.08, compared with 47.70 for mean-centering, 46.31 for rank-1 PCA, 46.22 for rank-2 PCA, and 47.01 for random projection.
(2) The leave-one-out direction is estimated separately for each query. Its residual supplies the primary risk score, and the original query supplies coverage protection and one-sided correction.
For each query , LOO averages and normalizes the other observation queries and removes the projection of onto the resulting direction. This prevents the query being scored from contributing to its own direction estimate. Estimating the direction requires a mean and normalization.
We fixed Llama-3.1-8B, 20% KV retention, and the rest of the eviction pipeline. Only the query transformation in the primary risk branch changed. Mean-centering subtracts the window mean. The PCA controls fit a centered one- or two-dimensional subspace using the other queries. The random-direction control uses one fixed seed.
| Transformation | NrtvQA | Qasper | MF-en | Hotpot | 2WikiMQA | Musique | GovRep | QMSum | MultiNews | TREC | TriviaQA | SAMSum | PCount | PR-en | Lcc | RB-P | Avg. |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Single-direction LOO | 29.86 | 41.57 | 51.20 | 56.97 | 41.50 | 28.77 | 31.57 | 23.90 | 24.46 | 68.00 | 91.94 | 43.58 | 7.75 | 99.50 | 69.44 | 59.23 | 48.08 |
| PCA rank 1 | 27.69 | 37.72 | 48.28 | 50.99 | 38.44 | 28.37 | 28.75 | 23.29 | 23.97 | 66.00 | 92.08 | 44.03 | 5.11 | 99.50 | 68.24 | 58.43 | 46.31 |
| PCA rank 2 | 28.21 | 36.24 | 46.17 | 52.05 | 35.63 | 29.19 | 28.43 | 23.08 | 24.34 | 66.50 | 91.92 | 43.98 | 7.98 | 100.00 | 68.11 | 57.76 | 46.22 |
| Mean-centering | 30.46 | 43.42 | 49.76 | 56.41 | 41.82 | 28.81 | 28.13 | 24.38 | 24.11 | 69.00 | 91.66 | 43.44 | 5.87 | 99.50 | 68.29 | 58.19 | 47.70 |
| Random-direction projection | 29.40 | 40.27 | 49.66 | 52.48 | 40.99 | 28.78 | 28.56 | 23.14 | 24.83 | 66.00 | 92.08 | 43.59 | 7.19 | 99.50 | 67.92 | 57.83 | 47.01 |
LOO's macro-average advantages over mean-centering, rank-1 PCA, rank-2 PCA, and random projection are +0.38, +1.77, +1.86, and +1.07 points, respectively. These are point estimates for the matched configuration. The comparison supports using a single leave-one-out direction in this setting, with the random-direction result scoped to the tested seed.
Response to BRkb (1/5) — Shared Query Structure and Evidence Eviction
Q1. Does shared query structure reflect redundant evidence or useful shared demand?
(1) DisenKV improved coverage of the least-retained gold supporting passage on HotpotQA by 0.78 percentage points on average (95% paired CI [0.58, 0.97]).
(2) Shared-component energy showed little per-example correlation with evidence eviction in either diagnostic. Query alignment alone is therefore insufficient to determine whether the shared evidence is semantically redundant.
We used Llama-3.1-8B at 20% KV retention and measured evidence eviction under matched raw-only scoring. Evidence is defined by gold supporting passages on HotpotQA and target-word occurrences on RULER CWE.
In the default HotpotQA setting, DisenKV changes aggregate supporting-passage KV-slot retention by −0.29 percentage points (95% paired CI [−0.68, 0.07]) relative to matched raw-only. Taking the minimum retention rate among each example's supporting passages and then averaging across examples yields a gain of +0.78 percentage points. These metrics distinguish aggregate retention from coverage of the least-retained supporting passage.
On CWE, target-word KV-slot retention falls from 25.89% with DefensiveKV to 24.37% with DisenKV. This result is consistent with useful shared demand in tasks that require repeated evidence. DisenKV keeps the original queries available for coverage protection and one-sided correction to accommodate such demand. The diagnostics show a change in how evidence is retained, while the relationship between shared energy and individual eviction errors remains unresolved.