- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
Summary
This paper proposes RelaxKV, a training-free method for selectively recomputing independently cached document chunks in RAG. It uses layer-wise query attention to select repair anchors, derives active recomputation paths from each token's deepest selected layer, and restricts recomputation attention to the causal subset of the anchor union. Experiments across four models and several long-context benchmarks show competitive quality–latency trade-offs.
Critical strengths
Strengths
- The method is clearly motivated and described. Distinguishing repair targets from the context available during recomputation provides a useful perspective on selective KV cache repair.
- The evaluation is reasonably comprehensive, including multiple models, natural and synthetic tasks, quality–TTFT curves, and sensitivity to anchor ratio and chunk size.
- The controlled experiments are valuable. RelaxKV-G, the context-construction ablations, and the full-prefix control help separate the effects of repair-target selection and recomputation context.
Critical weaknesses
Weaknesses
-
Missing comparison with InfoFlow KV. InfoFlow KV is directly relevant as it also uses query-conditioned attention to select tokens for KV recomputation and uses reordering to modify the recomputation scope. A direct comparison under a unified evaluation protocol is needed to establish the incremental benefits of RelaxKV's layer-specific repair schedule and sparse recomputation context. Without this comparison, the reported quality–latency trade-off is not fully positioned against closely related work.
-
The repair schedule needs stronger validation. Recomputing each token from the input through its deepest selected layer and stopping thereafter is intuitive, but layer-wise attention importance alone does not establish that this is the most effective schedule. Alternative schedules should be compared, particularly recomputing the entire anchor union through all layers. The appendix's Target-Union configuration matches the target count rather than directly using the anchor union as the repair set, so it does not fully address this question. Such comparisons would clarify the benefit of the proposed stopping rule.
-
The denoising interpretation lacks direct evidence. The existing controls support the effectiveness of context restriction, but do not directly establish the suggested implicit denoising effect. Restricting the context can remove distracting information, but can also discard useful within-chunk dependencies that were present in the original cache. Case studies and failure analyses would help explain when sparse recomputation improves the representations relevant to answering the query and when it loses necessary information.
-
Generality across token selection strategies remains unclear. RelaxKV-G is a useful control using ProphetKV's repair targets, but both target-selection approaches rely on query attention. Evaluating sparse recomputation with selectors based on different criteria, such as those used by CacheBlend and Lego, would clarify whether its benefits extend beyond query-guided selection and whether the recomputation strategy can serve as a more general component.
Questions for authors
Questions
-
Could the authors compare against InfoFlow KV under the same evaluation protocol and report the quality–latency trade-off?
-
How does the proposed repair schedule compare with alternative schedules, particularly recomputing all tokens in the anchor union through all layers?
-
Could the authors provide case studies or failure analyses to clarify the proposed denoising effect and the potential loss of useful within-chunk context?
-
Does the sparse recomputation mechanism also benefit different token selection strategies, such as CacheBlend and Lego?
-
Beyond the current RULER-MV evaluation up to 32K, could the authors evaluate longer contexts within the models' supported ranges and examine sensitivity to evidence position?
7 comments
Response (Part 2)
W2&Q2:
We will add theoretical elaboration and analysis on the cost of RelaxKV compared with alternative schedules in the appendix. Thank you for the questions raised by the reviewers.
The all-layer union schedule would recompute the union of anchors at every layer, causing the total repair cost to grow approximately as , where is the number of layers.
This can be substantially larger than RelaxKV’s layer-wise cost , making it impractical under a fixed recomputation budget. RelaxKV therefore uses layer-specific repair sets to maintain a controlled repair cost while adapting the schedule to each layer’s query relevance.
We further validate this analysis experimentally, as shown in Figure 9 of Appendix A.7. On RULER-MV 16K, RelaxKV achieves a 100% score with approximately 3.5s TTFT, whereas the all-layer anchor union requires approximately 4.9s to reach the same score, reducing latency by about 28%. These results demonstrate the superior quality–latency trade-off of layer-wise recomputation scheduling.
Response (Part 3)
W3&Q3:
To investigate the proposed denoising effect and the potential loss of useful within-chunk context, we conduct an additional evidence-preservation analysis and provide representative success and failure cases.
(1) Evidence preservation. We map RelaxKV's selected context union to 512-token chunks on RULER-MV, covering context lengths from 4K to 32K with evidence distributed across 1, 2, or 4 chunks based on the [1]. At a 15% anchor ratio, RelaxKV achieves 100% evidence-chunk coverage across all evaluated settings, retaining 98.20%–99.96% of tokens within evidence-bearing chunks. Even when evidence spans four chunks, token retention remains above 99%.
| Metric | Evidence Chunks | 4K | 8K | 16K | 32K |
|---|---|---|---|---|---|
| Evidence Chunk Coverage (%) | 1 | 100 | 100 | 100 | 100 |
| 2 | 100 | 100 | 100 | 100 | |
| 4 | 100 | 100 | 100 | 100 | |
| Evidence Token Retention (%) | 1 | 99.18 | 99.74 | 99.62 | 98.20 |
| 2 | 99.86 | 99.96 | 99.91 | 99.87 | |
| 4 | 99.91 | 99.91 | 99.93 | 99.93 |
-
Success case. On MuSiQue, answering the question requires connecting the evidence chain INSAT-4CR → ISRO → ATS-6. RelaxKV preserves the relevant evidence across both supporting passages and correctly generates “ATS-6”, matching full recomputation while retaining only 64.1% of the context tokens in its selected context union.
-
Failure case. In another MuSiQue example, RelaxKV incorrectly generates “James Chadwick” instead of “Manhattan Project”, although the complete evidence span “worked on the Manhattan Project” remains in the selected union and full recomputation answers correctly. This indicates that preserving the answer-bearing evidence alone does not guarantee correct answer generation.
(2) Discussion and limitations. These results demonstrate that RelaxKV can substantially restrict the recomputation context while preserving evidence-bearing regions, but also reveal that evidence retention does not necessarily ensure effective evidence utilization. The observed quality improvements are consistent with reduced interference from irrelevant context, although the cases do not establish denoising as a causal mechanism. We will incorporate these analyses into the revised manuscript and clarify the potential limitations of context restriction.
Response (Part 1)
W1&Q1: Thank you for the helpful suggestion.
In the revision, we cite InfoFlow-KV [1] and add a comparison under the same evaluation protocol with MuSiQue and Qwen3-14B. We report both answer quality and latency to provide a direct quality–latency comparison with this baseline, with the results presented in Figure 6 of the revised PDF.
[1] https://arxiv.org/pdf/2603.05353v3. InfoFlow KV: Information-Flow-Aware KV Recomputation for Long Context.