- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper studies efficient KV-cache reuse for long-context question answering over shared corpora. The challenge is that independently cached document blocks can be reused efficiently across queries, but their KV states lack the cross-block contextual information available when the retrieved blocks are jointly encoded. The paper proposes PRISM, a training-free framework for efficient KV-cache reuse. Experiments show that PRISM improves accuracy over existing KV-cache reuse methods while achieving substantially lower time-to-first-token (TTFT) than full re-encoding.
Critical strengths
-
Efficient KV-cache reuse is an important problem for long-context LLM applications. The paper addresses a meaningful tradeoff between computational efficiency and reasoning accuracy.
-
The C4 framework provides a useful perspective for analyzing cache-reuse methods, although the individual factors are largely known.
-
Extensive empirical evaluation across models and tasks was conducted. The reported efficiency improvements are substantial.
Critical weaknesses
-
The C4 framework is presented as a general characterization of the factors governing cache-reuse quality. However, the 4 factors are primarily motivated and validated through empirical observations. The 4 factors may not represent fundamentally independent mechanisms. The framework is essentially qualitative. It does not provide quantitative measures of 4 four factors or a predictive model connecting them to reasoning accuracy.
-
The technical novelty looks limited. First, the main technical components build heavily on existing techniques. The contribution of the paper appears to lie in the integration of existing techniques, rather than in a fundamentally new caching or contextualization mechanism.
-
The main technical content (Section 4) looks somewhat incomplete. The entire technical framework and many important technical details are missing. I think PRISM deserves a substantially more rigorous presentation. In particular, the conditional attention-based extraction procedure is not sufficiently specified. It is unclear how attention scores are computed between the conditioning context and independently cached blocks, how previously extracted spans are represented during subsequent extraction steps, and what additional computation is required.
Questions for authors
-
What is the essential technical innovation distinguishing PRISM from a straightforward combination of evidence extraction and selective KV recomputation?
-
How to determine whether the selected spans contain all information necessary for cross-block reasoning?
-
How does the method behave when the reasoning process requires multiple rounds of information exchange between blocks?
6 comments
W1:
The factors in C4 come from controlled experiments that change one cache property at a time (Section 3). Each factor is a measurable property of the cache (Figure 2 now reports its accuracy cost on six models), which tells us what a reusable cache must preserve and led to PRISM's design. We will make this clear in Section 3, and explore predicting accuracy from these measurements in future work.
W3:
We will make Section 4 clearer in the revision (see also Algorithm 1 in Appendix A).
For each retrieved block in order, the extraction context is (Algorithm 1): the instruction is already cached, and the question and earlier spans are added as plain text (Figure 7). We run one forward pass of these tokens over the cached KV of and only, and score each 10-token chunk of by the QRHead attention it receives from them, minus the attention it receives from the "Question:" prefix (following Chen et al., 2025; Zhang et al., 2025,). Chunks scoring above () are expanded to sentence/paragraph boundaries to form .
Because is already cached, extraction only adds one forward pass over per block: no tokens are generated and no new KV states are saved. This cost is included in the TTFT we measure, which is 2.2–6.8× lower than Re-encode's on LongMemEval and CLIPPER.
Q3:
PRISM extracts spans in a single pass over the retrieved blocks, so spans from earlier blocks guide selection in later ones. This handles multi-step reasoning that follows the block order (as in the 3-hop chains and multi-session questions above). To see what happens when links go backward or jump back and forth across blocks, we test chain traversal under reversed and shuffled block orders (500-token blocks, n=100):
| Model | Order | Hops | Re-encode | BlockSink | PRISM |
|---|---|---|---|---|---|
| Llama-3.1-8B | forward | 2/3 | 0.98/0.93 | 0.58/0.24 | 0.98/0.65 |
| Llama-3.1-8B | reversed | 2/3 | 0.87/0.71 | 0.59/0.35 | 0.83/0.53 |
| Llama-3.1-8B | shuffled | 3 | 0.84 | 0.36 | 0.49 |
| Qwen3-8B | forward | 2/3 | 0.99/0.97 | 0.81/0.65 | 0.98/0.92 |
| Qwen3-8B | reversed | 2/3 | 1.00/0.96 | 0.70/0.49 | 0.95/0.57 |
| Qwen3-8B | shuffled | 3 | 0.97 | 0.53 | 0.77 |
PRISM still beats BlockSink in every setting, but accuracy drops on reversed and shuffled chains, as expected from a single pass. Extending extraction to multiple passes, so that spans found in later blocks can also guide earlier ones, is a natural next step.
Dear Reviewer,
With less than 24 hours remaining in the discussion period, we would appreciate your feedback on our responses, particularly on what PRISM adds beyond combining evidence extraction with selective recomputation, the span-completeness analysis, and the clarified extraction procedure. If these address your concerns, we kindly ask you to consider raising your rating. We would also be happy to discuss any remaining questions.
Thank you for your time and consideration.
We thank the reviewer for finding the C4 framework a useful perspective. To clarify PRISM's contribution, we compared it with other methods that use query attention to select tokens for recomputation, and performed a more focused analysis to isolate the impact of iterative extraction. TL;DR from the additional experiments: PRISM's accuracy is higher by 14 to 56 points compared to all existing methods on the synthetic chain traversal task, where iterative extraction is important. Under matched budgets, PRISM is the best in 5 out of 6 settings on the real LongMemEval task. More details follow.
W2, Q1, Q2:
While attention-based selection and selective KV recomputation exist separately, a simple combination falls short across blocks. The closest "extraction + selective recomputation" methods are InfoFlow KV (2603.05353, ICML 2026) and ProphetKV (2602.02579, ICML 2026): like PRISM, they use query attention to select tokens, but they score tokens in a single pass over the assembled cache. PRISM instead scores each block's tokens conditioned on both the question and spans already selected from earlier blocks, replacing global top-k with per-block thresholds.
To see why conditioning across blocks matters, we first test on the chain traversal task, where chain links are spread across blocks and the model must follow them to the final destination (averaged over 500-, 1000-, and 2000-token blocks, n=100 per setting):
PRISM outperforms InfoFlow and ProphetKV in every setting at matched budget and comes closest to Re-encode. Scoring each block's tokens independently (PRISM, indep.) drops accuracy sharply, so spans from earlier blocks help select the right tokens in later ones.
On the full LongMemEval set, with new runs at PRISM's per-question budget:
PRISM leads in 5 of 6 settings across models and retrievers. On multi-session questions needing facts from several sessions, iterative extraction is also ahead of independent scoring (0.476 vs 0.453 over five models, p=0.010).
To check that these gains come from better KV rewriting, we take LongMemEval samples that Re-encode answers correctly and measure the negative log-likelihood (NLL) of Re-encode's answer under each method. At matched budgets, we report the share of the NLL gap between BlockSink (0% recomputed) and Re-encode (100%) each method closes:
Random tokens barely help, so what matters is which tokens are rewritten, and PRISM's selection closes the most.
To see the impact of span completeness (Q2), we compare PRISM with answering from the extracted spans alone (OIE) on LongMemEval (oracle), using Gemini 3.1 Pro to label whether the spans are complete:
OIE performs well when spans are complete, but drops sharply when a fact is missing. PRISM is much more robust to incomplete spans (0.54–0.64 vs. 0.22–0.44), even on Qwen3-8B where only 22% of questions have complete spans. Since complete spans cannot be guaranteed, this robustness matters: over all questions, PRISM is ahead of OIE in 4 of 5 models.
> Together, these results show that neither half is enough alone: scoring the cache in one pass (InfoFlow KV, ProphetKV) picks less useful tokens, and answering from the spans alone (OIE) fails when a fact is missed. PRISM instead selects tokens iteratively across blocks and rewrites their KV in-place.