acceptodds
Under review as a conference paper at ICLR 2027

KVShareArena: KV-Cache Reuse Across Contexts and Model Checkpoints

Abstract

Reusing key–value (KV) caches speeds up LLM inference by avoiding repeated computation on shared text. The standard method, prefix caching, reuses a KV cache only when the LLM is the same and all text before the reused content is identical. However, real workloads often break both conditions. The preceding text changes when a RAG system places different documents before the same document, or when agents with different system prompts read the same file or tool output. The LLM changes when multi-agent workflows use specialized LLMs on shared material, or when an updated model reads documents cached by its previous version. Because KV caches depend on both the preceding text and the model weights, direct reuse in these settings can reduce answer quality. Many methods repair the reused cache or compress it, each with its own balance between quality and cost, so choosing among them requires a comparison under the same conditions. However, each method paper uses its own tasks, models, and cost measures, so the reported results cannot be compared directly. Existing benchmarks do not provide such a comparison either, because they mainly test long-context processing or reuse of an unchanged prefix. To fill this gap, we introduce KVShareArena, a benchmark and open evaluation framework that compares these methods under the same conditions. KVShareArena has (1) reuse tests on 2,150 questions from three QA datasets, in which the preceding text, the LLM that wrote the cache, or both change while the answering LLM and the input text stay fixed; (2) five dense and mixture-of-experts LLMs from 4B to 30B parameters and six LLM pairs in which one version of an LLM reads caches written by another, for 33 model–dataset settings in total; (3) 11 repair and compression methods from six method classes; (4) four evaluation perspectives: answer quality, prefill computation, KV-cache memory, and latency; and (5) a common interface for adding new methods and an interactive leaderboard that lets users set their own balance between quality and speed. Experiments on KVShareArena yield two findings. First, both the quality loss from reuse and which repairs help depend on the LLM, even between two 8B models. Second, most repairs keep their quality when another version of the LLM wrote the cache, but a trained repair adapter loses quality in 12 of the 18 pair–dataset tests.

Primary area:
datasets and benchmarks
Keywords:
KV cache reuse, large language models, benchmarking, retrieval-augmented generation, multi-agent systems, cross-model transfer

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_2d1oOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a benchmark for KV-cache reuse in the cases prefix caching cannot serve.

  1. cross-context reuse: a document block is cached on its own and later read after different preceding text
  2. cross-checkpoint reuse: the cache was written by a different checkpoint of the same architecture and tokenizer.

KVShareArena encodes every document block alone from position 0 and assembles the cached blocks into the receiver's prompt. Context and producer weights are varied as two separate factors.

The main evaluation covers 2,150 questions from Qasper, MultiFieldQA, and HotpotQA; five receivers (Qwen3-4B/8B/14B/30B-A3B and Llama-3.1-8B); and 14 configurations of 11 repair and compression methods, with position alignment as the reference. Six same-family producer–receiver pairs test the cross-checkpoint condition.

Critical strengths
  • A well-designed follow-up experiment. The paper asks whether KVPacket's drop needs a new design or only retraining, and answers with three conditions: the original adapter, an adapter retrained on the producer under test, and one cross-trained on the other producer. The cross-trained control is what separates "train per producer" from "just train more".

  • Careful dataset construction. Qasper and HotpotQA are extended to 1,000 questions each with LongBench's own procedure, which reproduces LongBench's Qasper samples exactly. The questions are also checked for overlap with the trained methods' training data.

  • Broad coverage, with training costs reported. (1) Five models from 4B to 30B, dense and MoE. (2) Eleven methods from six classes. (3) The trained methods are re-trained with their published recipes, and the training costs are reported.

Critical weaknesses
  1. The cross-checkpoint evidence is under-reported.

Table 1 positions KVShareArena as the only evaluation that covers reuse across checkpoints, and the abstract presents this as the second main finding. The supporting evidence has several problems.

(a) Most pairs have no per-method results. For the six pairs × three subsets, the only per-method, per-pair table is Table 18 (Llama-3.1-8B base --> Instruct). Table 17 gives no-repair numbers for the two Qwen3-8B pairs, on two of the three subsets. Table 19 covers one method, KVPacket, on the same two pairs. The 4B, 14B, and 30B pairs appear only in aggregate form in Table 14 (mean and worst change over 18 tests). A reader cannot check, for example, which pair or subset produces any given worst case entry.

(b) The size of the weight change is asserted, not measured, and the pair selection only loosely matches the motivation.

The introduction motivates cross-checkpoint reuse with task-specific fine-tunes in a multi-agent workflow (DroidSpeak) and an updated model reading its predecessor's caches. Four of the six pairs are instead base --> post-trained (Table 16). Section 3.6 justifies these as "the largest weight change within one family", but no measurement of the change is given, and the paper's own tables do not agree on which producer shifts the KV more. Without repair, in the same-context cells of Table 17, the base model lowers F1 more than the distilled sibling (DeepSeek-R1-0528-Qwen3-8B): .484 vs. .527 on HotpotQA, against .559 for the receiver's own cache. In the different-context cells the order reverses (.361 vs. .315, against .394). For KVPacket, the sibling is much more harmful (Table 19: -.199 vs. +.020 on multi-hop). Even position alignment changes by only -.008 F1 on average (Table 14). So the conclusion "most repairs keep their quality" may mainly reflect how close these particular pairs' KV caches are, not a property of the repairs.

  1. The leaderboard does not measure the quality-cost trade-off it is built to rank.

(a) The speed axis leaves out the main benefit of compression. TTFT is measured "with the cache already available", and cache building and loading are excluded (Table 13). In this setting every method that recomputes no document tokens has nearly the same TTFT: 12 of 14 entries fall within .86 - .92 (Table 2), and the paper itself notes that "quality decides most of the order". Compression methods save 25~72% of KV memory (Table 4), but in practice that saving matters through storage capacity and load and transfer time, the very costs the measurement leaves out. 4-bit quantization even ranks lower on speed (.860) than the uncompressed repairs, presumably because of dequantization. The score therefore ranks compression on the one axis where it cannot win and reports its real benefit "separately". The conclusion that repair methods "take the top six places" (Table 2) follows partly from this choice of measurement.

(b) The context change is synthetic, and the multi agent half of the motivation is not evaluated in the main benchmark. Every block is encoded alone, and the "different preceding text" is created by assembly. Blocks recur across questions only for Qasper and MultiFieldQA, and never for HotpotQA. This is a reasonable controlled proxy for quality. But it is the same construction that earlier method papers already evaluate, and it does not reflect how often caches are actually reused, which determines whether reuse pays off. The multi agent scenario that motivates half of the introduction appears only in the Agent Reports experiment (Appendix C). That appendix reports context value (Table 11) and two diagnostic numbers, but no per-method results, although it says that results "are reported separately".

Questions for authors
  1. What question set and sample size do Tables 16~19 use? How can KVPacket's worst producer change of -.146 F1 (Table 14) be reconciled with the -.199 implied by Table 19 (.447 → .248, distilled sibling, multi hop)? If these come from a 100 question subset, would you restate the abstract's numbers on the main sets?

  2. In what order are blocks assembled into the receiver prompt? Figure 4 shows d2, d1, d3 while the text does not specify an order. Does the full recomputation ceiling use the same order? This decides whether the ceiling and the method read exactly the same token sequence.

3 comments
AuthorsOpenReview ↗

Response to Reviewer 2d1o (1/2)

Thank you for the careful review. We answer each point below and have revised the paper accordingly. Tables R3 and R5 and shortened versions of Tables R1 and R7 are included below, and all full tables and Figure R1 are in the new Appendix R of the revised PDF.

W1(a). Per-method results for every pair

Thank you for pointing this out. We now report every method on every pair, with the pair and subset of each worst case (Table R1), and complete Table 17 (Table R2). The conclusion holds. Most repairs lose less than .02 F1 on average, and all but two methods stay within .040 F1 in every test. Ten of the 14 worst cases are on Llama base → Instruct, MultiFieldQA, where position alignment alone already loses .035 F1, so these drops come from the pair and subset rather than from the methods. The two exceptions lose most with the distilled sibling on multi-hop QA, the same-family producer that shifts the KV the most (Table R3). With position alignment, changing the context still costs .080–.328 F1, far more than the .008 that changing the producer costs on average (Table R2).

Table R1 (shortened). Worst producer-change test of position alignment and of each repair method over the six pairs and three subsets (paired ΔF1). The full table, with the compression methods and every method's mean on each pair, is in Appendix R.

MethodPair (worst test)SubsetΔF1
Position alignmentLlama base → InstructMultiFieldQA−.035
CacheBlendLlama base → InstructMultiFieldQA−.025
LegoLink (k=2)Llama base → InstructMultiFieldQA−.037
LegoLink (k=0)R1-distill → 8BHotpotQA−.151
APELlama base → InstructMultiFieldQA−.040
Block-AttentionLlama base → InstructMultiFieldQA−.034
KVPacketR1-distill → 8BHotpotQA−.146

W1(b). The size of the weight change and the choice of pairs

Thank you for this point. We now measure each producer change (Table R3). Same-family producers change the key-projection weights by 3.8–7.1%, against 81% for Qwen3-Coder, and shift the K cache by .19–.32 in relative L2 distance, about as much as a context change (.24–.29). On the main question sets, the tables agree that the distilled sibling shifts the KV more than the base model and, without repair, lowers F1 more (Tables R2 and R3). So the pairs' caches are not close, yet a same-family producer change costs twenty times less F1 than a context change. The reason is that only a context change makes the first token of each block draw much of the attention, an attention sink (Table R4). The test still detects real drops. For example, the Qwen3-Coder producer lowers F1 by .187, and KVPacket falls significantly in 12 of 18 tests and must be retrained on the producer it reads (Table R5). For the motivation, the base → post-trained pairs match an updated model that reads its predecessor's caches, and the two sibling pairs match specialized models fine-tuned from one base. The revision states these measurements and this mapping in §3.6, in place of "the largest weight change within one family".

Table R3. Weight change and KV shift per pair. Weight change: relative difference of the key-projection weights, averaged over layers. KV shift, producer change: relative L2 distance between the producer's and the receiver's K (V) for the same block encoded alone, averaged over tokens, layers, and the three subsets. KV shift, context change: the same distance between the receiver's position-aligned block cache and the same block inside the full prompt. F1 losses use position alignment, mean over the three subsets (Table R2; Qwen3-Coder from Table 15).

Pair (producer → receiver)Weight changeKV shift, producer change (K / V)F1 loss, producer changeKV shift, context change (K / V)F1 loss, context change
Qwen3-8B-Base → Qwen3-8B4.86%.263 / .384.003.286 / .337.228
DeepSeek-R1-0528-Qwen3-8B → Qwen3-8B7.12%.318 / .446.005.286 / .337.228
Qwen3-4B-Base → Qwen3-4B5.37%.276 / .371.011.293 / .368.205
Qwen3-14B-Base → Qwen3-14B3.94%.219 / .327.006.259 / .296.176
Llama-3.1-8B → Llama-3.1-8B-Instruct5.71%.241 / .358.023.238 / .305.139
Qwen3-30B-A3B-Thinking-2507 → Instruct-25073.79%.186 / .269−.002.247 / .322.164
Qwen3-Coder-30B-A3B-Instruct → Qwen3-30B-A3B-Instruct-250781%.893 / 1.127.187.247 / .322.164
AuthorsOpenReview ↗

Response to Reviewer 2d1o (2/2)

W2(a). The speed axis and compression

Thank you for pointing this out. We added the load time from host memory and over InfiniBand (Table R6), and the five leading repairs keep their places in every storage tier (Table R7). Loading lowers TTFT saved by .105 for a full cache but only .031 for 4-bit quantization, so compression gains speed, but not enough to offset its lower quality. In host memory, for example, Knorm (r=.25) saves .042 more TTFT than KVPacket but scores .245 lower on quality. 4-bit quantization is slower when the cache is already available because dequantization takes 11–32 ms per request, but it overtakes position alignment once loading counts. We measure storage capacity in our request trace (Figure R1), where 4-bit caches reach a hit rate of .913 against .782 for full caches, and a repair on a 4-bit cache is nearly as fast as 4-bit while staying well above position alignment (Table R8). The revision changes "top six" to "top five" in Table 2 and §3.2.

Table R7 (shortened). Leaderboard score (rank) with the cache in GPU memory and in host memory, averaged over the four dense models and the three subsets (Table 2 also includes Qwen3-30B-A3B, so its scores differ slightly). Score = ½ quality + ½ TTFT saved. Ranks are among the 13 configurations of Table 2 other than KVCOMM, merged when scores are within .005. With the cache loaded over InfiniBand, the top five keep the same places. The full table is in Appendix R.

MethodKV size vs BF16QualityGPU memoryHost memory
KVPacket1.030.683.793 (1).739 (1)
LegoLink (k=2)1.000.659.779 (2).727 (2)
APE1.000.602.755 (3).702 (3)
LegoLink (k=0)1.000.549.729 (4).676 (4)
CacheBlend1.000.678.712 (5).660 (5)
Block-Attention1.000.466.686 (6).634 (6–9)
Knorm (r=.25).750.438.677 (7–8).638 (6–9)
Quant. 4-bit.281.323.597 (12–13).581 (12)

W2(b). The context change and the multi-agent setting

Thank you for this point. We keep the controlled construction, which, as you note, is a reasonable proxy for quality and keeps our results comparable with the method papers. For the multi-agent half, Agent Reports now has per-method results, and repair leads and compression trails there, as in the main results. CacheBlend is significantly above position alignment on four of the five models, and every compression setting scores below it on all five (Table R9). We also added a multi-agent workload in which one agent hands the articles it opened to another, so the text before each reused article comes from the agents' run; it ranks methods like the main multi-hop QA results (Kendall τ = .83 and .94; Table R10). For reuse frequency, retrieved blocks recur often in deployed systems (Agarwal et al., 2025; Jin et al., 2024), and over our request trace reuse still saves .42–.89 of TTFT, even though every missing block is encoded on the spot (Figure R1).

Q1. Question sets and sample sizes of Tables 16–19

Thank you for checking this. Tables 17 and 19 used a smaller sample of questions; we re-ran both on the main question sets (1,000 / 150 / 1,000 questions), and every caption now states the question set and sample size. Table 16 lists the producer pairs, and Table 18 already uses the main question sets. On the main sets, the Table 19 setting gives −.146, the same as Table 14, so the abstract's numbers stand (Table R5). Retraining on the producer recovers .150 F1 (.282 → .432), and the retrained adapter is within .013 F1 of the same-producer score in all six tests.

Table R5. Table 19 on the main question sets (KVPacket, Qwen3-8B as the receiver). F1, with ΔF1 against the same-producer score (first row) in parentheses. * = 95% CI excludes zero (original adapter).

ProducerAdapterSci.-paper QASingle-doc QAMulti-hop QA
Qwen3-8B (same producer)original.403.409.428
DeepSeek-R1-0528-Qwen3-8B (distilled sibling)original.335 (−.068*).340 (−.069*).282 (−.146*)
retrained on this producer.399 (−.004).421 (+.012).432 (+.004)
cross-trained on the base producer.366 (−.037).362 (−.047).369 (−.059)
Qwen3-8B-Base (base predecessor)original.389 (−.014).352 (−.057*).411 (−.017*)
retrained on this producer.414 (+.011).422 (+.013).425 (−.003)
cross-trained on the sibling.355 (−.048).334 (−.075).404 (−.024)

Q2. Block order and the ceiling

Thank you for this question. Blocks keep the dataset's order, which is the window order of the document for Qasper and MultiFieldQA and LongBench's passage order for HotpotQA. Each method and the full-recomputation ceiling read exactly the same token sequence; the only difference is that the method's block KV was computed with each block alone. Figure 4 used d2, d1, d3 only to show that a block is read after text it was not encoded with; the revision redraws it with the actual order and states the order in §2.2. Shuffling the block order does not change the findings (Table R11).

Thank you for the detailed response and the new experiments. They address most of my concerns, and I am raising my rating from 2 to 3.

Reviewer_K3xsOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces KVShareArena, a benchmark for reusing document KV caches in two settings: cross-context reuse, where a cache is read after different preceding text than it was encoded with, and cross-checkpoint reuse, where the cache was written by a different checkpoint of the same architecture. The benchmark evaluates 11 KV reuse methods under a common protocol and ranks them on a quality–cost leaderboard, on which repair methods achieve the best overall performance.

Critical strengths

S1. A systematic benchmark. The benchmark varies context and producer independently while holding the receiver and text fixed. Quality, compute, memory and latency are reported carefully. The reported leaderboard enables direct comparisons of quality and efficiency across different methods.

S2. The careful reproduction of exsiting works. The paper runs 11 methods, including trained ones, on 5 receiver LLMs.

Critical weaknesses

W1. Limited novelty, and the headline findings are largely expected. Comparing the quality and cost of KV reuse on QA datasets is not new. Cestola et al. (2026) already compare eleven chunk-level reuse methods on quality. The original method papers (CacheBlend, EPIC) already evaluate isolated-chunk encoding on LongBench-style QA. The two findings are largely expected observations and given without a mechanism. Beyond the leaderboard, the new contribution is thin.

W2. Workloads are unrepresentative of the motivated scenarios. The paper motivates the problem with multi-agent and dynamic settings. Its main evaluation, however, uses three existing QA datasets with fixed evidence and a single-turn prompt.

Questions for authors

Q1. Could the authors include more representative workloads, particularly multi-turn agent workflows with dynamic tool outputs and changing contexts?

Q2. In Qasper and MultiFieldQA, the full document always appears in order after the same instruction, so the preceding text of each window never changes. Why is isolated window encoding the right comparison here, rather than whole-document prefix caching?

Q3. How sensitive are the main findings to document chunk size and block ordering?

2 comments
AuthorsOpenReview ↗

Response to Reviewer K3xs

Thank you for the review, and for recognizing the systematic benchmark and its leaderboard (S1, S2). A shortened version of Table R10 is included below, and all tables numbered R are in the new Appendix R of the revised PDF.

W1. Novelty, the main contribution, and the headline findings

Thank you for this point. Our main contribution is an arena, with one fixed protocol and an open leaderboard that new methods can join over time. This matters because each reuse paper uses its own tasks, models, and cost measures. Cestola et al. compare methods once, and CacheBlend and EPIC test only their own method (table below). Building an arena is hard, because the methods change different things, from recomputed tokens to trained adapters, and the cache may come from another model checkpoint, which none of these works tests. We reproduced eleven methods, including trained ones, on five models under one protocol, as you note in S1 and S2, and tested the methods across checkpoints. The revised appendix also gives the mechanism behind both findings (Table R4). Testing across checkpoints changes the choice of method. KVPacket ranks first, but it loses quality in 12 of 18 cross-checkpoint tests and must be retrained for each new producer.

Reuse across checkpoints and an open arena, as reported in each paper.

EvaluationReuse across checkpointsOpen arena (one protocol, public leaderboard)
Cestola et al. (2026)××
CacheBlend (Yao et al., 2025), EPIC (Hu et al., 2025)××
SCBench (Li et al., 2025)××
KVShareArena (ours)✓✓

W2 and Q1. Workloads and multi-turn agent workflows

Thank you for pointing this out. We added the multi-turn agent workflow you suggest, and it agrees with our QA results. One agent searches and opens articles, so the tool outputs change at run time, and a second agent with its own prompt reads them from cache. The method ranking agrees with the main multi-hop QA ranking on both models (Kendall τ = .83 and .94), and the same method ranks first (Table R10). This is expected, because KV-cache reuse works below the task, and in both settings a method must handle a cached block that is read after different text.

Table R10 (shortened). Multi-turn agent workflow, PGR at the Answerer call (300 questions per model). ↑ = significantly above position alignment. ‡ = context value below .25 F1, where methods are hard to separate. Kendall τ is computed over the nine shared methods. The full table, with TTFT saved and the rows not shown, is in Appendix R.

MethodQwen3-8BLlama-3.1-8B-Instruct ‡
LegoLink (k=2).895↑.646
CacheBlend.835↑.533
APE.779↑.786
KVPacket.693.904↑
Quant. 4-bit.461.594
Kendall τ vs main multi-hop QA ranking.83.94

Q2. Isolated windows vs. whole-document prefix caching

Thank you for this question. The two subsets stand for chunked RAG, where each chunk of a long document is read after whichever chunks are retrieved with it, so a whole-document prefix cache does not apply. We give every method all windows in order only to keep retrieval quality out of the comparison. When each question instead retrieves its top 5 windows, a whole-document prefix cache serves only 3–4% of them, and the method ranking agrees with the main setting (Kendall τ = .96 and .87; Table R12).

Q3. Chunk size and block order

Thank you for this question. Neither changes the main findings (Tables R11 and R13). With chunks of 256 to 1,024 tokens and with three random block orders, the same methods lead, and the difference between the two 8B models remains. Smaller chunks only make repair more useful, because they add more block boundaries.

Dear Reviewer K3xs,

Thank you again for your review. As the discussion period closes soon, we would like to check whether our response addresses your concerns. If any point is still unclear, we would be glad to clarify before the discussion ends.

Best regards, The authors

Reviewer_uW3LOpenReview ↗
Rating
3
Confidence
1
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

KVShareArena benchmarks KV-cache reuse. It standardizes evaluation across 2,150 questions from three QA datasets, five receiver models, six producer-receiver pairs, and 11 repair/compression methods. Answer quality is normalized against no-context and full-recomputation references. Its main novelty is the unified evaluation protocol and extensible framework, rather than a new repair algorithm or task dataset.

Critical strengths
  1. A useful, controlled evaluation problem.

  2. The model-dependent results are informative.

Critical weaknesses
  1. The framework addresses a meaningful evaluation gap, and the model-dependent observations are useful. However, the operational assumptions limit the central method rankings, and the unreconciled cross-checkpoint results weaken one of the paper's main distinctive findings.

  2. The overall ranking mixes operational conditions and does not establish which method is preferable in a shared deployment setting.

  3. The evidence for the distinctive cross-checkpoint finding is not internally reconciled.

Questions for authors
  1. How does the method ranking change when all methods run on a common request trace with explicitly controlled settings (e.g., cache readiness, reuse frequency, construction/loading costs)? In particular, what happens to KVCOMM after warm-up?

  2. Under a common memory or transfer budget, do the repair-versus-compression conclusions survive, and how does a strong repair combined with compression compare with the separate methods?

1 comment
AuthorsOpenReview ↗

Response to Reviewer uW3L

Thank you for the review and for two concrete questions, which we answer with new experiments. Table R8 and the data behind Figure R1 are included below, and the plot and the other tables are in the new Appendix R of the revised PDF.

W1 and Q1. Operational conditions and KVCOMM after warm-up

Thank you for this suggestion. We ran all methods on a common request trace with controlled cache capacity, reuse frequency, and loading and construction costs (Figure R1). Over the trace, reuse still saves .42–.89 of TTFT, even though every missing block is encoded on the spot. Repairs keep their quality lead (multi-hop PGR .496 for LegoLink (k=2) against .041 for SnapKV (r=.5)), and compression gains speed only because a smaller cache keeps more blocks in memory (on Qasper-like requests with 50 GB of GPU memory, a hit rate of .913 for 4-bit caches against .782 for full caches); a repair on a compressed cache is just as small and nearly as fast, with much higher quality (Table R8). In the leaderboard, only KVCOMM mixes warm and cold requests; every other method is timed under the same condition. After warm-up, KVCOMM saves .77 of TTFT on requests that reuse its stored offsets, but its quality is then that of reuse rather than of full recomputation.

Figure R1 data (shortened). TTFT saved over the request trace (Qwen3-8B, s = 1.0; mean over the 5,000 timed requests) with 50 GB of GPU memory, alone or with 200 GB of host memory, next to each method's PGR from Table 6. The plot, with the 100 GB point, is in Appendix R.

MethodQasper-like, GPU 50 GBQasper-like, + host 200 GBHotpotQA-like, GPU 50 GBHotpotQA-like, + host 200 GBPGR (Table 6), Qasper / HotpotQA
LegoLink (k=2).665.805.579.702.853 / .496
KVPacket.668.804.582.706.750 / .469
CacheBlend.514.650.424.544.857 / .440
SnapKV (r=.5).759.888.665.806.499 / .041
Quant. 4-bit.755.832.652.799.388 / −.115

W2. The cross-checkpoint evidence

Thank you for pointing this out. We re-ran Tables 17 and 19 on the main question sets, and they now agree with Table 14 and the abstract, so the cross-checkpoint finding stands (Tables R2 and R5). The submitted tables used a smaller sample of questions. Our response to Reviewer 2d1o (W1 and Q1) gives the details, and the revised appendix reports every method on every pair (Table R1).

Q2. A common memory or transfer budget

Thank you for this question. Yes, the conclusion survives at equal memory, which at a given bandwidth is also equal transfer cost. At 28% of the BF16 bytes, LegoLink (k=2) on a 4-bit cache recovers .764 / .657 / .369 of the gap, against .388 / .382 / −.115 for 4-bit alone, at nearly the same speed (Table R8). So repair and compression work together. Compression sets the bytes, and the repair restores most of the quality.

Table R8. Equal memory: repairs on a 4-bit cache (Qwen3-8B, main question sets). PGR as scientific-paper / single-document / multi-hop QA; ↑ = significantly above position alignment. TTFT saved: mean of the three subsets, with the cache in GPU memory / loaded from host memory, as in Table R6.

MethodKV size vs BF16PGRTTFT saved, GPU / host memory
Position alignment1.000.568 / .403 / .030.905 / .799
Quant. 4-bit.281.388 / .382 / −.115.863 / .832
LegoLink (k=2)1.000.853↑ / .674↑ / .496↑.900 / .793
LegoLink (k=2) + 4-bit.281.764↑ / .657↑ / .369↑.858 / .827
CacheBlend1.000.857↑ / .748↑ / .440↑.742 / .636
CacheBlend + 4-bit.281.799↑ / .740↑ / .378↑.701 / .670

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.