- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes a benchmark for KV-cache reuse in the cases prefix caching cannot serve.
- cross-context reuse: a document block is cached on its own and later read after different preceding text
- cross-checkpoint reuse: the cache was written by a different checkpoint of the same architecture and tokenizer.
KVShareArena encodes every document block alone from position 0 and assembles the cached blocks into the receiver's prompt. Context and producer weights are varied as two separate factors.
The main evaluation covers 2,150 questions from Qasper, MultiFieldQA, and HotpotQA; five receivers (Qwen3-4B/8B/14B/30B-A3B and Llama-3.1-8B); and 14 configurations of 11 repair and compression methods, with position alignment as the reference. Six same-family producer–receiver pairs test the cross-checkpoint condition.
Critical strengths
-
A well-designed follow-up experiment. The paper asks whether KVPacket's drop needs a new design or only retraining, and answers with three conditions: the original adapter, an adapter retrained on the producer under test, and one cross-trained on the other producer. The cross-trained control is what separates "train per producer" from "just train more".
-
Careful dataset construction. Qasper and HotpotQA are extended to 1,000 questions each with LongBench's own procedure, which reproduces LongBench's Qasper samples exactly. The questions are also checked for overlap with the trained methods' training data.
-
Broad coverage, with training costs reported. (1) Five models from 4B to 30B, dense and MoE. (2) Eleven methods from six classes. (3) The trained methods are re-trained with their published recipes, and the training costs are reported.
Critical weaknesses
- The cross-checkpoint evidence is under-reported.
Table 1 positions KVShareArena as the only evaluation that covers reuse across checkpoints, and the abstract presents this as the second main finding. The supporting evidence has several problems.
(a) Most pairs have no per-method results. For the six pairs × three subsets, the only per-method, per-pair table is Table 18 (Llama-3.1-8B base --> Instruct). Table 17 gives no-repair numbers for the two Qwen3-8B pairs, on two of the three subsets. Table 19 covers one method, KVPacket, on the same two pairs. The 4B, 14B, and 30B pairs appear only in aggregate form in Table 14 (mean and worst change over 18 tests). A reader cannot check, for example, which pair or subset produces any given worst case entry.
(b) The size of the weight change is asserted, not measured, and the pair selection only loosely matches the motivation.
The introduction motivates cross-checkpoint reuse with task-specific fine-tunes in a multi-agent workflow (DroidSpeak) and an updated model reading its predecessor's caches. Four of the six pairs are instead base --> post-trained (Table 16). Section 3.6 justifies these as "the largest weight change within one family", but no measurement of the change is given, and the paper's own tables do not agree on which producer shifts the KV more. Without repair, in the same-context cells of Table 17, the base model lowers F1 more than the distilled sibling (DeepSeek-R1-0528-Qwen3-8B): .484 vs. .527 on HotpotQA, against .559 for the receiver's own cache. In the different-context cells the order reverses (.361 vs. .315, against .394). For KVPacket, the sibling is much more harmful (Table 19: -.199 vs. +.020 on multi-hop). Even position alignment changes by only -.008 F1 on average (Table 14). So the conclusion "most repairs keep their quality" may mainly reflect how close these particular pairs' KV caches are, not a property of the repairs.
- The leaderboard does not measure the quality-cost trade-off it is built to rank.
(a) The speed axis leaves out the main benefit of compression. TTFT is measured "with the cache already available", and cache building and loading are excluded (Table 13). In this setting every method that recomputes no document tokens has nearly the same TTFT: 12 of 14 entries fall within .86 - .92 (Table 2), and the paper itself notes that "quality decides most of the order". Compression methods save 25~72% of KV memory (Table 4), but in practice that saving matters through storage capacity and load and transfer time, the very costs the measurement leaves out. 4-bit quantization even ranks lower on speed (.860) than the uncompressed repairs, presumably because of dequantization. The score therefore ranks compression on the one axis where it cannot win and reports its real benefit "separately". The conclusion that repair methods "take the top six places" (Table 2) follows partly from this choice of measurement.
(b) The context change is synthetic, and the multi agent half of the motivation is not evaluated in the main benchmark. Every block is encoded alone, and the "different preceding text" is created by assembly. Blocks recur across questions only for Qasper and MultiFieldQA, and never for HotpotQA. This is a reasonable controlled proxy for quality. But it is the same construction that earlier method papers already evaluate, and it does not reflect how often caches are actually reused, which determines whether reuse pays off. The multi agent scenario that motivates half of the introduction appears only in the Agent Reports experiment (Appendix C). That appendix reports context value (Table 11) and two diagnostic numbers, but no per-method results, although it says that results "are reported separately".
Questions for authors
-
What question set and sample size do Tables 16~19 use? How can KVPacket's worst producer change of -.146 F1 (Table 14) be reconciled with the -.199 implied by Table 19 (.447 → .248, distilled sibling, multi hop)? If these come from a 100 question subset, would you restate the abstract's numbers on the main sets?
-
In what order are blocks assembled into the receiver prompt? Figure 4 shows d2, d1, d3 while the text does not specify an order. Does the full recomputation ceiling use the same order? This decides whether the ceiling and the method read exactly the same token sequence.
3 comments
Response to Reviewer 2d1o (2/2)
W2(a). The speed axis and compression
Thank you for pointing this out. We added the load time from host memory and over InfiniBand (Table R6), and the five leading repairs keep their places in every storage tier (Table R7). Loading lowers TTFT saved by .105 for a full cache but only .031 for 4-bit quantization, so compression gains speed, but not enough to offset its lower quality. In host memory, for example, Knorm (r=.25) saves .042 more TTFT than KVPacket but scores .245 lower on quality. 4-bit quantization is slower when the cache is already available because dequantization takes 11–32 ms per request, but it overtakes position alignment once loading counts. We measure storage capacity in our request trace (Figure R1), where 4-bit caches reach a hit rate of .913 against .782 for full caches, and a repair on a 4-bit cache is nearly as fast as 4-bit while staying well above position alignment (Table R8). The revision changes "top six" to "top five" in Table 2 and §3.2.
Table R7 (shortened). Leaderboard score (rank) with the cache in GPU memory and in host memory, averaged over the four dense models and the three subsets (Table 2 also includes Qwen3-30B-A3B, so its scores differ slightly). Score = ½ quality + ½ TTFT saved. Ranks are among the 13 configurations of Table 2 other than KVCOMM, merged when scores are within .005. With the cache loaded over InfiniBand, the top five keep the same places. The full table is in Appendix R.
| Method | KV size vs BF16 | Quality | GPU memory | Host memory |
|---|---|---|---|---|
| KVPacket | 1.030 | .683 | .793 (1) | .739 (1) |
| LegoLink (k=2) | 1.000 | .659 | .779 (2) | .727 (2) |
| APE | 1.000 | .602 | .755 (3) | .702 (3) |
| LegoLink (k=0) | 1.000 | .549 | .729 (4) | .676 (4) |
| CacheBlend | 1.000 | .678 | .712 (5) | .660 (5) |
| Block-Attention | 1.000 | .466 | .686 (6) | .634 (6–9) |
| Knorm (r=.25) | .750 | .438 | .677 (7–8) | .638 (6–9) |
| Quant. 4-bit | .281 | .323 | .597 (12–13) | .581 (12) |
W2(b). The context change and the multi-agent setting
Thank you for this point. We keep the controlled construction, which, as you note, is a reasonable proxy for quality and keeps our results comparable with the method papers. For the multi-agent half, Agent Reports now has per-method results, and repair leads and compression trails there, as in the main results. CacheBlend is significantly above position alignment on four of the five models, and every compression setting scores below it on all five (Table R9). We also added a multi-agent workload in which one agent hands the articles it opened to another, so the text before each reused article comes from the agents' run; it ranks methods like the main multi-hop QA results (Kendall τ = .83 and .94; Table R10). For reuse frequency, retrieved blocks recur often in deployed systems (Agarwal et al., 2025; Jin et al., 2024), and over our request trace reuse still saves .42–.89 of TTFT, even though every missing block is encoded on the spot (Figure R1).
Q1. Question sets and sample sizes of Tables 16–19
Thank you for checking this. Tables 17 and 19 used a smaller sample of questions; we re-ran both on the main question sets (1,000 / 150 / 1,000 questions), and every caption now states the question set and sample size. Table 16 lists the producer pairs, and Table 18 already uses the main question sets. On the main sets, the Table 19 setting gives −.146, the same as Table 14, so the abstract's numbers stand (Table R5). Retraining on the producer recovers .150 F1 (.282 → .432), and the retrained adapter is within .013 F1 of the same-producer score in all six tests.
Table R5. Table 19 on the main question sets (KVPacket, Qwen3-8B as the receiver). F1, with ΔF1 against the same-producer score (first row) in parentheses. * = 95% CI excludes zero (original adapter).
| Producer | Adapter | Sci.-paper QA | Single-doc QA | Multi-hop QA |
|---|---|---|---|---|
| Qwen3-8B (same producer) | original | .403 | .409 | .428 |
| DeepSeek-R1-0528-Qwen3-8B (distilled sibling) | original | .335 (−.068*) | .340 (−.069*) | .282 (−.146*) |
| retrained on this producer | .399 (−.004) | .421 (+.012) | .432 (+.004) | |
| cross-trained on the base producer | .366 (−.037) | .362 (−.047) | .369 (−.059) | |
| Qwen3-8B-Base (base predecessor) | original | .389 (−.014) | .352 (−.057*) | .411 (−.017*) |
| retrained on this producer | .414 (+.011) | .422 (+.013) | .425 (−.003) | |
| cross-trained on the sibling | .355 (−.048) | .334 (−.075) | .404 (−.024) |
Q2. Block order and the ceiling
Thank you for this question. Blocks keep the dataset's order, which is the window order of the document for Qasper and MultiFieldQA and LongBench's passage order for HotpotQA. Each method and the full-recomputation ceiling read exactly the same token sequence; the only difference is that the method's block KV was computed with each block alone. Figure 4 used d2, d1, d3 only to show that a block is read after text it was not encoded with; the revision redraws it with the actual order and states the order in §2.2. Shuffling the block order does not change the findings (Table R11).
Thank you for the detailed response and the new experiments. They address most of my concerns, and I am raising my rating from 2 to 3.
Response to Reviewer 2d1o (1/2)
Thank you for the careful review. We answer each point below and have revised the paper accordingly. Tables R3 and R5 and shortened versions of Tables R1 and R7 are included below, and all full tables and Figure R1 are in the new Appendix R of the revised PDF.
W1(a). Per-method results for every pair
Thank you for pointing this out. We now report every method on every pair, with the pair and subset of each worst case (Table R1), and complete Table 17 (Table R2). The conclusion holds. Most repairs lose less than .02 F1 on average, and all but two methods stay within .040 F1 in every test. Ten of the 14 worst cases are on Llama base → Instruct, MultiFieldQA, where position alignment alone already loses .035 F1, so these drops come from the pair and subset rather than from the methods. The two exceptions lose most with the distilled sibling on multi-hop QA, the same-family producer that shifts the KV the most (Table R3). With position alignment, changing the context still costs .080–.328 F1, far more than the .008 that changing the producer costs on average (Table R2).
Table R1 (shortened). Worst producer-change test of position alignment and of each repair method over the six pairs and three subsets (paired ΔF1). The full table, with the compression methods and every method's mean on each pair, is in Appendix R.
W1(b). The size of the weight change and the choice of pairs
Thank you for this point. We now measure each producer change (Table R3). Same-family producers change the key-projection weights by 3.8–7.1%, against 81% for Qwen3-Coder, and shift the K cache by .19–.32 in relative L2 distance, about as much as a context change (.24–.29). On the main question sets, the tables agree that the distilled sibling shifts the KV more than the base model and, without repair, lowers F1 more (Tables R2 and R3). So the pairs' caches are not close, yet a same-family producer change costs twenty times less F1 than a context change. The reason is that only a context change makes the first token of each block draw much of the attention, an attention sink (Table R4). The test still detects real drops. For example, the Qwen3-Coder producer lowers F1 by .187, and KVPacket falls significantly in 12 of 18 tests and must be retrained on the producer it reads (Table R5). For the motivation, the base → post-trained pairs match an updated model that reads its predecessor's caches, and the two sibling pairs match specialized models fine-tuned from one base. The revision states these measurements and this mapping in §3.6, in place of "the largest weight change within one family".
Table R3. Weight change and KV shift per pair. Weight change: relative difference of the key-projection weights, averaged over layers. KV shift, producer change: relative L2 distance between the producer's and the receiver's K (V) for the same block encoded alone, averaged over tokens, layers, and the three subsets. KV shift, context change: the same distance between the receiver's position-aligned block cache and the same block inside the full prompt. F1 losses use position alignment, mean over the three subsets (Table R2; Qwen3-Coder from Table 15).