- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates the feasibility of direct KV-cache communication between frozen LLMs and identifies a fundamental limitation of cache substitution: geometric similarity does not guarantee that a receiving model can effectively use the transferred information. Its central contribution is to distinguish cache fidelity from consumability and explain their divergence through the competitive nature of attention, where small perturbations can disrupt critical attention patterns. Through controlled experiments on a SQuAD-based factual recall task, the authors show that even highly similar caches can cause severe performance degradation, while eight diverse substitution methods fail to meaningfully exceed the baseline. In contrast, retaining the receiver’s own cache and incorporating the sender’s cache as additional context substantially improves accuracy, with adaptation further enabling effective transfer from multiple senders. These findings motivate a shift from cache reconstruction toward attention-based augmentation, although the evidence remains largely limited to a single model architecture and task setting.
Critical strengths
- Insightful problem formulation. The paper reveals a fundamental disconnect between KV-cache fidelity and consumability, supported by a compelling mechanistic explanation based on competitive attention.
- Systematic validation and actionable insights. Consistent failures across eight substitution methods strengthen the central claim, while the success of cache augmentation offers a promising alternative for latent communication.
Critical weaknesses
- Insufficient mechanistic evidence. The proposed explanation based on competitive attention is not directly validated through attention-margin or ranking analyses. Moreover, the key fidelity comparison is confounded by answer availability, and the on-manifold experiments do not preserve coherent key–value pairs. These limitations weaken the causal interpretation of the central findings.
- Limited evidence for the broader design conclusions. The negative results exclude previously successful fitted-bridge approaches, while the positive augmentation results are restricted to the same model checkpoint and a single factual recall task. Given that prior work has already identified attention-sensitive reconstruction errors and explored cache augmentation, the paper does not yet sufficiently establish the generality or distinctiveness of its proposed design principle.
Questions for authors
- Can the authors directly establish that changes in attention margins or key rankings explain the performance collapse, rather than merely ruling out error amplification? In particular, would the fidelity–consumability gap persist under matched, answer-bearing cache contents and controlled perturbations?
- Since the nearest-neighbor experiments replace individual keys while retaining the original values, how would the conclusions change when transferring coherent key–value pairs? What evidence establishes that failure arises from the receiver's tolerable set rather than inconsistencies introduced by the substitution protocol?
- How would a fitted bridge trained on independent paired-cache calibration data perform on the same split-context task? Since such methods have demonstrated successful cache substitution in prior work, this comparison seems essential for determining whether the observed limitation is intrinsic to substitution or specific to the tested constructions.
- Section 5.2 attributes the 0.695 accuracy to query-side LoRA, whereas Appendix F reports 0.600 for query-only adaptation and 0.695 when adapting queries, keys, and values. Which configuration produces the reported result, and does the claimed effectiveness of query-only adaptation still hold under the corresponding negative controls?
8 comments
---- Comment 2/6: Weakness 1 (a, b) ----
W1(a). The competitive-attention explanation is not validated by margin or ranking analyses.
- Response: We agree that ruling out amplification is not the same as showing the mechanism, and the revision adds direct tests. To state the scope plainly: we measured rankings (whether the most-attended position changes) but did not measure attention margins.
- New results: The strongest evidence is an intervention on the receiver's own full-passage cache (Section 4.1, "Direction, not distance"; Figure 2). Moving only its keys 3% of the way toward those of DeepSeek-R1-Distill-Qwen-1.5B, with the receiver's answer-bearing values untouched, lowers accuracy from 0.665 to 0.160, although the blended keys differ from the receiver's by a relative error of only 0.0002. The same blend applied to the values alone is harmless, and replacing all of the receiver's keys with those of Qwen2.5-1.5B-Instruct leaves accuracy at 0.685. Both changes are tiny by any distance measure; what separates them is the direction in which the keys move.
Second, at matched per-layer cosine on the answer-bearing cache, we aim noise at the directions the receiver's queries are most sensitive to, or at random directions. Because we choose the direction and observe the effect, this is also an intervention rather than only a correlation. Geometric distance alone would predict equal damage:
| Per-layer cosine | Random directions | Query-sensitive directions |
|---|---|---|
| 0.999 | 0.105 | 0.040 |
| 0.995 | 0.060 | 0.010 |
| 0.990 | 0.025 | 0.005 |
Third, for rankings: at the same per-layer cosine, noise changes the most-attended position more often than the genuine cache does (0.30 versus 0.21 of contests; 0.15 versus 0.08 for sharply focused heads). Across 58 perturbations, this change rate predicts accuracy better than cosine (rank correlation 0.91 versus 0.80 to 0.87), and within one condition it modestly identifies failing questions (AUC 0.66 versus 0.48). Section 4.1 also states its limits: the relation works mostly through the first layer; in later layers genuine keys move attention less than noise yet are no more usable; and the change rate does not predict which fitted bridge works. These measurements are correlational and taken at the first decoding step.
The better behavioral test is the receiver's output divergence. Across 49 conditions, the KL divergence at the first answer token has a rank correlation of 0.955 with accuracy and explains 93% of held-out variance, against 0.81 and 45% for cosine.
- Where in the paper: Section 4.1 ("Direct evidence"; "Direction, not distance"), Figure 2, Section 5.6, and Figure 5.
W1(b). The key fidelity comparison is confounded by answer availability.
- Response: The reviewer is right.
- New results: We add noise directly to the receiver's cache of the full passage, which contains the answer. Accuracy still falls from 0.665 to 0.105 at per-layer cosine 0.999, so the fragility does not depend on whether the answer is present. Noise on the answer-bearing cache, calibrated globally, gives 0.682, 0.467, and 0.003 at cosines 0.9999, 0.999, and 0.995; calibrated layer by layer, a per-layer cosine of 0.999 already gives 0.100. We also found that the original genuine-cache blend was position-misaligned after the masked span. Once filler tokens stand in for the answer one for one and the positions line up, the genuine full-passage cache (mean per-layer cosine 0.982) recovers 0.665. Noise at that cosine in every layer, applied to keys and values separately, gives 0.122 ± 0.013, below the 0.155 floor of that setup. This is the matched, answer-bearing comparison the reviewer asked for in Q1.
- Where in the paper: Section 3.1 and Appendix B.
---- Comment 3/6: Weakness 1 (c) and Weakness 2 (a) ----
W1(c). The on-manifold experiments do not preserve coherent key–value pairs.
- Response: This question led us to a real error. On auditing the code, we found that the key and value banks were subsampled independently, so each selected key was paired with the value of an unrelated token (Appendix E, formerly Appendix D, had incorrectly said that only keys were replaced). We reran the experiment with matched pairs.
- New results:
| Substitution (nearest genuine entries, Qwen2.5-1.5B) | Accuracy | Key / value cosine |
|---|---|---|
| Original run (keys with unrelated values) | 0.005 | 0.82 / 0.10 |
| Matched key–value pairs | 0.005 | 0.83 / 0.39 |
| Matched pairs, positional rotation made consistent | 0.015 | 0.89 / 0.51 |
| Keys only (receiver's own values kept) | 0.170 | 0.83 / 1.00 |
| Values only | 0.005 | 1.00 / 0.39 |
Floor 0.180. The key and value cosines in the last column are reported with the corrected results in Appendix E. With matched pairs the conclusion is unchanged: genuine entries from other passages cannot stand in for the right ones, and the same holds on four further receivers. The rerun also shows that most of this effect comes from the values, which carry the content; genuine keys alone leave the receiver at the floor. We have revised Section 3.3 to say so, and we now treat this experiment as weaker evidence than we originally did.
- Where in the paper: Section 3.3 and Appendix E (corrected).
W2(a). The negative results exclude previously successful fitted-bridge approaches.
- Response: We agree, and the revision now tests a fitted bridge directly and explains why earlier ones succeeded (Appendix K). Closed-form ridge maps (Heo et al.) work within one model family, whose caches start much closer together than those of unrelated models. Our account predicts that a fidelity-trained map works only when the sender's cache already lies near the receiver's tolerable set. Among our senders, Qwen2.5-1.5B-Instruct is that case: its raw cache already scores 0.630, and every bridge leaves that accuracy in place. DeepSeek-R1-Distill-Qwen-1.5B and Qwen2.5-0.5B are not: with them the fidelity bridges stay below the floor (ridge 0.078 with the DeepSeek-distill sender), while the same bridge trained through the receiver reaches 0.613 and 0.318, respectively. So fitting alone does not decide the outcome; the objective does, given where the sender starts (Q3).
- So that this condition is not circular, whether a cache lies in the tolerable set can be checked without accuracy labels, through the receiver's output divergence (Section 5.6), which separates working from failing caches where cosine cannot.
- Where in the paper: Section 2 (C1–C3), Section 5.4, Table 3, Section 5.6, and Appendix K (Table 11).
---- Comment 1/6: Overview ----
This was a thoughtful review, and its questions identified the weakest links in our argument. We are glad the reviewer found the problem formulation insightful. In response we ran new experiments, corrected an error the reviewer identified, and sharpened the central claim.
Your questions about the mechanism and about why some methods work while others fail shaped the main change. The paper now argues one thesis from start to finish: success is decided by the objective a method optimizes, either closeness to the receiver's cache (fidelity) or the receiver's behavior (consumability), and the mechanism in Section 4 explains why only the second reliably works. We also rewrote the text for clarity and tone. New results and passages are marked in blue in the revised PDF. A summary of the full revision is in the general response posted at the top of the forum, and we answer each of your points below.
Summary of the main changes