- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3(+1)
Read review
Paper summary
This paper proposes Gricean Cache Sharing (GCS), an inter-agent communication protocol that replaces natural language text exchange with selective KV-cache. Guided metaphorically by Grice’s Cooperative Principle, GCS sparsifies the transmitted cache along three axes: attention heads, layer depth, and token positions. Evaluated on 5 models across 3 benchmarks (ARC-C, GSM8K, HumanEval+), GCS achieves a 3.3× average speedup over text-mediated collaboration while matching accuracy on single-GPU homogeneous setups.
Critical strengths
-
The overall idea is nice. Replacing NL generation with a sparse KV-cache can boost efficiency, and the methods are documented in detail.
-
Rigorous engineering on latent feature probing. The authors do solve essential latent communication issues, such as mitigating distribution shift by using a closed-form ridge regression alignment map and managing RoPE position ID continuity across agents.
Critical weaknesses
-
Trivial and overpacked theoretical claim. The theoretical results are mostly cliche; Theorem 3.6 spends an entire page in the appendix to prove k largest numbers maximize a sum. I wonder if it is necessary to use so many theorems in the paper. Besides, mapping Grice’s Cooperative Principle to standard pruning techniques is pure rhetoric. The overall method is essentially a 3D cache pruning pipeline where the "Maxims" barely add any substance.
-
The architecture requires all model beusing exactly the same architecture. And the methods are running on one hardware where memory copy latency is negligible, whereas in realistic multi-agent cases, where distributed deployment may introduce heavy KV cache transmission cost, making it impractical.
Questions for authors
How will the method handle the network transfer latency of 10–90 MiB KV-caches in distributed multi-node settings?
3 comments
Response to Reviewer MnYf (Part 2/3)
Response to Reviewer MnYf (Part 2/3)
W2 (Architecture): Same-Family vs. Cross-Family Heterogeneous Collaboration
We share your view that cross-architecture KV sharing is a hard open problem. We added a mixed-size Qwen3-8B -> 1.7B -> 4B study to Table 4b of Section 4.4 and Table 12 of Appendix C.9. Below, we also report an exploratory cross-family Mistral-7B -> Qwen3-1.7B -> Qwen3-4B experiment.
First, we tested heterogeneous sizes within Qwen3, with layer counts of (36, 28, 36) and hidden dimensions of (4096, 2048, 2560):
Same-Family Heterogeneous (Qwen3-8B -> 1.7B -> 4B) | Channel / KV | 3-Bench Acc (%) | Latency (s) | Speedup |
|---|---|---|---|---|
| Heterogeneous Text Communication | Text | 83.9 ± 0.3 | 36.5 | 1.00× |
| Heterogeneous C2C† [1] (Trained MLP) | 100% | 85.8 ± 0.3 | 16.4 | 2.23× |
| Heterogeneous GCS (Ours, CKA + Ridge) | 12% | 86.4 ± 0.3 | 9.8 | 3.72× |
In Appendix C.9, we extend Module II with 100 × 100 Linear CKA [10] Gram matrices on the calibration set to match receiver role layers with sender layers across depths and widths. On the public calibration hidden states, we fit closed-form ridge adapters for keys and values per matched head block, using a ridge regularization coefficient of 0.001. Without gradient training, we reach 86.4 ± 0.3% in 9.8s, beating both Heterogeneous Text and trained C2C† [1], and running faster than standalone Qwen3-4B at 11.1s. In Table 12, we also confirm gains on 4-agent chains at 88.3%, 2-round debate at 88.1%, star graphs at 88.0%, and SWE-bench Lite at 28.3% vs. 26.7% text.
Second, we examined cross-family KV sharing, where lightweight alignment remains an open problem. While C2C [1] trains end-to-end cross-family fusers, training-free KV protocols [2, 3, 6] and serving systems like DroidSpeak [5] and CacheGen [11] require a shared model family. In fact, [5] shows that even fine-tuning drift on identical architectures hurts KV reuse unless sensitive layers are recomputed. Distinct families use incompatible tokenizers such as 32k SentencePiece in Mistral-7B-Instruct-v0.3 vs. 152k BPE in Qwen3 [12], unaligned KV subspaces, and QK-Norm in Qwen3 [12] alongside diverse attention layouts [13]. Tokenizer mismatch breaks token-to-token KV alignment, and adapter errors in keys can be amplified inside receiver attention softmax across layers.
We tested this directly by swapping the Qwen3-8B Planner for Mistral-7B-Instruct-v0.3 with linear token-position interpolation at the Mistral -> Qwen handoff:
Cross-Family Pipeline (Mistral-7B -> Qwen3-1.7B -> Qwen3-4B) | Channel / KV Budget | 3-Bench Avg Acc (%) | vs. No-Comm. (83.3%) | Latency (s) |
|---|---|---|---|---|
Single-Agent No-Communication (Qwen3-4B) | None, 0% KV | 83.3 ± 0.2 | 0.0 pp | 11.1 |
| Cross-Family Text Communication | Natural language | 81.2 ± 0.3 | -2.1 pp | 29.8 |
| Cross-Family Adapted KVComm [2] (+ interp. & ridge) | 1D layer KV, 14% | 69.4 ± 0.5 | -13.9 pp | 9.1 |
| Cross-Family Adapted C2C† [1] (2-layer MLP, 2k steps) | Full KV, 100% | 74.6 ± 0.4 | -8.7 pp | 14.2 |
| Cross-Family GCS (Ours, CKA + Ridge) | 3D Gricean KV, 12% | 78.1 ± 0.4 | -5.2 pp | 8.6 |
We observe heavy losses across all cross-family KV baselines. Standalone Mistral-7B-Instruct-v0.3 scores 40.0% in Table 2 vs. 86.4% for Qwen3-8B, so lossless Cross-Family Text already dips from 83.9% to 81.2%. On top of that, Adapted KVComm [2] drops to 69.4%, and even the adapted 100%-payload C2C† [1] MLP projector drops to 74.6%, or 8.7 points below No-Communication.
Even in this harsh setting, GCS holds up much better at 78.1% in 8.6s, beating Adapted KVComm by +8.7 points and C2C† by +3.5 points at 1.65× lower latency. We export only a 12% total KV slice restricted to salient positions on the public calibration hidden states and receiver role layers, so much less projection noise enters receiver attention. We plan to explore tokenizer-agnostic cross-family alignment in future work.
Response to Reviewer MnYf (Part 3/3)
Response to Reviewer MnYf (Part 3/3)
W1 (Framing & Baselines): Gricean Framing and Comparison to 1D KV Pruning
Thank you for the candid feedback on both our framing and our baselines. As noted in Section 3.1, we use Grice's principle [14] strictly as an intuitive lens for grouping our three filters and it inspired us to think about how to borrow this high-level principle to design our framework. We added citations and discussion of H2O [7] and SnapKV [8] in Section 2, Section 4.1, and Appendix C.6, and contrasted single-agent eviction in Section 3.1. Under the same sender rollout protocol, we also benchmarked GCS directly against both methods across all 5 backbones and 3 benchmarks in Table 2 and Table 9:
| Method / Selection Rule ( Rollout Protocol) | KV Budget | 5-BB Mean Acc (%) | 5-BB Time (s) | Qwen3-4B Acc (%) | 1.7B GSM8K (%) |
|---|---|---|---|---|---|
No-Communication (Single Agent) | 0% | 8.8 | |||
Table 3 Single: Module III (Token Top- Only) | 50% | -- | -- | -- | |
| Adapted H2O [7] (1D Heavy-Hitter Eviction) | 12% | 10.1 | |||
| Adapted SnapKV [8] (1D Window Pooling) | 12% | 9.9 | |||
| LatentMAS [6] (Unfiltered Rollout Cache) | 100% | 14.8 | |||
| GCS (Ours, Head + Layer + Token Scoping) | 12% | 9.7 |
We find that 1D token eviction falls well short of GCS across agents. At the same 12% budget over all 15 settings, Adapted H2O [7] reaches 69.8% and Adapted SnapKV [8] reaches 70.3%. Both trail GCS at 72.6% by 2.3 to 2.8 points on average, and by 2.6 to 3.0 points on GSM8K, reaching 4.1 to 4.7 points on Qwen3-1.7B. Our module ablation in Table 3 of Section 4.2 shows this failure mode directly. When we run Top- token selection alone at a 50% budget in Single: Module III, Qwen3-1.7B GSM8K accuracy drops to 70.2%, which is 6.2 points below No-Communication at 76.4%.
We also verified why head and layer scoping matter for multi-agent handoffs. In single-agent inference [7, 8], one model prunes its cache to continue its own decoding under a single role. Across two agents with distinct roles, sending all heads and layers injects the sender's private heads and off-role layers into the receiver. That is why 100%-KV LatentMAS [6] trails 12%-budget GCS at 71.4% vs. 72.6%, and why random head or layer masks lose 2.1 to 2.5 points in Table 6 of Appendix C.5. We filter public heads and role-matched layers before token selection, so we avoid this cross-agent interference.
[1] T. Fu et al. Cache-to-Cache: Direct Semantic Communication Between Large Language Models. ICLR'26.
[2] X. Shi et al. KVComm: Enabling Efficient LLM Communication through Selective KV Sharing. ICLR'26.
[3] H. Ye et al. KVCOMM: Online Cross-context KV-cache Communication for Efficient LLM-based Multi-agent Systems. NeurIPS'25.
[4] Y. Zheng et al. Thought Communication in Multiagent Collaboration. NeurIPS'25.
[5] Y. Liu et al. DroidSpeak: KV Cache Sharing Across Fine-tuned Model Variants. NSDI'26.
[6] J. Zou et al. Latent Collaboration in Multi-Agent Systems. ICML'26.
[7] Z. Zhang et al. H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models. NeurIPS'23.
[8] Y. Li et al. SnapKV: LLM Knows What You Are Looking For Before Generation. NeurIPS'24.
[9] G. Tzachristas et al. A Mathematical Theory of Top-k Sparse Attention via Total Variation Distance. arXiv'25.
[10] S. Kornblith et al. Similarity of Neural Network Representations Revisited. ICML'19.
[11] Y. Liu et al. CacheGen: KV Cache Compression and Streaming for Fast Large Language Model Serving. SIGCOMM'24.
[12] A. Yang et al. Qwen3 Technical Report. arXiv'25.
[13] D. Guo et al. DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning. arXiv'25.
[14] H. P. Grice. Logic and Conversation. Syntax and Semantics'75.
Response to Reviewer MnYf (Part 1/3)
Response to Reviewer MnYf (Part 1/3)
We sincerely thank Reviewer MnYf for the thoughtful and constructive review, as well as for recognizing the merits of our engineering design, particularly latent feature probing, ridge alignment, and RoPE position continuity. These comments have been valuable in helping us strengthen the empirical evidence and clarify the presentation of our work. In response, we have added a detailed cross-network latency analysis and further clarified the role and limitations of our theoretical framework. We address each concern below and highlight the corresponding revisions to the manuscript.
W2 (Network) & Q1: KV Transfer Latency in Distributed Multi-Node Deployments
Network transfer cost is a real concern in multi-node serving, and we appreciate you highlighting it. We estimated 3-agent distributed runtime on
Qwen3-4Bacross NVLink, 100GbE, 10GbE, and 1GbE WAN by combining measured single-node execution with socket- and NCCL-verified link transfer costs. We reported these results inTable 4aofSection 4.4andTable 11ofAppendix C.8:Qwen3-4B, 2 Handoffs)fp16)fp16, 12%)int8int4We broke down inference and wire time across links using
Table 7. Text messages are under0.1 MiB, yet sequential token decoding takes34.50s, so Text Communication still takes44.07son every link. Unfiltered Full-KV sends90.0 MiB, or94.37 MB, per hop and adds1.51sover two hops on 1GbE WAN, reaching19.51s. With GCS at 12%, we shrink each hop by 8.5× to10.63 MiB. Two-hop transfer then adds only17.8 mson 10GbE and178 mson 1GbE WAN, giving12.61stotal and a 3.50× speedup over text. Even if Agent 2 re-forwards Agent 1's slice for3Ptotal traffic, 1GbE time rises by just89 msto12.70sversus20.26sfor Full-KV. When we quantize our 12% slice toint4at2.66 MiBper hop, two-hop 1GbE transfer takes44.6 msand total latency drops to11.44sat86.6%accuracy.W1 (Theory): Role of the Problem Formulation and Analytical Bounds
We added
Remark E.5on softmax renormalization and checked our bound terms over 500 traces inSection 4.2andAppendix C.7, while noting our focus is algorithm and system design:Qwen3-4B, 500 Traces)Our theory serves as a clean problem formulation and design guide. Prior multi-agent latent works [1–5] study learned fusion [1], 1D layer-count budgets [2], prefix reuse [3, 5], or latent identifiability [4], without bounding 3D KV pruning error. LatentMAS [6] analyzes only 100% unconstrained latent steps, while single-agent eviction works [7–9] operate within one model and role without cross-agent query shift. To our knowledge, our formalization including
Equation 1inSection 3.3is the first unified formulation of budgeted cross-agent KV sharing. It separates capacity vs. discrete text inTheorem 3.3, joint-memory readout shift inTheorem 3.5, and the sender-receiver proxy gap ∥p−s∥1 inTheorem 3.6. That proxy term directly motivated our Wa-aligned probing inAppendix C.1–C.2, which cuts ∥p−sˉ∥1 from0.486to0.142above.