acceptodds
Under review as a conference paper at ICLR 2027

Gricean Cache Sharing for Multi-Agent Collaboration

Abstract

Existing multi-agent systems rely heavily on natural language for coordination. While flexible, text imposes a strict sequential bottleneck and often transmits redundant tokens, leading to high latency and noise interference. To bypass this limit without incurring the redundancy of naïve broadcasting, we introduce Gricean Cache Sharing, a controllable thought sharing protocol grounded in Grice’s Cooperative Principle. Instead of treating the thought as a monolithic block, our framework reformulates inter-agent communication as a constrained optimization problem. We design a transparent interface that exports only carefully selected thoughts, and give a mechanism analysis in which Top- saliency minimizes discarded attention mass for fixed scores and a per-block budget. Empirically, our method reduces latency by on average (up to ) while matching or exceeding the accuracy of text baselines on 12 of 15 model–benchmark settings. Ablation studies show that the three modules are complementary, with the full combination outperforming every single- and dual-module variant, while the communication budget acts as a smooth accuracy–latency control knob.

Primary area:
foundation or frontier models, including LLMs
Keywords:
efficient LLM inference, multi-agent LLM systems, inter-agent communication

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_MnYfOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
3(+1)
Read review
Paper summary

This paper proposes Gricean Cache Sharing (GCS), an inter-agent communication protocol that replaces natural language text exchange with selective KV-cache. Guided metaphorically by Grice’s Cooperative Principle, GCS sparsifies the transmitted cache along three axes: attention heads, layer depth, and token positions. Evaluated on 5 models across 3 benchmarks (ARC-C, GSM8K, HumanEval+), GCS achieves a 3.3× average speedup over text-mediated collaboration while matching accuracy on single-GPU homogeneous setups.

Critical strengths
  1. The overall idea is nice. Replacing NL generation with a sparse KV-cache can boost efficiency, and the methods are documented in detail.

  2. Rigorous engineering on latent feature probing. The authors do solve essential latent communication issues, such as mitigating distribution shift by using a closed-form ridge regression alignment map and managing RoPE position ID continuity across agents.

Critical weaknesses
  1. Trivial and overpacked theoretical claim. The theoretical results are mostly cliche; Theorem 3.6 spends an entire page in the appendix to prove k largest numbers maximize a sum. I wonder if it is necessary to use so many theorems in the paper. Besides, mapping Grice’s Cooperative Principle to standard pruning techniques is pure rhetoric. The overall method is essentially a 3D cache pruning pipeline where the "Maxims" barely add any substance.

  2. The architecture requires all model beusing exactly the same architecture. And the methods are running on one hardware where memory copy latency is negligible, whereas in realistic multi-agent cases, where distributed deployment may introduce heavy KV cache transmission cost, making it impractical.

Questions for authors

How will the method handle the network transfer latency of 10–90 MiB KV-caches in distributed multi-node settings?

3 comments
AuthorsOpenReview ↗

Response to Reviewer MnYf (Part 1/3)

Response to Reviewer MnYf (Part 1/3)

We sincerely thank Reviewer MnYf for the thoughtful and constructive review, as well as for recognizing the merits of our engineering design, particularly latent feature probing, ridge alignment, and RoPE position continuity. These comments have been valuable in helping us strengthen the empirical evidence and clarify the presentation of our work. In response, we have added a detailed cross-network latency analysis and further clarified the role and limitations of our theoretical framework. We address each concern below and highlight the corresponding revisions to the manuscript.

W2 (Network) & Q1: KV Transfer Latency in Distributed Multi-Node Deployments

Network transfer cost is a real concern in multi-node serving, and we appreciate you highlighting it. We estimated 3-agent distributed runtime on Qwen3-4B across NVLink, 100GbE, 10GbE, and 1GbE WAN by combining measured single-node execution with socket- and NCCL-verified link transfer costs. We reported these results in Table 4a of Section 4.4 and Table 11 of Appendix C.8:

Protocol (Qwen3-4B, 2 Handoffs)Payload/HopNVLink (300 GB/s)100GbE (12.5 GB/s)10GbE (1.25 GB/s)1GbE WAN (125 MB/s)Acc (%)
Text Communication<0.1 MiB<0.1\text{ MiB}44.07s44.07\text{s} / 1.00×1.00\times44.07s44.07\text{s} / 1.00×1.00\times44.07s44.07\text{s} / 1.00×1.00\times44.07s44.07\text{s} / 1.00×1.00\times85.5
Full-KV Relay (fp16)90.00 MiB90.00\text{ MiB}18.00s18.00\text{s} / 2.45×2.45\times18.01s18.01\text{s} / 2.45×2.45\times18.15s18.15\text{s} / 2.43×2.43\times19.51s19.51\text{s} / 2.26×2.26\times88.5
GCS Default (fp16, 12%)10.63 MiB10.63\text{ MiB}12.43s12.43\text{s} / 3.55×3.55\times12.43s12.43\text{s} / 3.55×3.55\times12.45s12.45\text{s} / 3.54×3.54\times12.61s12.61\text{s} / 3.50×3.50\times87.7
GCS + int85.31 MiB5.31\text{ MiB}11.90s11.90\text{s} / 3.70×3.70\times11.90s11.90\text{s} / 3.70×3.70\times11.91s11.91\text{s} / 3.70×3.70\times11.99s11.99\text{s} / 3.68×3.68\times87.3
GCS + int42.66 MiB2.66\text{ MiB}11.40s11.40\text{s} / 3.87×3.87\times11.40s11.40\text{s} / 3.87×3.87\times11.40s11.40\text{s} / 3.87×3.87\times11.44s11.44\text{s} / 3.85×3.85\times86.6

We broke down inference and wire time across links using Table 7. Text messages are under 0.1 MiB, yet sequential token decoding takes 34.50s, so Text Communication still takes 44.07s on every link. Unfiltered Full-KV sends 90.0 MiB, or 94.37 MB, per hop and adds 1.51s over two hops on 1GbE WAN, reaching 19.51s. With GCS at 12%, we shrink each hop by 8.5×8.5\times to 10.63 MiB. Two-hop transfer then adds only 17.8 ms on 10GbE and 178 ms on 1GbE WAN, giving 12.61s total and a 3.50×3.50\times speedup over text. Even if Agent 2 re-forwards Agent 1's slice for 3P total traffic, 1GbE time rises by just 89 ms to 12.70s versus 20.26s for Full-KV. When we quantize our 12% slice to int4 at 2.66 MiB per hop, two-hop 1GbE transfer takes 44.6 ms and total latency drops to 11.44s at 86.6% accuracy.

W1 (Theory): Role of the Problem Formulation and Analytical Bounds

We added Remark E.5 on softmax renormalization and checked our bound terms over 500 traces in Section 4.2 and Appendix C.7, while noting our focus is algorithm and system design:

Probe Policy (Qwen3-4B, 500 Traces)∥p−sˉ∥1↓\|p - \bar{s}\|_1 \downarrowTop-kk Ovlp ↑\uparrowMean Γ^\hat{\Gamma}3-Task Acc (%)
Unaligned rollout, ϕ=50\phi=50, no WaW_a0.486±0.0520.486 \pm 0.05256.4%2.1886.1±0.386.1 \pm 0.3
GCS Default, ϕ=50\phi=50, with WaW_a0.142±0.0310.142 \pm 0.03184.7%1.8487.7±0.287.7 \pm 0.2
Oracle Receiver S⋆=Top-k(p)S^\star = \text{Top-}k(p)0.000100.0%1.7688.0±0.288.0 \pm 0.2

Our theory serves as a clean problem formulation and design guide. Prior multi-agent latent works [1–5] study learned fusion [1], 1D layer-count budgets [2], prefix reuse [3, 5], or latent identifiability [4], without bounding 3D KV pruning error. LatentMAS [6] analyzes only 100% unconstrained latent steps, while single-agent eviction works [7–9] operate within one model and role without cross-agent query shift. To our knowledge, our formalization including Equation 1 in Section 3.3 is the first unified formulation of budgeted cross-agent KV sharing. It separates capacity vs. discrete text in Theorem 3.3, joint-memory readout shift in Theorem 3.5, and the sender-receiver proxy gap ∥p−s^∥1\|p - \widehat{s}\|_1 in Theorem 3.6. That proxy term directly motivated our WaW_a-aligned probing in Appendix C.1–C.2, which cuts ∥p−sˉ∥1\|p - \bar{s}\|_1 from 0.486 to 0.142 above.

AuthorsOpenReview ↗

Response to Reviewer MnYf (Part 2/3)

Response to Reviewer MnYf (Part 2/3)

W2 (Architecture): Same-Family vs. Cross-Family Heterogeneous Collaboration

We share your view that cross-architecture KV sharing is a hard open problem. We added a mixed-size Qwen3-8B -> 1.7B -> 4B study to Table 4b of Section 4.4 and Table 12 of Appendix C.9. Below, we also report an exploratory cross-family Mistral-7B -> Qwen3-1.7B -> Qwen3-4B experiment.

First, we tested heterogeneous sizes within Qwen3, with layer counts of (36, 28, 36) and hidden dimensions of (4096, 2048, 2560):

Same-Family Heterogeneous (Qwen3-8B -> 1.7B -> 4B)Channel / KV3-Bench Acc (%)Latency (s)Speedup
Heterogeneous Text CommunicationText83.9 ± 0.336.51.00×
Heterogeneous C2C† [1] (Trained MLP)100%85.8 ± 0.316.42.23×
Heterogeneous GCS (Ours, CKA + Ridge)12%86.4 ± 0.39.83.72×

In Appendix C.9, we extend Module II with 100 × 100 Linear CKA [10] Gram matrices on the calibration set to match receiver role layers with sender layers across depths and widths. On the public calibration hidden states, we fit closed-form ridge adapters for keys and values per matched head block, using a ridge regularization coefficient of 0.001. Without gradient training, we reach 86.4 ± 0.3% in 9.8s, beating both Heterogeneous Text and trained C2C† [1], and running faster than standalone Qwen3-4B at 11.1s. In Table 12, we also confirm gains on 4-agent chains at 88.3%, 2-round debate at 88.1%, star graphs at 88.0%, and SWE-bench Lite at 28.3% vs. 26.7% text.

Second, we examined cross-family KV sharing, where lightweight alignment remains an open problem. While C2C [1] trains end-to-end cross-family fusers, training-free KV protocols [2, 3, 6] and serving systems like DroidSpeak [5] and CacheGen [11] require a shared model family. In fact, [5] shows that even fine-tuning drift on identical architectures hurts KV reuse unless sensitive layers are recomputed. Distinct families use incompatible tokenizers such as 32k SentencePiece in Mistral-7B-Instruct-v0.3 vs. 152k BPE in Qwen3 [12], unaligned KV subspaces, and QK-Norm in Qwen3 [12] alongside diverse attention layouts [13]. Tokenizer mismatch breaks token-to-token KV alignment, and adapter errors in keys can be amplified inside receiver attention softmax across layers.

We tested this directly by swapping the Qwen3-8B Planner for Mistral-7B-Instruct-v0.3 with linear token-position interpolation at the Mistral -> Qwen handoff:

Cross-Family Pipeline (Mistral-7B -> Qwen3-1.7B -> Qwen3-4B)Channel / KV Budget3-Bench Avg Acc (%)vs. No-Comm. (83.3%)Latency (s)
Single-Agent No-Communication (Qwen3-4B)None, 0% KV83.3 ± 0.20.0 pp11.1
Cross-Family Text CommunicationNatural language81.2 ± 0.3-2.1 pp29.8
Cross-Family Adapted KVComm [2] (+ interp. & ridge)1D layer KV, 14%69.4 ± 0.5-13.9 pp9.1
Cross-Family Adapted C2C† [1] (2-layer MLP, 2k steps)Full KV, 100%74.6 ± 0.4-8.7 pp14.2
Cross-Family GCS (Ours, CKA + Ridge)3D Gricean KV, 12%78.1 ± 0.4-5.2 pp8.6

We observe heavy losses across all cross-family KV baselines. Standalone Mistral-7B-Instruct-v0.3 scores 40.0% in Table 2 vs. 86.4% for Qwen3-8B, so lossless Cross-Family Text already dips from 83.9% to 81.2%. On top of that, Adapted KVComm [2] drops to 69.4%, and even the adapted 100%-payload C2C† [1] MLP projector drops to 74.6%, or 8.7 points below No-Communication.

Even in this harsh setting, GCS holds up much better at 78.1% in 8.6s, beating Adapted KVComm by +8.7 points and C2C† by +3.5 points at 1.65× lower latency. We export only a 12% total KV slice restricted to salient positions on the public calibration hidden states and receiver role layers, so much less projection noise enters receiver attention. We plan to explore tokenizer-agnostic cross-family alignment in future work.

AuthorsOpenReview ↗

Response to Reviewer MnYf (Part 3/3)

Response to Reviewer MnYf (Part 3/3)

W1 (Framing & Baselines): Gricean Framing and Comparison to 1D KV Pruning

Thank you for the candid feedback on both our framing and our baselines. As noted in Section 3.1, we use Grice's principle [14] strictly as an intuitive lens for grouping our three filters and it inspired us to think about how to borrow this high-level principle to design our framework. We added citations and discussion of H2O [7] and SnapKV [8] in Section 2, Section 4.1, and Appendix C.6, and contrasted single-agent eviction in Section 3.1. Under the same ϕ=50\phi=50 sender rollout protocol, we also benchmarked GCS directly against both methods across all 5 backbones and 3 benchmarks in Table 2 and Table 9:

Method / Selection Rule (ϕ=50\phi=50 Rollout Protocol)KV Budget5-BB Mean Acc (%)5-BB Time (s)Qwen3-4B Acc (%)1.7B GSM8K (%)
No-Communication (Single Agent)0%68.5±0.268.5 \pm 0.28.883.3±0.283.3 \pm 0.276.4±0.376.4 \pm 0.3
Table 3 Single: Module III (Token Top-kk Only)50%------70.2±0.470.2 \pm 0.4
Adapted H2O [7] (1D Heavy-Hitter Eviction)12%69.8±0.369.8 \pm 0.310.184.8±0.384.8 \pm 0.379.2±0.479.2 \pm 0.4
Adapted SnapKV [8] (1D Window Pooling)12%70.3±0.370.3 \pm 0.39.985.3±0.385.3 \pm 0.379.8±0.379.8 \pm 0.3
LatentMAS [6] (Unfiltered Rollout Cache)100%71.4±0.371.4 \pm 0.314.886.7±0.286.7 \pm 0.282.9±0.382.9 \pm 0.3
GCS (Ours, Head + Layer + Token Scoping)12%72.6±0.272.6 \pm 0.29.787.7±0.287.7 \pm 0.283.9±0.283.9 \pm 0.2

We find that 1D token eviction falls well short of GCS across agents. At the same 12% budget over all 15 settings, Adapted H2O [7] reaches 69.8% and Adapted SnapKV [8] reaches 70.3%. Both trail GCS at 72.6% by 2.3 to 2.8 points on average, and by 2.6 to 3.0 points on GSM8K, reaching 4.1 to 4.7 points on Qwen3-1.7B. Our module ablation in Table 3 of Section 4.2 shows this failure mode directly. When we run Top-kk token selection alone at a 50% budget in Single: Module III, Qwen3-1.7B GSM8K accuracy drops to 70.2%, which is 6.2 points below No-Communication at 76.4%.

We also verified why head and layer scoping matter for multi-agent handoffs. In single-agent inference [7, 8], one model prunes its cache to continue its own decoding under a single role. Across two agents with distinct roles, sending all heads and layers injects the sender's private heads Hpri\mathcal{H}_{\text{pri}} and off-role layers into the receiver. That is why 100%-KV LatentMAS [6] trails 12%-budget GCS at 71.4% vs. 72.6%, and why random head or layer masks lose 2.1 to 2.5 points in Table 6 of Appendix C.5. We filter public heads and role-matched layers before token selection, so we avoid this cross-agent interference.

[1] T. Fu et al. Cache-to-Cache: Direct Semantic Communication Between Large Language Models. ICLR'26.

[2] X. Shi et al. KVComm: Enabling Efficient LLM Communication through Selective KV Sharing. ICLR'26.

[3] H. Ye et al. KVCOMM: Online Cross-context KV-cache Communication for Efficient LLM-based Multi-agent Systems. NeurIPS'25.

[4] Y. Zheng et al. Thought Communication in Multiagent Collaboration. NeurIPS'25.

[5] Y. Liu et al. DroidSpeak: KV Cache Sharing Across Fine-tuned Model Variants. NSDI'26.

[6] J. Zou et al. Latent Collaboration in Multi-Agent Systems. ICML'26.

[7] Z. Zhang et al. H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models. NeurIPS'23.

[8] Y. Li et al. SnapKV: LLM Knows What You Are Looking For Before Generation. NeurIPS'24.

[9] G. Tzachristas et al. A Mathematical Theory of Top-k Sparse Attention via Total Variation Distance. arXiv'25.

[10] S. Kornblith et al. Similarity of Neural Network Representations Revisited. ICML'19.

[11] Y. Liu et al. CacheGen: KV Cache Compression and Streaming for Fast Large Language Model Serving. SIGCOMM'24.

[12] A. Yang et al. Qwen3 Technical Report. arXiv'25.

[13] D. Guo et al. DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning. arXiv'25.

[14] H. P. Grice. Logic and Conversation. Syntax and Semantics'75.

Reviewer_va5AOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper presents Gricean Cache Sharing (GCS) a method for sharing KV cache values across agents for a given task. Assuming a setup with an agent pipeline of the form Planner→Solver→Verifier, the authors provide a method to share parts of the KV cache across the agents. Their method loosely uses Grice's cooperative principle to choose at which heads, layers and token positions the KV cache will be shared across agents.

Based on this method, the authors propose a theoretical framework to analyze their method. Using this framework, they show a separation result between cache sharing and discrete text-based communication for a synthetic "cache reproduction task. They give a lower bound on the number of tokens needed to achieve this and show that under that bound KV cache sharing can still obtain zero loss. Second, they show (under some assumptions) that the attention mass which is lost by truncating the KV cache in GCS leads to small deviation from performance when appending the entire KV cache. Finally, they use this insight to provide an error bound on the top kk token position selection heuristic they use.

Empirically, the authors validate their method across multiple 1B to 8B backbones across known reasoning benchmarks, and find their method is competitive and generally better in terms of latency.

Critical strengths
  • The proposed method is simple, training-free and easily adaptable to different backbones. Compared to other approaches, there is no need for training a projector (as in C2C) and generally the selection of the parts of the KV cache depends on rather simple rules which are computationally cheap

  • The empirical evaluation is comprehensive. The authors evaluate five backbones across 2 model families and three task types. The chosen baselines seem fair to me (they evaluate against similar KV cache sharing methods). Most key parameters of the method are ablated over and the latency evaluation/computation is fair.

Critical weaknesses
  • There are no error bars on the main results; Table 2 seems to report single seed results. This undermines the statistical validity of the results in Table 2. Especially when considering the accuracy values reported, I believe running all baselines for a couple more seeds would make the results more convincing

  • Although the KV cache scheme is quite general, the method is only tested on a Planner→Solver→Verifier scheme. It would have been interesting to see how it fairs on different agent topologies/roles.

Questions for authors
  • Do the authors have any idea how their method can be adapted to mismatching backbones? This could be interesting if one wants e.g. a bigger planning model and a smaller solver.
  • Is using the sender attention scores instead of the receiver scores a good proxy? It seems there are no ablations showing how faithful of a proxy it would be.
4 comments
AuthorsOpenReview ↗

Response to Reviewer va5A (Part 1/2)

Response to Reviewer va5A (Part 1/2)

We thank Reviewer va5A for the supportive review and for noting the simplicity, training-free nature, and broad evaluation of GCS. Below, we describe the new experiments and error bars we added in response to your comments.

W1: Multi-Seed Error Bars Across Main Results, Baselines, and Ablations

Thank you for this suggestion; multi-seed error bars are indeed important when accuracy gaps are around 1 to 2 points. We ran 3 evaluation seeds across all 5 backbones and 3 benchmarks in Table 2 of Section 4.1 and Table 9 of Appendix C.6. Each seed varies the 100-prompt calibration draw Dcal\mathcal{D}_{\text{cal}} and role-prompt exemplar order. We also ran an 8-seed selector sweep for the module ablation in Table 3 of Section 4.2.

Method / ProtocolKV5-BB Mean Acc (%)5-BB Time (s)Qwen3-4B Acc (%)Qwen3-4B Time (s)1.7B GSM8K (%)
Text CommunicationText71.0±0.371.0 \pm 0.332.3±0.232.3 \pm 0.285.5±0.385.5 \pm 0.344.182.6±0.382.6 \pm 0.3
Self-Consistency [4], k=3k=30%70.4±0.370.4 \pm 0.316.2±0.116.2 \pm 0.185.6±0.385.6 \pm 0.319.879.5±0.379.5 \pm 0.3
Adapted H2O [2] / SnapKV [3]12%69.869.8 / 70.3±0.370.3 \pm 0.310.110.1 / 9.99.984.884.8 / 85.3±0.385.3 \pm 0.313.0 / 12.879.279.2 / 79.879.8
KVComm [6]13–14%71.0±0.371.0 \pm 0.310.4±0.110.4 \pm 0.186.2±0.386.2 \pm 0.313.382.2±0.382.2 \pm 0.3
LatentMAS [1], ϕ=50\phi=50100%71.4±0.371.4 \pm 0.314.8±0.114.8 \pm 0.186.7±0.286.7 \pm 0.218.582.9±0.382.9 \pm 0.3
C2C$^\dagger$ [5]100%72.3±0.372.3 \pm 0.315.2±0.215.2 \pm 0.288.0±0.388.0 \pm 0.319.484.6±0.384.6 \pm 0.3
GCS (Ours)12%72.6±0.272.6 \pm 0.29.7±0.19.7 \pm 0.187.7±0.287.7 \pm 0.212.483.9±0.283.9 \pm 0.2

Across all methods in Table 2, standard deviations stay at most 0.3% on benchmark averages, 0.5% on ARC-C and GSM8K, 0.6% on HumanEval+, and 0.2s on latency. On the 5-backbone mean, we reach 72.6 ± 0.2% in 9.7 ± 0.1s, well ahead of Text, KVComm [6], LatentMAS [1], and adapted H2O/SnapKV [2, 3]. We also match or edge out trained C2C$^\dagger$ [5] while running 1.57×1.57\times faster. In Table 3, our 8-seed mean of 83.3 ± 0.4% beats every Single row at 78.5%–80.8% and every Dual row at 81.0%–81.7%.

W2: Diverse Multi-Agent Topologies and Long-Horizon SWE-bench Lite

We appreciate the push to test beyond a single three-agent chain. We evaluated GCS on three new topologies on Qwen3-4B and on 300 GitHub issues from SWE-bench Lite with Qwen3-8B. We added these results to Table 4b of Section 4.4 and Table 12 of Appendix C.9:

System Setting / TopologyBackbone / BenchmarkText Acc (%)Text (s)Full-KV$^\ddagger$ (%)Full-KV (s)GCS 12% (%)GCS (s)Speedup
3-Agent ChainQwen3-4B, 3-Bench Avg85.544.188.518.087.712.43.55×3.55\times
4-Agent Plan->Crit->Solv->VerQwen3-4B, 3-Bench Avg85.958.488.624.188.315.83.70×3.70\times
3-Agent 2-Round DebateQwen3-4B, 3-Bench Avg86.179.288.433.588.121.43.70×3.70\times
Star 2 Solvers -> 1 VerifierQwen3-4B, 3-Bench Avg85.849.688.220.388.013.93.57×3.57\times
SWE-bench Lite, T≈4,200T\approx 4{,}200Qwen3-8B, 300 Issues26.7148.627.368.428.341.23.61×3.61\times

Across all three new topologies on Qwen3-4B, we beat Text Communication by 2.0 to 2.4 points and cut latency by 3.57×3.57\times to 3.70×3.70\times. In the star graph, two Solver agents send their 12% slices with offset RoPE position_ids as in Appendix C.1 to one Verifier, reaching 88.0% in 13.9s. On SWE-bench Lite, repository contexts average T≈4,200T\approx 4{,}200 tokens, so GCS shrinks each hop from ~591 MiB to ~70 MiB. We resolve 28.3% of issues in 41.2s of inference and communication time, excluding external Docker test execution, beating both Text at 26.7% and Full-KV at 27.3%.

AuthorsOpenReview ↗

Response to Reviewer va5A (Part 2/2)

Response to Reviewer va5A (Part 2/2)

Q1: Adapting GCS to Mismatching Backbones (Larger Planner + Smaller Solver)

This is a very practical deployment pattern, and we are glad you brought it up. We built a mixed-size Qwen3-8B Planner -> Qwen3-1.7B Solver -> Qwen3-4B Verifier pipeline and added its results to Table 4b of Section 4.4 and Table 12 of Appendix C.9:

Heterogeneous Pipeline (Qwen3-8B -> 1.7B -> 4B)KV Budget3-Bench Avg Acc (%)Latency (s)Speedup vs. Text
Homogeneous Qwen3-1.7B GCS (Ref.)12%79.4±0.279.4 \pm 0.211.6--
Heterogeneous Text CommunicationText83.9±0.383.9 \pm 0.336.51.00×1.00\times
Heterogeneous C2C$^\dagger$ [5] (Trained MLP)100%85.8±0.385.8 \pm 0.316.42.23×2.23\times
Heterogeneous GCS (Ours, CKA + Ridge)12%86.4±0.386.4 \pm 0.39.83.72×3.72\times

We adapt GCS across these three sizes with L∈36,28,36L \in \\{36, 28, 36\\} and d∈4096,2048,2560d \in \\{4096, 2048, 2560\\} without any gradient training. Modules I and III run inside the sender as usual. In Appendix C.9, we extend Module II with 100×100100\times 100 Linear CKA [7] Gram matrices on Dcal\mathcal{D}_{\text{cal}} to match receiver role layers in L(ri+1)\mathcal{L}(r_{i+1}) with sender layers across widths and depths. All three Qwen3 models share nkv=8n_{\text{kv}}=8 and dh=128d_h=128. On Dcal\mathcal{D}_{\text{cal}}, we fit a closed-form ridge map (WK,WV)(W_K, W_V) per matched head block at λridge=10−3\lambda_{\text{ridge}} = 10^{-3} on Hpub\mathcal{H}_{\text{pub}}. With this setup, we reach 86.4% in 9.8s, beating Heterogeneous Text by 2.5 points at a 3.72×3.72\times speedup and beating trained 100%-payload C2C$^\dagger$ [5] at 85.8% in 16.4s. We also gain 7.0 points over homogeneous 1.7B while running 2.6s faster than homogeneous 4B at 12.4s.

Q2: Faithfulness of Sender Proxy Scores sˉ\bar{s} Relative to Receiver Scores pp

Thanks for raising this question. We tested how closely our sender scores sˉ\bar{s} track oracle receiver attention pp across 500 instances on Qwen3-4B and Qwen3-1.7B. We added this comparison to Section 4.2 and Table 10 of Appendix C.7:

BackboneSender Proxy Policy∥p−sˉ∥1↓\\\|p - \bar{s}\\\|_1 \downarrowTop-kk Ovlp ↑\uparrowSpearman ρs↑\rho_s \uparrow3-Task Acc (%)
Qwen3-4BUniform weights, sˉt=1/Ti\bar{s}_t = 1/T_i0.612±0.0580.612 \pm 0.05841.2%--85.1±0.385.1 \pm 0.3
Qwen3-4BSender prompt-only, ϕ=0\phi=00.348±0.0450.348 \pm 0.04566.8%0.6486.5±0.386.5 \pm 0.3
Qwen3-4BUnaligned rollout, ϕ=50\phi=50, no WaW_a0.486±0.0520.486 \pm 0.05256.4%0.4986.1±0.386.1 \pm 0.3
Qwen3-4BAligned rollout, ϕ=10\phi=10, with WaW_a0.214±0.0380.214 \pm 0.03878.3%0.8187.2±0.287.2 \pm 0.2
Qwen3-4BGCS Default, ϕ=50\phi=50, with WaW_a0.142±0.0310.142 \pm 0.03184.7%0.8987.7±0.287.7 \pm 0.2
Qwen3-4BOracle Receiver S⋆=Top-k(p)S^\star = \text{Top-}k(p)0.000100.0%1.0088.0±0.288.0 \pm 0.2
Qwen3-1.7BGCS Default, ϕ=50\phi=50, with WaW_a0.151±0.0340.151 \pm 0.03483.9%0.8879.4±0.279.4 \pm 0.2
Qwen3-1.7BOracle Receiver S⋆=Top-k(p)S^\star = \text{Top-}k(p)0.000100.0%1.0079.7±0.279.7 \pm 0.2

We find that both latent rollouts and the WaW_a map are required for a faithful proxy. Prompt-only attention at ϕ=0\phi=0 misses forward reasoning queries and gets only 66.8% overlap, while ϕ=50\phi=50 without WaW_a drifts to 56.4% overlap. With WaW_a-aligned rollouts at ϕ=50\phi=50, we reach 84.7% Top-kk overlap and Spearman ρs=0.89\rho_s = 0.89 on Qwen3-4B, coming within 0.3 points of the two-pass receiver oracle at 87.7% vs. 88.0%.

References

[1] J. Zou et al. Latent Collaboration in Multi-Agent Systems. ICML'26.

[2] Z. Zhang et al. H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models. NeurIPS'23.

[3] Y. Li et al. SnapKV: LLM Knows What You Are Looking For Before Generation. NeurIPS'24.

[4] X. Wang et al. Self-Consistency Improves Chain of Thought Reasoning in Language Models. ICLR'23.

[5] T. Fu et al. Cache-to-Cache: Direct Semantic Communication Between Large Language Models. ICLR'26.

[6] X. Shi et al. KVComm: Enabling Efficient LLM Communication through Selective KV Sharing. ICLR'26.

[7] S. Kornblith et al. Similarity of Neural Network Representations Revisited. ICML'19.

Reviewer_va5AOpenReview ↗

Official Reviewer Comment

Thank you for your detailed response and answer to my questions. I will be maintaining my positive score

We sincerely appreciate your constructive feedback and your help in improving our submission.

Reviewer_njhcOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

Gricean Cache Sharing (GCS) lets same-backbone agents in a sequential pipeline share sparsified KV-cache slices instead of text. It uses three filters: magnitude-ranked public heads, CKA-selected role-specific layers, and Top-k attention-saliency tokens estimated from latent probe steps. The theory includes a text-vs-KV counting separation, a tail-mass bound on excess loss, and Top-k optimality. Across 5 backbones and 3 benchmarks, it reports a 3.3× speedup over text and matches or beats text accuracy in 12 of 15 settings.

Critical strengths
  1. A practical, tunable middle ground between text and full-cache relay. The method is training-free and inspectable, and its budget ρ\rho trades accuracy for payload smoothly. It comes within 0.8 points of Full-KV at about 8.5×8.5\times smaller payload. This is the paper's most credible and adoptable contribution.

  2. Transparent latency accounting. Selection overhead is included, and the stage-level breakdown in Table 6 reconciles exactly with Table 2. Because efficiency is the headline claim, this makes the speedup verifiable, even though it also shows that most of the gain comes from skipping text generation.

Critical weaknesses
  1. Unclear novelty and a missing key baseline. Token selection follows the H2O/SnapKV heavy-hitter criterion [3, 4], layer selection mirrors KVComm, and the latent steps and alignment map come from LatentMAS [1]. LatentMAS is not compared against. Most of the speedup over text (34.5 of 44.1 s) comes from skipping intermediate text generation, which any latent method achieves. This is consequential because, without LatentMAS and a compute-matched single-agent baseline such as self-consistency [2], the paper does not establish what GCS itself contributes.

  2. The ablation contradicts the paper's own results. Table 3's no-module baseline (76.7) is far below Full-KV on the same setting (84.9). Every 12% variant also beats its ~50% counterpart, which conflicts with the monotone budget curves in Figures 2–3 and Table 4. This is consequential because the ablation is the only evidence that the three modules are necessary and complementary. The theory does not compensate: Assumption B.4 is implausible under greedy decoding, and the proxy gap ∥p−s^∥1\|p - \hat{s}\|_1 is never measured.

References

[1] Zou, Jiaru, et al. "Latent collaboration in multi-agent systems." arXiv preprint arXiv:2511.20639 (2025). [2] Wang, Xuezhi, et al. "Self-consistency improves chain of thought reasoning in language models." arXiv preprint arXiv:2203.11171 (2022). [3] Zhang, Zhenyu, et al. "H2o: Heavy-hitter oracle for efficient generative inference of large language models." Advances in Neural Information Processing Systems 36 (2023): 34661–34710. [4] Li, Yuhong, et al. "Snapkv: Llm knows what you are looking for before generation." Advances in Neural Information Processing Systems 37 (2024): 22947–22970.

Questions for authors
  1. What is Table 3's baseline row, and why does a smaller budget outperform a larger one in the ablation?
  2. Can you add LatentMAS as a baseline under the same harness and ϕ\phi?
  3. Can you measure ∥p−s^∥1\|p - \hat{s}\|_1 empirically, and give any evidence that Assumption B.4 holds with modest Γ\Gamma?
  4. How was C2C's projector trained for the same-backbone setting?
3 comments
AuthorsOpenReview ↗

Response to Reviewer njhc (Part 1/3)

Response to Reviewer njhc (Part 1/3)

We appreciate Reviewer njhc for the detailed review and sharp questions. Below, we describe the new experiments and revisions we made for each point.

W1 & Q2: Isolating GCS Beyond Skipping Text Generation, Plus Related Work

We agree this is a central point. We added all five baselines across all 5 backbones and 3 benchmarks over 3 seeds in Table 2 of Section 4.1 and Table 9 of Appendix C.6. Thank you as well for pointing out the missing related work [1–4]. We added citations and discussion of all four papers in Section 2, Section 4.1, and Appendix C.6, with single-agent eviction also contrasted in Section 3.1.

Method / Protocol (Table 2 & Table 9)KVQ-1.7BDS-1.5BQ-4BM-7BQ-8B5-BB Mean (%)5-BB Time
Text CommunicationText76.463.085.543.886.271.0±0.371.0 \pm 0.332.3s
Token-Matched Text-5050 tok71.256.380.439.883.366.2±0.366.2 \pm 0.311.4s
Self-Consistency [4], k=3k=30%77.359.585.642.387.370.4±0.370.4 \pm 0.316.2s
Adapted H2O [2]12%76.259.984.842.585.969.8±0.369.8 \pm 0.310.1s
Adapted SnapKV [3]12%76.760.385.342.986.270.3±0.370.3 \pm 0.39.9s
KVComm [6]13–14%77.860.986.243.386.971.0±0.371.0 \pm 0.310.4s
LatentMAS [1], ϕ=50\phi=50100%78.161.186.743.587.471.4±0.371.4 \pm 0.314.8s
C2C$^\dagger$ [5]100%79.361.788.044.388.472.3±0.372.3 \pm 0.315.2s
GCS (Ours)12%79.462.487.744.988.472.6±0.272.6 \pm 0.29.7s

We ran LatentMAS [1], Adapted H2O [2], Adapted SnapKV [3], and GCS under the same ϕ=50\phi=50 WaW_a-aligned rollout protocol, sharing the identical Agent-1 cache at the first handoff. By contrast, Full-KV‡ at 73.3% in Table 2 relays the unfiltered KV cache prefilled from offline-cached sender traces, excluding upstream text-decoding time as an unranked reference. Both [1] and GCS skip intermediate text decoding, yet we beat unfiltered [1] on all 5 backbones. On the 5-backbone mean, we improve accuracy from 71.4% to 72.6% and cut latency from 14.8s to 9.7s. In [1], passing 100% of the continuous rollout cache leaks private heads Hpri\mathcal{H}_{\text{pri}} and off-role layers, while transferring and attending over an 8.5×8.5\times larger cache adds 5.1s in end-to-end runtime. On Qwen3-4B, our stage breakdown in Table 7 shows that selection and serialization together cost just 1.78s:

Stage (Table 7, Qwen3-4B Mean Time in s)GCS (12%)Text Comm.No-Comm.
Prompt prefill / Latent probe (ϕ=50\phi=50)2.162.16 / 5.455.452.462.46 / --2.782.78 / --
Saliency + Top-kk gather / Serialization1.071.07 / 0.710.71-- / 0.430.43-- / --
Intermediate text / Final answer decoding-- / 3.043.0434.5034.50 / 6.686.68-- / 8.358.35
Total End-to-End Latency (s)12.4312.4344.0744.0711.1311.13

We also compared GCS against 1D eviction at the same 12% budget. Adapted [2] and [3] reach 69.8% and 70.3%, trailing GCS by 2.3 to 2.8 points on average. On GSM8K, our lead reaches 2.6 to 3.0 points on average and 4.1 to 4.7 points on Qwen3-1.7B, as 1D pruning still exports private heads and off-role layers. Against single-agent sampling, we beat Self-Consistency [4] by 2.2 points at 1.67×1.67\times lower latency since [4] still decodes full text on each trace. Finally, when we cap text handoffs at 50 tokens in Text-50, derivations get cut off mid-step and accuracy falls to 66.2%.

Q4: How C2C's Projector Was Trained for the Same-Backbone Setting

Thanks for asking for this implementation detail. We added the full C2C$^\dagger$ [5] training protocol to Section 4.1. Even with shared weights, the Planner, Solver, and Verifier caches come from distinct role prompts. To adapt [5] to our unidirectional handoff, we use a two-layer per-head MLP projector dh→2dh→dhd_h \to 2d_h \to d_h with GELU and a residual link over 100% of keys and values. We trained it for 2,000 AdamW steps at learning rate 10−410^{-4} and batch size 16 on 5,000 held-out traces using receiver next-token cross-entropy. Without any training and at an 8.5×8.5\times smaller payload, we reach 72.6% vs. 72.3% for [5] and cut latency from 15.2s to 9.7s.

AuthorsOpenReview ↗

Response to Reviewer njhc (Part 2/3)

Response to Reviewer njhc (Part 2/3)

W2 & Q1: Table 3's Baseline Row and Why 12% Beats 24%–50% in Incomplete Rows

The original label in Table 3 was too terse, and we appreciate you flagging the contrast with Figures 2–3. We updated Section 4.2 and Table 3 to rename that baseline row Unfilt. 12%, report both seed protocols, and contrast incomplete rows with full 3-module scaling in Table 5.

Config (Qwen3-1.7B GSM8K)Modules I / II / IIIFree KV (%)Free Acc (%)Matched 12% (%)Δ\Delta vs. Unfilt. 12%δ\delta (Matched −- Free)
No-Communication-- / -- / --0.076.4------
Unfilt. 12%-- / -- / --12.076.776.7--0.0
Single: Module I✓\checkmark / -- / --50.074.279.1+2.4+2.4+4.9+4.9
Single: Module II-- / ✓\checkmark / --48.078.980.8+4.1+4.1+1.9+1.9
Single: Module III-- / -- / ✓\checkmark50.070.278.5+1.7+1.7+8.3+8.3
Dual: Modules I + II✓\checkmark / ✓\checkmark / --24.077.481.4+4.7+4.7+4.0+4.0
Dual: Modules I + III✓\checkmark / -- / ✓\checkmark25.078.281.7+5.0+5.0+3.6+3.6
Dual: Modules II + III-- / ✓\checkmark / ✓\checkmark24.076.981.0+4.2+4.2+4.1+4.1
GCS (All 3 Modules)✓\checkmark / ✓\checkmark / ✓\checkmark12.083.383.3+6.6+6.6--
Full-KV-- / -- / --100.084.9------

We first clarified what Unfilt. 12% measures in Table 3. In Table 3, the matched-budget Δ\Delta column compares every row at 12%. The 76.7% row is neither No-Communication at 76.4% nor Full-KV at 84.9%; it is a 12% uniform-stride baseline with all three filters turned off. It keeps noisy private heads and off-role layers while dropping 88% of tokens without saliency ranking, so it scores only 76.7%. When we turn on our three filters at the same 12% budget, accuracy rises by +6.6 points to 83.3%. We also note the two seed setups in Section 4.2. Table 2 averages 3 evaluation seeds at 83.9 ± 0.2%, while Table 3 runs an 8-seed sweep over calibration subsamples and tie-breaking at 83.3 ± 0.4%.

We also checked why 12% beats 24%–50% with δ>0\delta > 0 in Table 3, whereas scaling is monotone in Figures 2–3 and Table 5. In every Single and Dual row, at least one filter is off, so the 24%–50% Free setting passes that entire unfiltered axis to the receiver. When Module I or II is off, the sender leaks private heads or off-role layers. In Single: Module III, sending top tokens from all heads and layers drops Free accuracy to 70.2%. When Module III is off in Dual: I + II, the 24% Free slice keeps unranked filler tokens that dilute attention relative to 12% subsampling, giving 77.4% vs. 81.4%. When we subsample an incomplete row to 12%, we trim this cross-agent noise, yet every incomplete row from 78.5% to 81.7% stays well below full GCS at 83.3%.

By contrast, in Figures 2–3 and Table 5 of Appendix C.5, we keep all three filters active together and scale ρ\rho on Qwen3-4B. With public heads and aligned layers enforced, accuracy rises smoothly at every step:

GCS Budget (Qwen3-4B, All 3 Modules)Payload (MiB)ARC-C (%)GSM8K (%)HE+ (%)3-Task Avg (%)Latency (s)
ρ=0.5%\rho = 0.5\%0.589.987.176.284.411.9
ρ=1%\rho = 1\%0.990.988.078.085.612.0
ρ=5%\rho = 5\%4.592.189.179.386.812.2
ρ=12%\rho = 12\% (Default)10.692.989.880.587.712.4
ρ=25%\rho = 25\%22.593.290.281.188.213.4
Full-KV Relay, 100%90.093.990.681.188.518.0
AuthorsOpenReview ↗

Response to Reviewer njhc (Part 3/3)

Response to Reviewer njhc (Part 3/3)

Q3: Empirical Measurement of the Proxy Gap ∥p−s^∥1\lVert p - \hat{s} \rVert_1

Thank you for suggesting this direct empirical check on our sender proxy. We logged the sender rollout proxy s^\hat{s}, written as sˉ\bar{s} in Section 3.3, alongside oracle receiver attention pp across 500 evaluation instances on Qwen3-4B and Qwen3-1.7B. We added the full comparison to Section 4.2 and Table 10 of Appendix C.7.

BackboneSaliency / Probe Policy∥p−sˉ∥1↓\lVert p - \bar{s} \rVert_1 \downarrowTop-kk Ovlp ↑\uparrowSpearman ρs↑\rho_s \uparrowMean Γ^\hat{\Gamma}95th % Γ^\hat{\Gamma}3-Task Acc (%)
Qwen3-4BUniform weights, sˉt=1/Ti\bar{s}_t = 1/T_i0.612±0.0580.612 \pm 0.05841.2%--2.313.8985.1±0.385.1 \pm 0.3
Qwen3-4BSender prompt-only, ϕ=0\phi=00.348±0.0450.348 \pm 0.04566.8%0.642.043.4586.5±0.386.5 \pm 0.3
Qwen3-4BUnaligned rollout, ϕ=50\phi=50, no WaW_a0.486±0.0520.486 \pm 0.05256.4%0.492.183.6786.1±0.386.1 \pm 0.3
Qwen3-4BAligned rollout, ϕ=10\phi=10, with WaW_a0.214±0.0380.214 \pm 0.03878.3%0.811.913.2487.2±0.287.2 \pm 0.2
Qwen3-4BGCS Default, ϕ=50\phi=50, with WaW_a0.142±0.0310.142 \pm 0.03184.7%0.891.843.1287.7±0.287.7 \pm 0.2
Qwen3-4BOracle S⋆=Top-k(p)S^\star = \text{Top-}k(p)0.000100.0%1.001.762.9888.0±0.288.0 \pm 0.2
Qwen3-1.7BGCS Default, ϕ=50\phi=50, with WaW_a0.151±0.0340.151 \pm 0.03483.9%0.881.793.0579.4±0.279.4 \pm 0.2
Qwen3-1.7BOracle S⋆=Top-k(p)S^\star = \text{Top-}k(p)0.000100.0%1.001.712.9179.7±0.279.7 \pm 0.2

We see in Table 10 that prompt-only sender attention at ϕ=0\phi=0 leaves an ℓ1\ell_1 gap of 0.348. When we run ϕ=50\phi=50 steps without WaW_a, hidden states drift and the gap widens to 0.486. Once we apply WaW_a, just 10 rollout steps cut ∥p−sˉ∥1\lVert p - \bar{s} \rVert_1 to 0.214 and reach 87.2% accuracy in 9.1s. At our default ϕ=50\phi=50, we bring ∥p−sˉ∥1\lVert p - \bar{s} \rVert_1 down to 0.142 ± 0.031 on Qwen3-4B, match 84.7% of oracle Top-kk tokens, and reach 87.7% vs. 88.0% for the two-pass receiver oracle.

W2 (Theory) & Q3: Sensitivity Γ^\hat{\Gamma}, Greedy Decoding, and Softmax Renormalization

We measured the empirical sensitivity ratio Γ^\hat{\Gamma} and greedy token agreement across the same 500 traces in Table 10 above. We also added Remark E.5 to write out the exact softmax renormalization across Ω\Omega.

First, we checked Assumption B.4 and greedy decoding empirically. Assumption B.4 and Theorem 3.5 bound continuous logit shifts ∥z(x;g^)−z(x;gfull)∥2\lVert z(x;\hat{g}) - z(x;g_{\text{full}}) \rVert_2 under an LtaskL_{\text{task}}-Lipschitz surrogate loss in Assumption B.3, rather than treating hard argmax decisions as Lipschitz. Empirically across all 500 instances on Qwen3-4B, we find that Γ^(x)\hat{\Gamma}(x) stays modest, with mean 1.84, 95th percentile 3.12, and maximum 4.65. Greedy decoding keeps the exact same token whenever the top-two logit margin exceeds 2,∥z(g^)−z(gfull)∥2\sqrt{2}\\,\lVert z(\hat{g}) - z(g_{\text{full}}) \rVert_2. In practice, our 12% gate g^\hat{g} at 87.7% accuracy picks the exact same greedy token as gfullg_{\text{full}} at 88.1% on 94.2% of steps.

Second, we verified softmax renormalization over background memory. In Definition B.9 and Lemma E.3, we define PtP_t over the full joint memory Ω=Ji∪(Ω∖Ji)\Omega = J_i \cup (\Omega \setminus J_i). When we prune Ji∖SJ_i \setminus S, every kept key in R=S∪(Ω∖Ji)R = S \cup (\Omega \setminus J_i) is rescaled by 1/(1−η)1/(1-\eta) with η=βi,ℓ,h,τ(x),δℓ,h(i)(τ;x)\eta = \beta_{i,\ell,h,\tau}(x)\\,\delta^{(i)}_{\ell,h}(\tau;x). In Remark E.5, we show that Lemma E.1 applies directly to RR on Ω\Omega, so this exact renormalization across background memory is already built into our bound.

[1] J. Zou et al. Latent Collaboration in Multi-Agent Systems. ICML'26.

[2] Z. Zhang et al. H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models. NeurIPS'23.

[3] Y. Li et al. SnapKV: LLM Knows What You Are Looking For Before Generation. NeurIPS'24.

[4] X. Wang et al. Self-Consistency Improves Chain of Thought Reasoning in Language Models. ICLR'23.

[5] T. Fu et al. Cache-to-Cache: Direct Semantic Communication Between Large Language Models. ICLR'26.

[6] X. Shi et al. KVComm: Enabling Efficient LLM Communication through Selective KV Sharing. ICLR'26.

Comments on the paper

3 comments
AuthorsOpenReview ↗

Global Response to All Reviewers (Part 1/2)

Global Response to All Reviewers (Part 1/2)

We sincerely thank Reviewers njhc, va5A, and MnYf for their careful reviews and helpful suggestions. We have revised the manuscript and marked main updates in blue across Section 2, Section 3.1, Section 3.4, Section 4.1–4.4, Appendix C.1–C.9, Remark E.5, and Appendix F. Below we summarize our new experiments and manuscript updates.

1. Extended Baselines, Related Work, Multi-Seed Error Bars, and Ablations

We appreciate all three reviewers' feedback on baselines, error bars, related work, and ablation clarity. We ran 3 evaluation seeds across all 5 backbones and 3 benchmarks by varying the calibration draw Dcal\mathcal{D}_{\text{cal}} and role-prompt exemplar order. Under the same harness, we added five baselines in Table 2 of Section 4.1 and Table 9 of Appendix C.6. These cover LatentMAS [1] at ϕ=50\phi=50, adapted H2O [2], adapted SnapKV [3], a 50-token text cap Text-50, and Self-Consistency [4] at k=3k=3. We also added citations and discussion of [1–4] in Section 2, Section 4.1, and Appendix C.6, and contrasted single-agent pruning in Section 3.1. In Section 4.1, we also detailed our adapted C2C$^\dagger$ [5] projector setup.

Here we present Table R1 (Table 2 in Section 4.1 and Table 9 in Appendix C.6):

Method / ProtocolKVPayload (MiB)5-BB Mean Acc (%)5-BB Time (s)Qwen3-4B Acc (%)Qwen3-4B Time (s)Qwen3-1.7B GSM8K (%)
No-Communication0%0.068.5±0.268.5 \pm 0.28.883.3±0.283.3 \pm 0.211.176.4±0.376.4 \pm 0.3
Text CommunicationText<0.1<0.171.0±0.371.0 \pm 0.332.385.5±0.385.5 \pm 0.344.182.6±0.382.6 \pm 0.3
Token-Matched Text-50Text<0.1<0.166.2±0.366.2 \pm 0.311.480.4±0.380.4 \pm 0.314.974.8±0.474.8 \pm 0.4
Self-Consistency [4], k=3k=30%0.070.4±0.370.4 \pm 0.316.285.6±0.385.6 \pm 0.319.879.5±0.379.5 \pm 0.3
Adapted H2O [2]12%2.1–10.669.8±0.369.8 \pm 0.310.184.8±0.384.8 \pm 0.313.079.2±0.479.2 \pm 0.4
Adapted SnapKV [3]12%2.1–10.670.3±0.370.3 \pm 0.39.985.3±0.385.3 \pm 0.312.879.8±0.379.8 \pm 0.3
KVComm [6]13–14%2.5–12.571.0±0.371.0 \pm 0.310.486.2±0.386.2 \pm 0.313.382.2±0.382.2 \pm 0.3
LatentMAS [1], ϕ=50\phi=50100%17.5–90.071.4±0.371.4 \pm 0.314.886.7±0.286.7 \pm 0.218.582.9±0.382.9 \pm 0.3
C2C$^\dagger$ [5]100%17.5–90.072.3±0.372.3 \pm 0.315.288.0±0.388.0 \pm 0.319.484.6±0.384.6 \pm 0.3
GCS (Ours)12%2.1–10.672.6±0.272.6 \pm 0.29.787.7±0.287.7 \pm 0.212.483.9±0.283.9 \pm 0.2

We also updated Section 4.2 and Table 3 to rename the 76.7% row as Unfilt. 12%, a 12% uniform-stride baseline with all filters off. We report both the 3-seed main evaluation at 83.9 ± 0.2% and the 8-seed selector sweep at 83.3 ± 0.4%. In Section 4.2, we explain why subsampling incomplete rows cuts cross-agent noise δ>0\delta > 0, whereas scaling all three filters together raises accuracy from 84.4% to 88.2% in Table 5.

Table R2 (Table 3 in Section 4.2): 8-seed module ablation on Qwen3-1.7B GSM8K.

ConfigModules I / II / IIIFree KV (%)Free Acc (%)Matched 12% (%)Δ\Delta vs. Unfilt. 12%δ\delta
Unfilt. 12%-- / -- / --12.076.776.7--0.0
Module I / II / III onlySingle filter50 / 48 / 5074.2 / 78.9 / 70.279.1 / 80.8 / 78.5+2.4+2.4 / +4.1+4.1 / +1.7+1.7+4.9+4.9 / +1.9+1.9 / +8.3+8.3
I+II / I+III / II+IIIDual filters24 / 25 / 2477.4 / 78.2 / 76.981.4 / 81.7 / 81.0+4.7+4.7 / +5.0+5.0 / +4.2+4.2+4.0+4.0 / +3.6+3.6 / +4.1+4.1
GCS (All 3 Modules)✓\checkmark / ✓\checkmark / ✓\checkmark12.083.383.3+6.6+6.6--
AuthorsOpenReview ↗

Global Response to All Reviewers (Part 2/2)

Global Response to All Reviewers (Part 2/2)

2. Proxy Error ∥p−sˉ∥1\|p-\bar{s}\|_1, Top-kk Overlap, and Sensitivity Γ^\hat{\Gamma}

Thank you to Reviewers njhc and va5A for suggesting direct checks on our sender proxy and sensitivity assumptions. We logged sender proxy scores sˉ\bar{s}, oracle receiver attention pp, and logit shifts over 500 evaluation instances in Section 4.2 and Table 10 of Appendix C.7. We also added Remark E.5 to write out the exact softmax renormalization across Ω\Omega. On Qwen3-4B, we reach ∥p−sˉ∥1=0.142±0.031\|p-\bar{s}\|_1 = 0.142 \pm 0.031, match 84.784.7\\% of oracle Top-kk tokens, and get mean Γ^=1.84\hat{\Gamma}=1.84 with 94.294.2\\% greedy token agreement:

BackboneSaliency / Probe Policy∥p−sˉ∥1↓\|p - \bar{s}\|_1 \downarrowTop-kk Ovlp ↑\uparrowSpearman ρs↑\rho_s \uparrowMean Γ^\hat{\Gamma}95th % Γ^\hat{\Gamma}3-Task Acc (%)
Qwen3-4BUniform weights, sˉt=1/Ti\bar{s}_t = 1/T_i0.612±0.0580.612 \pm 0.05841.2%--2.313.8985.1±0.385.1 \pm 0.3
Qwen3-4BSender prompt-only, ϕ=0\phi=00.348±0.0450.348 \pm 0.04566.8%0.642.043.4586.5±0.386.5 \pm 0.3
Qwen3-4BUnaligned rollout, ϕ=50\phi=50, no WaW_a0.486±0.0520.486 \pm 0.05256.4%0.492.183.6786.1±0.386.1 \pm 0.3
Qwen3-4BAligned rollout, ϕ=10\phi=10, with WaW_a0.214±0.0380.214 \pm 0.03878.3%0.811.913.2487.2±0.287.2 \pm 0.2
Qwen3-4BGCS Default, ϕ=50\phi=50, with WaW_a0.142±0.0310.142 \pm 0.03184.7%0.891.843.1287.7±0.287.7 \pm 0.2
Qwen3-4BOracle S⋆=Top-k(p)S^\star = \text{Top-}k(p)0.000100.0%1.001.762.9888.0±0.288.0 \pm 0.2
Qwen3-1.7BGCS Default, ϕ=50\phi=50, with WaW_a0.151±0.0340.151 \pm 0.03483.9%0.881.793.0579.4±0.279.4 \pm 0.2
Qwen3-1.7BOracle S⋆=Top-k(p)S^\star = \text{Top-}k(p)0.000100.0%1.001.712.9179.7±0.279.7 \pm 0.2

3. Distributed Network Latency Across Interconnect Tiers

We thank Reviewer MnYf for raising multi-node network transfer overhead. We estimated two-handoff distributed runtime on Qwen3-4B across four interconnect tiers by combining measured local execution with socket- and NCCL-verified transfer costs in Table 4a and Table 11:

ProtocolPayload/HopNVLink (s / Spd.)100GbE (s / Spd.)10GbE (s / Spd.)1GbE WAN (s / Spd.)Acc (%)
Text Comm.<0.1 MiB<0.1\text{ MiB}44.0744.07 / 1.00×1.00\times44.0744.07 / 1.00×1.00\times44.0744.07 / 1.00×1.00\times44.0744.07 / 1.00×1.00\times85.5
Full-KV fp1690.00 MiB90.00\text{ MiB}18.0018.00 / 2.45×2.45\times18.0118.01 / 2.45×2.45\times18.1518.15 / 2.43×2.43\times19.5119.51 / 2.26×2.26\times88.5
GCS fp16, 12%10.63 MiB10.63\text{ MiB}12.4312.43 / 3.55×3.55\times12.4312.43 / 3.55×3.55\times12.4512.45 / 3.54×3.54\times12.6112.61 / 3.50×3.50\times87.7
GCS + int85.31 MiB5.31\text{ MiB}11.9011.90 / 3.70×3.70\times11.9011.90 / 3.70×3.70\times11.9111.91 / 3.70×3.70\times11.9911.99 / 3.68×3.68\times87.3
GCS + int42.66 MiB2.66\text{ MiB}11.4011.40 / 3.87×3.87\times11.4011.40 / 3.87×3.87\times11.4011.40 / 3.87×3.87\times11.4411.44 / 3.85×3.85\times86.6

4. Diverse Topologies, Heterogeneous Backbones, and SWE-bench Lite

We also appreciate Reviewers va5A and MnYf pushing us on broader system settings. In Table 4b of Section 4.4 and Table 12 of Appendix C.9, we tested 4-agent chains, 2-round debate, star graphs, mixed-size Qwen3, and SWE-bench Lite:

System Setting / TopologyBackbone / BenchmarkText Acc (%)Text (s)Full-KV$^\ddagger$ / C2C$^\dagger$ [5] (%)Full-KV (s)GCS 12% (%)GCS (s)Speedup
3-Agent ChainQwen3-4B, 3-Bench Avg85.544.188.518.087.712.43.55×3.55\times
4-Agent ChainQwen3-4B, 3-Bench Avg85.958.488.624.188.315.83.70×3.70\times
3-Agent 2-Round DebateQwen3-4B, 3-Bench Avg86.179.288.433.588.121.43.70×3.70\times
Star 2→12\to 1 AggregatorQwen3-4B, 3-Bench Avg85.849.688.220.388.013.93.57×3.57\times
Heterogeneous PipelineQwen3-8B$\to$1.7B$\to$4B83.936.585.8†85.8^\dagger16.486.49.83.72×3.72\times
SWE-bench Lite, T≈4.2kT\approx 4.2\text{k}Qwen3-8B, 300 Issues26.7148.627.368.428.341.23.61×3.61\times

[1] J. Zou et al. Latent Collaboration in Multi-Agent Systems. ICML'26.

[2] Z. Zhang et al. H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models. NeurIPS'23.

[3] Y. Li et al. SnapKV: LLM Knows What You Are Looking For Before Generation. NeurIPS'24.

[4] X. Wang et al. Self-Consistency Improves Chain of Thought Reasoning in Language Models. ICLR'23.

[5] T. Fu et al. Cache-to-Cache: Direct Semantic Communication Between Large Language Models. ICLR'26.

[6] X. Shi et al. KVComm: Enabling Efficient LLM Communication through Selective KV Sharing. ICLR'26.

AuthorsOpenReview ↗

Summary of Discussion and Revisions

We sincerely thank the Area Chair and Reviewer va5A, Reviewer njhc, and Reviewer MnYf for their time and helpful feedback. All three reviewers noted that Gricean Cache Sharing (GCS) offers a simple, training-free, and inspectable KV-sharing protocol with a smooth accuracy–latency trade-off, clear latency accounting, and careful latent probing. Following our rebuttal and revised paper, Reviewer va5A thanked us for answering their questions and maintained their positive score.

Overview of Our Work and Revisions

Multi-agent text handoffs are slow due to sequential decoding, while sending the full KV cache (17.5–90.0 MiB/hop) adds traffic and leaks private heads and off-role layers. GCS selects shared KV states across heads, layers, and tokens without training, cutting latency by 3.3× on average (up to 5.0×) while matching or beating text on 12 of 15 settings. Our blue-highlighted revision addresses every question across all three reviews:

ReviewerMain FocusStatus
Reviewer va5AError bars, topologies, heterogeneous models, sender proxyAddressed · Positive Score Maintained
Reviewer njhcBaselines, projector setup, Table 3 ablation, proxy & softmaxAddressed
Reviewer MnYfTheory role, 3D vs. 1D pruning, network latency, heterogeneityAddressed

How Each Concern Was Addressed

  1. Baselines, 1D Pruning & Error Bars (Reviewer njhc W1/Q2/Q4; Reviewer va5A W1; Reviewer MnYf W1) Addressed We ran 3 seeds across all 5 backbones and 3 benchmarks (Table 2 std ≤0.3%\le 0.3\%; Appendix C.6, Table 9), expanded our discussion of previous work (Section 2, Section 3.1), detailed the projector setup (Section 4.1), and added five baselines from previous work. GCS beats both unfiltered latent relay and 1D pruning under the same rollout setup: at 12% KV, GCS reaches 72.6 ± 0.2% in 9.7 ± 0.1s, ahead of 100%-KV latent relay on all 5 backbones (71.4%, 14.8s), 1D pruning (69.8%–70.3%), single-agent voting (70.4%, 16.2s), 50-token text (66.2%), and the trained projector (72.3%, 15.2s), with just 1.78s selection and serialization overhead (Table 7).

  2. Module Ablation & Monotone Scaling (Reviewer njhc W2/Q1; Reviewer MnYf W1) Addressed In Section 4.2 and Table 3, we renamed the 76.7% row to Unfilt. 12% (12% uniform-stride with all filters off, vs. 76.4% No-Comm. and 84.9% Full-KV) and reported 3-seed (83.9 ± 0.2%) and 8-seed (83.3 ± 0.4%) results. All three filters are needed together to prevent cross-agent noise: disabling a filter at 24%–50% Free budget leaks private heads, off-role layers, or filler tokens (token selection alone drops to 70.2%; Table 6), while keeping all three active gives a strictly increasing curve from 0.5% to 25% budget (84.4% to 88.2%; Figure 3, Table 5).

  3. Sender Proxy, Sensitivity & Softmax Renormalization (Reviewer njhc W2/Q3; Reviewer va5A Q2; Reviewer MnYf W1) Addressed We clarified in Section 3.1 that Grice's Principle is an intuitive guide for grouping our three filters (Section 3.3 – 3.4, Appendix F). Over 500 instances (Section 4.2, Appendix C.7, Table 10), our aligned sender rollout closely tracks receiver attention with small logit shifts: aligned probing (ϕ=50\phi=50) cuts ∥p−s^∥1\|p-\hat{s}\|_1 from 0.348 (ϕ=0\phi=0) and 0.486 (unaligned) to 0.142 ± 0.031 (84.7% Top-kk overlap, ρs=0.89\rho_s=0.89; 87.7% vs. 88.0% oracle), with mean sensitivity Γ^=1.84\hat{\Gamma}=1.84 and 94.2% greedy agreement. Remark E.5 writes out exact softmax renormalization over joint memory.

  4. Network Latency, Topologies, Heterogeneous Models & SWE-bench Lite (Reviewer va5A W2/Q1; Reviewer MnYf W2/Q1) Addressed Network overhead stays tiny across links, and GCS generalizes across topologies, model sizes, and long contexts. Two-hop Qwen3-4B transfer adds 17.8/4.5 ms (fp16/int4) on 10GbE and 178/44.6 ms on 1GbE WAN, keeping a 3.50×–3.85× speedup on 1GbE WAN (Table 4a, Table 11). In Table 4b and Appendix C.9 (Table 12), GCS beats text by 2.0–2.4 points (3.57×–3.70× faster) on 4-agent chains, debate, and star graphs; reaches 86.4 ± 0.3% in 9.8s (3.72× faster) on a training-free Qwen3-8B → 1.7B → 4B pipeline via CKA and ridge adapters (beating the trained projector at 85.8%, 16.4s, and prior KV methods by 3.5–8.7 points in cross-family tests); and resolves 28.3% of 300 SWE-bench Lite issues in 41.2s (vs. 26.7% in 148.6s for text and 27.3% for Full-KV).

We thank the Area Chair and reviewers again for their feedback.

Best regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.