acceptodds
Under review as a conference paper at ICLR 2027

CacheWriter: Learning to Write Latent Messages for Frozen Language Models

13 views

Abstract

Large language model (LLM) agents often hold private information, so a shared task requires one agent to pass what it knows to another. Text is the usual channel, but it costs the sender a decode and the receiver a re-encode, and the message must fit into the receiver's context. Transmitting key–value (KV) cache entries avoids the decode. However, existing cache handoff only selects among states that the sender computed for its own continuation, and under a tight position budget, selection leaves most of the available gain unrecovered. In this work, we propose CacheWriter, in which a question-blind sender writes a fixed number of KV entries for a frozen receiver. Specifically, each entry is a selected state plus a zero-initialised low-rank update, so that training starts exactly at selection. The update learns to match the receiver's output distribution given the full private context, without any answer labels. Extensive experiments across four model families and four single-document question-answering benchmarks demonstrate that 32-entry messages improve validation exact match by 5.3 percentage points on average over the strongest position-matched baseline. On answer-exclusive, randomly ordered five-document HotpotQA, 64 written entries more than double the best selection policy for Qwen3-8B (19.6% vs. 9.4% held-out exact match). Replacing the learned update with its document-independent mean further shows that the instance-specific part of the gain grows as the budget binds. Across model families, one fixed calibration per pair beats a position-matched text note by 8.6 points (Llama-3.1→Qwen3) and 14.6 points (Qwen3→Falcon3) of held-out exact match, and on the first pair it is statistically indistinguishable from same-family writing.

Primary area:
foundation or frontier models, including LLMs
Keywords:
latent communication, large language models, multi-agent systems, KV cache compression, knowledge distillation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_7k2aOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes CacheWriter, a mechanism that trains a sender LLM to transmit a fixed number of key-value cache entries to a receiver LLM. The idea is an adaption of KV-Distill and involves training LoRA adapters on the sender’s key, value (and query and output) projections so that prepending the resulting cache entries to the receiver’s context produces an output distribution close to what the receiver would produce if it had access to the full private information. The authors compare their method with a number of selection-based baselines on various tasks. They show that adapting cache states help, particularly at settings where the transmission budget is tight. The paper then extensively tests whether improvements are the results of the sender's information, and under what conditions cachewritie is useful.

Critical strengths

1- The method is simple and easy to implement.

2- The existing experimental results are a major strengths of the paper and are quite insightful. This includes main results, controlled experiments, and ablations.

Critical weaknesses

1- The writing is strange and the paper is extremely hard to read. This seems to be the result of using an LLM to generate most of the discussions.

2- The paper provides a useful empirical study of cache communication, but the method is very similar to KV-Distill. In other words the contribution of the paper is the analysis rather than a new method.

3- There are two training-based baselines that are missing from this work. The paper should compare against a simple learned constant update (with the exact same loss and setting as CacheWriter) in order to justify the per-instance correction. The authors should also compare against KV-Distill. What I mean is a simple learned baseline is using KV-Distill to compress the sender’s private document into the same number of cache entries. Then pass them to the receiver. This would help show how better cacheWriter is compared to using an existing learned cache compressor to send the message.

Questions for authors

In addition to points raised in weaknesses:

1- How were the 32 or 64 positions chosen? What is the practical motivation of these values?

2- Can the authors provide results of scaling the number of positions budget to see whether CacheWriter remains competitive at more realistic budgets? It seems that the advantage of CahceWriter diminishes substantially at 64.

3- Did the authors observe any improvement from adapting all four projections over adapting only K and V? I'm trying to understand if there is benefit in the added training cost.

2 comments
AuthorsOpenReview ↗

Response to Reviewer 7k2a (1/2)

Thank you for the constructive suggestions, which helped us make the comparison with existing methods more complete.

W1. Writing. We carefully revised the discussion of the results for readability. All changes aim to make our design and results easier to understand, while keeping the original experiments and main conclusions.

W2. Similarity to KV-Distill. The two methods differ in four respects. First, the reader differs. In KV-Distill [1], the model that compressed the context later reads it itself. In CacheWriter, a different agent reads the message. This receiver holds its own evidence and a question that the sender never sees. Second, the training target differs. KV-Distill's target is the model's own output on its full context. CacheWriter's target is the receiver's output when the receiver sees both its own evidence and the sender's full context. In our five-document setting, KV-Distill's objective leaves out the receiver's evidence and lowers test exact match from 0.196 to 0.143 at 64 entries. Third, the learned part differs. KV-Distill learns which tokens to keep, whereas CacheWriter fixes the kept positions and learns the content written there. A reimplementation of KV-Distill's learned selector reaches 0.162 at 32 entries, significantly below our 0.189, and it matches CacheWriter at 64 entries. Fourth, the scope differs. KV-Distill works within one model and does not cover communication across model families, whose caches differ in format. CacheWriter covers this case, and its message to a receiver of another family beats a same-length note by +0.120 and +0.179 on test. The two methods share a LoRA adapter and a mix of forward and reverse KL divergence. Both are common practice in fine-tuning and distillation and not specific to KV-Distill.

W3. The two trained baselines. We added both baselines to Table 2 of the revised PDF, using the five-document setting with Qwen3-8B on HotpotQA.

Learned constant update. Following your suggestion, we train one constant tensor that is added to every input's message, with exactly the same loss, data and setting as CacheWriter. We try two initialisations and two learning rates and keep the best on validation. This constant falls below CacheWriter by 0.051 at both 32 and 64 entries. Part of the update must therefore depend on the input.

KV-Distill as the message compressor. KV-Distill's code is not public. We therefore reimplement it in our setting, compress the sender's documents into the same number of cache entries, and pass them to the receiver. We test three variants, each a row of Table 2. The first trains the message with KV-Distill's objective, which ignores the receiver's evidence. The second trains it only to continue the sender's own text, without the receiver's evidence or question. The third uses KV-Distill's learned token selector. All three fall significantly below CacheWriter at 32 entries, scoring 0.116, 0.098 and 0.162 against 0.189. The first two also stay significantly below CacheWriter at 64 entries, at 0.143 and 0.122 against 0.196, while the learned selector comes closer at 0.179. Appendix E of the revised PDF gives the details.

References

[1] KV-Distill: Nearly Lossless Learnable Context Compression for LLMs, Chari et al., 2025.

AuthorsOpenReview ↗

Response to Reviewer 7k2a (2/2)

Q1. Why 32 and 64 entries. We chose these budgets for three reasons. First, they keep the message much shorter than the sender's private context, which is where the choice of message matters. 32 entries cover 29% of an average HotpotQA paragraph and 5% of five documents. Selection must then drop most of the context. This tight regime is a stricter test of whether a written message helps. Second, they match the budgets of our baselines. A 32-entry message fills as many receiver positions as a 32-token note. It also takes 28% of the full cache, close to the 30% that KVComm [1] sends. Third, our conclusions do not depend on this choice. CacheWriter leads selection at every budget from 32 to 256 entries with five documents, as the table under Q2 shows.

Q2. Larger budgets. To answer this question, we trained CacheWriter at 128 and 256 entries, in addition to 32 and 64. We use the five-document setting with Qwen3-8B on HotpotQA, where the private context is long enough for these budgets to matter. We then compare it with selection at each budget on test. CacheWriter leads selection at every budget. Its lead narrows as selection keeps more of the context, but it stays significant at 256 entries. CacheWriter also gets closer to the full context as the budget grows. The receiver scores 0.064 with no message and 0.278 when it reads all five of the sender's documents directly. CacheWriter closes 58% of this gap at 32 entries and 84% at 256.

Five documents, Qwen3-8B on HotpotQA, test32 entries64 entries128 entries256 entries
Selection0.0760.0900.1200.170
CacheWriter0.1890.1960.2150.244
Share of the full-context gain recovered by CacheWriter58%62%71%84%

To clarify, the smaller advantage at 64 entries that you mentioned appears only in the single-document setting, where the sender holds one paragraph. Our average lead over the strongest same-length baseline in this setting falls from +0.053 at 32 entries to +0.017 at 64. We still lead at 64 entries in 11 of the 16 cases, which cover four models on four benchmarks. The reason is that a paragraph is short, 112 to 191 tokens on average, depending on the benchmark. Selection at 64 entries already keeps the cache entries of a third to more than half of it, and with them much of the answer. Going from 32 to 64 entries raises the best selection policy by 0.089 and our message by 0.041. This is why we also test the five-document setting, where the sender holds five documents. They are much longer, and 64 entries cover less than 10% of them. Our message more than doubles the exact match of the best selection policy with five documents, from 0.094 to 0.196.

Q3. Key and value adapters only. We trained an adapter on the key and value projections alone, in the five-document setting, and compared it with our adapter on all four projections. The key-value adapter has 23.6M parameters against 61.3M for all four projections. Training time barely differs, 9.5 against 9.7 seconds per step on one NVIDIA L40S GPU (48 GB), because most of the compute goes to the frozen 8B base model. Inference time is the same. Writing a five-document message on the same GPU takes 0.073 s with either adapter merged into the sender's weights, against 0.072 s for running the sender without any adapter. The key-value adapter scores 0.162 on test at 32 entries, significantly below 0.189 for all four projections. The two match at 64 entries, 0.188 against 0.196. Adapting all four projections therefore helps most when the budget is tight and each entry must hold more of the context. We keep it as the default.

References

[1] KVComm: Enabling Efficient LLM Communication through Selective KV Sharing, Shi et al., ICLR 2026.

Reviewer_bXyDOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes CacheWriter, a learned KV-cache communication method that uses LoRA-adapted sender models to modify selected KV entries for a frozen receiver. Experiments across multiple model families and QA benchmarks demonstrate improvements over KV selection and text-based communication under constrained position budgets. The paper further provides extensive analyses of component effectiveness, instance-specific performance gains, cross-family communication, and efficiency.

Critical strengths
  1. Simple and well-motivated method design. The proposed method is conceptually straightforward and addresses a clear limitation of existing KV selection approaches: the selected KV states are not necessarily optimized for communication under constrained position budgets. The selection-initialized design provides an exact baseline for evaluating learned KV modifications, while keeping the receiver frozen.

  2. Extensive experimental evaluation and analysis. The paper provides comprehensive comparisons across multiple models, benchmarks, and compression budgets. Beyond reporting performance improvements, it includes careful analyses of different components and investigates whether the gains arise from generic corrections or instance-specific information. In particular, the derangement and mean-update experiments provide useful evidence for understanding the effectiveness of learned KV communication.

Critical weaknesses
  1. Limited methodological distinction from existing learned KV compression methods. The proposed method is closely related to prior learned KV compression approaches, particularly KV-Distill, which also adapts selected KV states through trainable attention modifications and distillation. While the paper distinguishes itself through the private-view communication setting, most experiments use the same base model for the sender and receiver. In this setting, the problem is essentially similar to compressing an earlier context into KV entries and using them for subsequent inference with additional context and a question.

  2. Limited evaluation in realistic agentic settings. The evaluation is primarily conducted on short-answer QA tasks with constructed private-view splits. Although these experiments effectively isolate the information transfer problem, it remains unclear how well CacheWriter generalizes to more complex agentic workloads involving multi-round interactions, tool usage, and evolving information requirements.

Questions for authors
  1. Comparison with KV-Distill. Given the substantial similarities in learned KV adaptation, could the authors compare CacheWriter with KV-Distill under the same private-view setting, position budgets, and receiver models? What specific design choices account for the differences between the two methods?

  2. Generalization to agentic tasks. How would CacheWriter perform on realistic multi-agent tasks that require multi-round communication or long-horizon reasoning? In particular, can a written KV message be reused effectively across multiple downstream questions or agent interactions?

  3. Serving efficiency and compatibility. What are the exact experimental settings used for the latency measurements? How does CacheWriter perform in an end-to-end serving system, including both sender and receiver computation, KV transfer, and concurrent requests? Can the written KV entries be integrated efficiently into existing serving frameworks such as SGLang or vLLM, particularly with prefix caching and KV reuse?

  4. Mechanism of fixed calibration. The strong performance of document-independent calibration in cross-family communication is interesting, but its underlying mechanism remains unclear. Have the authors analyzed the similarity, variance, or low-rank structure of the learned updates across different documents? For example, how large and functionally important are the instance-specific residuals relative to their mean? Such analyses could help explain why a fixed calibration is sufficient.

  5. Evidence for information encoding. The derangement and mean-update experiments establish instance-dependent performance gains, but do not fully reveal how the learned updates encode information. Could the authors provide more controlled experiments, such as synthetic fact retrieval or counterfactual document modifications, to directly examine whether the written KV entries preserve information that the original selection misses?

  6. Choice of distillation trajectory. In Section 4.3, teacher-generated trajectories outperform student-generated trajectories at the tighter budget. Could the authors clarify why student rollouts are adopted as the default training strategy, and whether a more systematic comparison of different distillation objectives or rollout strategies was conducted?

3 comments
AuthorsOpenReview ↗

Response to Reviewer bXyD (1/2)

Thank you for the detailed review. Your suggestions helped us strengthen the experiments and clarify how CacheWriter differs from prior work.

W1 and Q1. KV-Distill and the shared base model. KV-Distill [1] trains a model to read its own compressed context. In contrast, CacheWriter trains a sender to write for a receiver. The receiver holds its own evidence and a question that the sender never sees, and it also serves as the teacher. Sharing the base model therefore does not make this self-compression. Our experiments also cover a sender and a receiver from two model families, such as a Llama-3.1 sender and a Qwen3 receiver. The revision adds a sender adapter to this case, and our message there beats a same-length note by +0.120 and +0.179 on the two model pairs we test.

KV-Distill has two components, a training objective and a learned token selector. We test each in our five-document setting, where the sender holds five HotpotQA documents. First, KV-Distill's objective ignores the receiver's evidence, and it lowers test exact match from 0.196 to 0.143 at 64 entries. To separate what the evidence says from its mere presence, we also train with evidence of the same length taken from another question. This control recovers about half of the drop. The other half comes from what the receiver's evidence says. Second, we reimplement KV-Distill's learned selector for our task. It scores 0.162 at 32 entries, significantly below 0.189 for CacheWriter. The two match at 64 entries. The low-rank adapter follows LoRA [2], and the mix of forward and reverse KL divergence is standard in distillation, as in MiniLLM [3] and GKD [4]. KV-Distill uses both as well. The revised §3.2 and §3.3 cite these works, including KV-Distill.

W2 and Q2. Agentic tasks and message reuse. We study one-way, single-round communication between two agents on question answering, as the closest prior work does. For example, activation communication [5] passes one agent's activations to another once. In its tasks, only the first agent holds the information that the second needs. KVComm [6] adopts these tasks and adds question-answering benchmarks such as HotpotQA and MuSiQue. Likewise, C2C [7] sends a cache one way and evaluates on question-answering benchmarks. This setup also lets us check that communication really happens. We give each receiver the message written for a different example and change nothing else. Our single-document message then drops from 0.356 to 0.216 in validation exact match, near 0.196 with no message. The receiver thus relies on the sender's information, not on a generic prefix. We state this single-round scope as a limitation in §5 and leave multi-round and tool-use tasks for future work.

The sender writes its message without seeing the question. The same message can therefore serve every question about the same document. In MuSiQue's test split, 61 sender documents are each paired with several questions. We write one message per document with the single-document writer trained on HotpotQA and reuse it for all of that document's questions. This single message still beats selection, by 0.067 at 32 entries and 0.050 at 64.

Q3. Latency and serving. We timed Qwen3-8B as sender and receiver on one L40S with five documents. Sending the raw document as text is faster at batch 1. From batch 4 on, however, ours is faster end to end.

Seconds per example, by batch size1481632
32-entry message0.5470.3980.3970.3930.393
Raw document sent as text0.4980.4190.4350.4560.457

Our message also needs little memory. It takes 4.7 MB at 32 entries, against 96 MB on average for the full cache of five documents. Sending the full cache runs out of memory at batch 32. Quantisation shrinks our message further, to 2.4 MB with int8 and 1.2 MB with int4. In the single-document setting, neither lowers exact match. For serving, a written message can be loaded into vLLM through its KV-connector interface, and we will add this connector with the code release.

References

[1] KV-Distill: Nearly Lossless Learnable Context Compression for LLMs, Chari et al., 2025.

[2] LoRA: Low-Rank Adaptation of Large Language Models, Hu et al., ICLR 2022.

[3] MiniLLM: Knowledge Distillation of Large Language Models, Gu et al., ICLR 2024.

[4] On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes, Agarwal et al., ICLR 2024.

[5] Communicating Activations Between Language Model Agents, Ramesh & Li, ICML 2025.

[6] KVComm: Enabling Efficient LLM Communication through Selective KV Sharing, Shi et al., ICLR 2026.

[7] Cache-to-Cache: Direct Semantic Communication Between Large Language Models, Fu et al., ICLR 2026.

AuthorsOpenReview ↗

Response to Reviewer bXyD (2/2)

Q4. Why a fixed calibration works across model families. Entries projected from another model family are hard for the receiver to read. Our first pair has Llama-3.1-8B writing for Qwen3-8B, and its projected entries score only 0.216 on test, against 0.180 with no message. To make them readable, we first train a low-rank correction to the projection by backpropagating through the frozen receiver. This correction turns out to be nearly the same for every input. Its average over training inputs keeps 95% of its squared norm, and the part that varies across inputs holds only 5%. Giving an input the correction computed for another input also barely changes exact match. We therefore replace the correction with its average. This average is a single fixed tensor added to every message, and we call it the calibration. The calibration raises exact match to 0.346 on the first pair. It is the same for every input and only fixes the format. The sender adapter then writes what is specific to each input, as in the same-family case. It adds 0.034 and 0.055 on the two pairs. This gain comes from the sender's information. Swapping in the update computed for a different input lowers exact match by 0.105 and 0.080. The adapter also brings the message to the level of a writer trained within the receiver's family. Our second pair has Qwen3-8B writing for Falcon3-7B, and its message without the adapter scores 0.049 below the Falcon3-7B writer. Adding the adapter raises it to 0.393, against 0.387 for that writer. Section 4.5 and Appendix G of the revised PDF give the details and the calibration controls.

Q5. Evidence that the message conveys information. We take the five-document setting at 64 entries and ask whether our message can pass on an answer that selection has dropped. Selection keeps 64 of the sender's positions. In 749 validation and test examples, the answer lies in the sender's documents but outside these 64 positions and outside the receiver's evidence. Selection scores 0.043 on these examples, about the same as no message at 0.040. Our message is built on the same 64 positions, yet it scores 0.139. The written entries therefore contain answers that the selected entries do not. Appendix E.6 of the revised PDF gives the details.

Q6. Student rollouts. We use student rollouts (also known as on-policy rollouts) by default for two reasons. First, they match deployment. The receiver sees only the written message at deployment, not the sender's full context. Student rollouts train the writer on the receiver's answers in this same condition. Second, they are significantly better at the tight budget of 32 entries. The two are close at larger budgets. Teacher rollouts lead slightly at 64 entries, and the two match at 128 and 256.

Five documents, first 1,000 test examples, 95% intervals32 entries64 entries128 entries256 entries
Student rollouts - teacher rollouts+0.013 [+0.002, +0.026]−0.013 [−0.026, −0.001]−0.007 [−0.021, +0.008]−0.011 [−0.026, +0.004]

Thank you for the detailed responses and additional experiments. The new comparisons with KV-Distill, serving efficiency measurements, and further analyses of information encoding, cross-family calibration, and student rollouts address most of my original concerns.

The message reuse and cross-family communication experiments also provide encouraging initial evidence of the method's broader applicability. While evaluation on more complex agentic tasks would further strengthen the paper, I think the additional results have improved my assessment. I am inclined to raise my score.

Reviewer_dQhEOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes CacheWriter, a method for learning to communicate latent messages between LLMs by learning to apply a LoRA transformation to selected KV-cache values from the sender such that they can be attended to directly by the receiver, which may even be from a different LLM family. Thus, communication between the sender and receiver can omit the decode of a text message and prefill in the receiver, as well as compress more information into shorter latent messages.

Critical strengths

The paper includes extensive experiments evaluating the method at different compression budgets (Table 8, Fig. 6, Table 7), finding at the smallest budgets (<30% of original KV cache size), CacheWriter out-performs baselines and utilizes fewer tokens.

Critical weaknesses
  1. The method requires two inference passes from the sender: first without the adapter and then with (L186-194), where most baselines would only need one. This adds overhead where the method is motivated by reducing the overhead induced by communication.
  2. I think the paper's communication setting is too simplistic for the multi-agent communication described in the introduction and realistic multi-agent communication generally. To my understanding, an LLM aa (sender) is given a document context xax_a, and an LLM bb (receiver) is given a query qq and document xbx_b. In the multi-hop QA setting, xa,xbx_a, x_b might be two passages which together are needed to answer question qq correctly. This work thus estimates the ability of the sender to compute a compact representation of the information in xax_a such that LLM bb can attend to it as keys and values. As a learning problem for the chosen benchmarks and experimental setting, I think this is essentially a document compression task, rather than actually modeling a realistic communication scenario that might normally occur between two agents as text.
  3. The paper is very difficult to read with many signals that the paper is largely AI generated: 5 uses of ‘sits’ as a verb including two ‘where ___ sits’, several compound words and terms which are not clearly defined and must instead be understood through use. I think the paper needs to be carefully re-written.
Questions for authors

None

1 comment
AuthorsOpenReview ↗

Response to Reviewer dQhE

Thank you for the careful reading. Your comments helped us make the paper clearer and easier to follow.

W1. Two sender passes. The single-document writer in the submission runs the sender twice, but the first pass is not needed. The first pass scores and selects the positions for the second pass, which writes their entries with the adapter. To remove the first pass, we let the adapted pass also score the positions. This one-pass writer changes test exact match by only 0.003. A deployed sender therefore needs only one pass. The five-document writer never needed two passes, because it always keeps the first token and the last tokens of the documents without scoring them. Even the two-pass writer is cheap. It takes 0.160 s per single-document message, 1.07 times the time that selection alone needs. We hope this resolves your concern.

W2. Problem setup and document compression. We follow the problem setup of prior work on communication between LLM agents. A sender holds information that a receiver needs to answer its question, and the sender sends it a single message. Activation communication [1], KVComm [2] and C2C [3] all send information one way and once. KVComm and C2C evaluate on question-answering benchmarks such as HotpotQA and MuSiQue. This setup also lets us test whether communication really takes place. We replace the single message with the one written for a different example and keep everything else fixed. In the single-document setting, this swap lowers exact match from 0.356 to 0.216 on validation, close to 0.196 with no message. The receiver therefore uses the sender's information, not a generic prefix shared by all inputs. Multi-round dialogue would not allow such a clean swap, because later turns depend on earlier ones. We state this single-round scope as a limitation in §5 and leave multi-round and tool-use tasks to future work.

CacheWriter also differs from document compression, such as KV-Distill [4], in four ways. First, a compressor compresses a document for its own use. Our message serves the receiver, a different agent that holds its own evidence and a question that the sender never sees. Second, our training target comes from the receiver. The target is the receiver's output when the receiver sees both its own evidence and the sender's full context. In our five-document setting, KV-Distill's objective leaves out the receiver's evidence and lowers test exact match from 0.196 to 0.143 at 64 entries. Third, KV-Distill learns which tokens to keep, while CacheWriter fixes them and learns what to write into them. We reimplement its learned selector for our task, and it scores 0.162 at 32 entries, significantly below 0.189 for CacheWriter. The two match at 64 entries. Fourth, compression stays within one model, while CacheWriter also communicates across model families. There, our message beats a same-length note by +0.120 and +0.179 on test.

W3. Writing. We carefully revised the writing throughout the paper for readability. Every change serves to make our design and results easier to follow, and the experiments and main conclusions stay the same.

S1. Gains at small budgets. Thank you for acknowledging our gains at small budgets. We would like to clarify that they also hold at larger budgets, which we test in both settings. In the single-document setting, the sender holds one paragraph of 112 to 191 tokens on average, depending on the benchmark. Selection alone at 64 entries already keeps the cache entries of a third to more than half of it, and less room is left for writing to add information. Even so, our message still leads the strongest same-length baseline on average and in 11 of the 16 cases, which cover four models on four benchmarks. In the five-document setting, we train CacheWriter at 32, 64, 128 and 256 entries. It leads selection at every budget, and the lead stays significant at 256 entries. We also compare with KVComm at a similar budget. On single-document HotpotQA, our 32-entry message sends 28% of the full cache and recovers 58% of the gap between no message and the full context. KVComm sends 30% and recovers only 15%.

References

[1] Communicating Activations Between Language Model Agents, Ramesh & Li, ICML 2025.

[2] KVComm: Enabling Efficient LLM Communication through Selective KV Sharing, Shi et al., ICLR 2026.

[3] Cache-to-Cache: Direct Semantic Communication Between Large Language Models, Fu et al., ICLR 2026.

[4] KV-Distill: Nearly Lossless Learnable Context Compression for LLMs, Chari et al., 2025.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.