- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper studies KV-cache compression for latent communication in multi-agent LLM systems. Agents communicates with KV caches rather than text. The paper compresses each agent’s prompt KV using attention-based eviction and proposes Orthogonal BackFill. Experiment on Qwen models across different benchmarks show about 4.7× lower relay volume and 5–15% lower peak GPU memory.
Critical strengths
- The paper provides a clear introduction to LatentMAS and its KV-cache communication mechanism.
- The evaluation covers nine benchmarks across diverse tasks.
Critical weaknesses
- I am not convinced of the necessity of latent communication in multi-agent systems, as most existing frameworks communicate through text. A stronger motivation in the Introduction would help clarify why latent communication is needed. Moreover, most evaluated benchmarks, such as GSM8K and QA tasks, do not clearly demonstrate the necessity of multi-agent collaboration over a single agent, let alone the need for lossless KV-cache communication. I believe comparisons with single-agent baselines in both accuracy and efficiency are necessary.
- The novelty of the proposed techniques is unclear to me. Attention sinks and KV eviction have been studied in existing LLM research. Orthogonal BackFill appears to combine existing KV eviction techniques with orthogonal projection and PCA compensation. It is not discussed in the paper how this distinguish from existing KV compression methods. Moreover, the evaluation results cannot prove the effectiveness. In Table 2, L-OBF achieves the best accuracy on only one of five benchmarks, and its average improvement is marginal. Without reporting variance or statistical significance, it is difficult to assess whether these improvements are meaningful. Moreover, some compressed methods even outperform Full-KV, but the paper does not fully explain why, which is confusing. Is it becasuse of compression itself can optimize the communication, or simply fall within experimental variability?
- The biggest concern is the evaluation. Although the paper claims to optimize inter-agent communication, it does not demonstrate that communication is a significant bottleneck in multi-agent systems. The evaluation mainly focuses on accuracy and memory reduction, without demonstrating end-to-end performance improvements from reduced communication. This raises concerns about the practical benefits of the proposed method. Furthermore, why is the evaluation limited to a sequential multi-agent setting?
- Writing is poor. The technique section is overloaded with mathematical formulations, making it difficult to follow.
- Figure 4 is overly crowded, with too many overlapping symbols and labels, making the main trends difficult to identify. I suggest using fewer representative benchmarks and clearer annotations to better illustrate the key findings
Questions for authors
See weaknesses above.
6 comments
Response to Reviewer 6GeX (1/3)
W1. Why latent communication, the benchmark choice, and single-agent baselines
This weakness holds three requests, a stronger motivation for latent communication, benchmarks that show multi-agent collaboration is needed, and single-agent comparisons in accuracy and efficiency. We take them in that order.
1.a. We argue that "most existing frameworks communicate through text" is not a valid reason for being "not convinced of the necessity of latent communication in multi-agent systems". How many systems use a medium today describes present practice, not whether the medium is right or an alternative worth studying. Work on any new channel necessarily begins while the old one still dominates. Latent reasoning is the closest case. When it appeared most systems reasoned in plain text, and the same doubt applied to it. Yet the area now has two surveys from top universities and companies (https://arxiv.org/abs/2507.06203, https://arxiv.org/abs/2604.02029). The second has a section on collaboration, covers latent communication between agents, and cites LatentMAS. Both start from an observation similar to the one our introduction makes. Exploring a medium before it becomes standard is part of what research is for.
1.b. The Introduction and Section 2 already provide the motivation for latent communication (lines 28–39 and 72–93). We present the development of latent communication and the motivation and mechanism behind each stage. We also discuss information fidelity with respect to the communication medium. If any step fails to explain the motivation of latent communication, we would be glad to know which and will expand that step in the revision.
1.c. We argue that the necessity of latent communication should be, and is, established by LatentMAS rather than by this paper. This paper builds on LatentMAS, a spotlight at ICML 2026. The relay mechanism is the core of what we build on, and the preliminaries explain it thoroughly. Proving that a paradigm is worthwhile is the job of the paper that introduces it. A follow-up that adds a compression method takes the paradigm as given and is responsible for that method. None of the three contributions we state argues that agents ought to communicate in latent space. We ask that the paper be assessed against the claims it makes.
2. Inheriting the LatentMAS benchmarks is the conservative choice, and we state its limits in the paper. We use the same nine benchmarks as LatentMAS. Changing the benchmarks and the relay method at the same time would make any difference impossible to attribute. Latent communication is at an early stage, and our limitations already state that long-form collaboration, tool use and full agentic workflows remain untested (lines 481–483). Showing the operator on the benchmarks the paradigm was introduced with is the first step, and harder settings are the natural next one.
3. A single-agent baseline does not belong in this paper's comparison. A single agent relays nothing, so it has no channel for a compression operator to act on. Section 2 cites the evidence that multi-agent systems outperform single agents. Our reference point is Full-KV relay, against which every compressed row is measured. Whether the relay beats a single agent follows the same reasoning as 1.c, which we do not repeat for brevity.
Response to Reviewer 6GeX (2/3)
W2. Novelty, and whether the gains are real
This weakness raises three points, the novelty of OBF, the size and significance of its gains in Table 2, and why some compressed methods exceed Full-KV. We take them in that order.
1.a. We argue that the statement "It is not discussed in the paper how this distinguish from existing KV compression methods" is inaccurate. The related work names each compensation method and states how OBF differs from them (lines 99–107).
RazorAttention summarizes removed context through compensation tokens in non-retrieval heads, while CaM and WeightedKV merge discarded information into retained states instead of removing it outright. All of these target single-agent online inference, where compression interleaves with token generation rather than acting at an inter-agent boundary. (Section 2)
In contrast, we compress once at the relay boundary, evicting a whole block rather than token by token during generation, and ask only whether the result still serves the next agent. [...] OBF also injects only the orthogonal residual and broadcasts one low-rank correction to all retained values rather than merging deleted tokens individually. (Section 2, Positioning)
1.b. "combine existing KV eviction techniques with [...] compensation" applies to every compensation method. Compensation exists only because something was evicted, and this holds for CaM and WeightedKV as well. What distinguishes one compensation method from another is how it compensates, which is the subject of 1.c.
1.c. The "orthogonal projection and PCA" are exactly our contribution, and the paper explains why each is there. The design rests on three decisions, compensating collectively over the whole discarded block, injecting only the part orthogonal to what is retained, and broadcasting one correction to every retained value. The boxed "Why" paragraphs in Section 4.4 give the reason for each step, and Appendix D ablates each one. Without the projection, compensation does worse than eviction with no compensation at all.
2.a. No method in Table 2 wins more than two of the five task columns, and the methods compare at different compression ratios. Three of the four columns that L-OBF does not win go to int8 and int4 quantization, which compress the relay 1.9x and 3.7x against L-OBF's 4.6x. What the table shows is that L-OBF stays in the same accuracy range at a higher compression ratio, and that it combines with int8 quantization to reach 9.0x.
2.b. The gains are small on any one benchmark and consistent across them. Significance on a single benchmark is out of reach at three runs, as the general response explains, so we test pooled across the 21 model and benchmark cells. For L-OBF, the variant in Table 2, the gain rises with how much layerwise eviction lost, r = 0.66 with p = 0.001 (Figure 4). For H-OBF the same relation gives r = 0.78 with p < 0.0001. Pooled over the 21 cells, H-OBF is also higher than headwise eviction on average and wins 17 of them, a two-sided sign test at p = 0.007. Gains that were seed noise would not line up with how much eviction lost. The general response gives the paired difference with its 95% interval, and the same test on the fast path.
3.a. The paper states its hypothesis in Section 6 (lines 449–452). Eviction also removes content that does not help the receiving agent, which acts as denoising.
Preserving the entire cache is not always necessary for effective downstream continuation. At 4B, compressed variants perform better on coding tasks, while Full remains stronger on reasoning and QA benchmarks (Table 1). These differences suggest that compression can remove useful context as well as content that does not help the receiving agent. (Section 6)
We keep the statement brief on purpose. It is a hypothesis, we have not run an experiment that isolates it, and we decline to draw a conclusion from it.
3.b. On the reviewer's question, both are present, and we state the denoising effect as a hypothesis. Most of the gains over Full fall within experimental variability. On 4B HumanEval+, L-OBF reaches 69.11 against Full's 68.90 with seed deviations of 2.75 and 2.66. One case is larger than seed variation. The generation-only relay on 4B MBPP+, which drops all prompt states, is 3.53 pp above Full and consistent across seeds, as the hypothesis would predict. The significance tests are addressed in the general response, and we consider pooled significance the appropriate level of evidence for a three-seed protocol.
Response to Reviewer 6GeX (Summary)
We thank the reviewer for the review. Our reply is long, so this comment summarises it. The details follow in three comments (W1, W2, and W3 to W5).
A. Two points in the review are already addressed in the submission
1. The review states that "It is not discussed in the paper how this distinguish from existing KV compression methods". Section 2 discusses it (lines 104–107).
The contribution is this compensation, which is collective over the whole discarded block, orthogonal to what is retained, and broadcast to every retained value.
2. The review asks why some compressed methods exceed Full-KV. Section 6 states our hypothesis (lines 449–452). We keep it brief on purpose, because we have no experiment that isolates it.
B. The first weakness concerns the LatentMAS paradigm, not this paper
1. "most existing frameworks communicate through text" is not a valid reason for being "not convinced of the necessity of latent communication in multi-agent systems". How many systems use a medium today does not show that an alternative is unnecessary.
2. Even if it were, establishing that necessity is not the responsibility of this paper. LatentMAS, a spotlight at ICML 2026 which we build on, has already done so, and none of our three stated contributions argues for it. We ask that the paper be judged against the claims it makes. The requests for other benchmarks and for single-agent baselines follow from the same premise.
C. The requests on motivation and on equations point to no specific step or equation
1. The Introduction and Section 2 argue the motivation step by step. Text is lossy, so work moved toward channels that keep more of the model's internal state, and LatentMAS completes this line by relaying the full KV cache. That fidelity is expensive, and this cost is what our paper addresses.
If any step fails to explain the motivation of latent communication, we ask the reviewer to point to it.
2. Reducing the cognitive load of reading and understanding the method is always our priority. We introduce the method in three steps, one at a time. After each step we added a boxed "Why" paragraph that explains it in words (lines 263–265, 280–282 and 295–298), and Figure 3 draws the three steps geometrically. If the reviewer finds one of the seven numbered equations redundant, we would be glad to know which.
D. The evaluation concerns are answered in the general response
A real cross-device relay is a separate infrastructure project, and its result would depend on the link. The hierarchical setting in LatentMAS is still a sequential chain, so it would not test the operator differently. End-to-end time for every method is in Section 5.2 and Appendix I.
E. What we accept
We now give pooled significance and paired intervals (see the general response), and we will simplify Figure 4 and the definitions at the start of Section 3.