acceptodds
Under review as a conference paper at ICLR 2027

Compressing KV-Cache Communication in Multi-Agent Systems with Collective Compensation

Abstract

Multi-agent LLM systems are moving beyond discrete-token messages toward richer relays that preserve internal state. Recent work such as LatentMAS transmits full key-value (KV) caches between agents but pays a high memory and communication cost. We adapt KV-cache eviction to inter-agent relay and introduce Orthogonal BackFill (OBF). OBF collectively recovers complementary information from discarded values and incorporates it into the retained values. Eviction reduces per-agent relay volume by about 4.7×, with peak GPU memory savings of 5%–15% across model sizes. Across seven benchmarks at Qwen3-4B, compressed relay retains only 9.9%–20.2% of prompt KV while remaining close to full relay in accuracy and exceeding it on both coding benchmarks. At the same relay budget, adding OBF improves accuracy over eviction alone in 31 of 42 comparisons across Qwen3-4B, 8B, and 14B, with headwise OBF improving in 17 of 21 comparisons.

Primary area:
foundation or frontier models, including LLMs
Keywords:
KV Cache Compression, Large Language Model, Multi-Agent System

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_6GeXOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies KV-cache compression for latent communication in multi-agent LLM systems. Agents communicates with KV caches rather than text. The paper compresses each agent’s prompt KV using attention-based eviction and proposes Orthogonal BackFill. Experiment on Qwen models across different benchmarks show about 4.7× lower relay volume and 5–15% lower peak GPU memory.

Critical strengths
  1. The paper provides a clear introduction to LatentMAS and its KV-cache communication mechanism.
  2. The evaluation covers nine benchmarks across diverse tasks.
Critical weaknesses
  1. I am not convinced of the necessity of latent communication in multi-agent systems, as most existing frameworks communicate through text. A stronger motivation in the Introduction would help clarify why latent communication is needed. Moreover, most evaluated benchmarks, such as GSM8K and QA tasks, do not clearly demonstrate the necessity of multi-agent collaboration over a single agent, let alone the need for lossless KV-cache communication. I believe comparisons with single-agent baselines in both accuracy and efficiency are necessary.
  2. The novelty of the proposed techniques is unclear to me. Attention sinks and KV eviction have been studied in existing LLM research. Orthogonal BackFill appears to combine existing KV eviction techniques with orthogonal projection and PCA compensation. It is not discussed in the paper how this distinguish from existing KV compression methods. Moreover, the evaluation results cannot prove the effectiveness. In Table 2, L-OBF achieves the best accuracy on only one of five benchmarks, and its average improvement is marginal. Without reporting variance or statistical significance, it is difficult to assess whether these improvements are meaningful. Moreover, some compressed methods even outperform Full-KV, but the paper does not fully explain why, which is confusing. Is it becasuse of compression itself can optimize the communication, or simply fall within experimental variability?
  3. The biggest concern is the evaluation. Although the paper claims to optimize inter-agent communication, it does not demonstrate that communication is a significant bottleneck in multi-agent systems. The evaluation mainly focuses on accuracy and memory reduction, without demonstrating end-to-end performance improvements from reduced communication. This raises concerns about the practical benefits of the proposed method. Furthermore, why is the evaluation limited to a sequential multi-agent setting?
  4. Writing is poor. The technique section is overloaded with mathematical formulations, making it difficult to follow.
  5. Figure 4 is overly crowded, with too many overlapping symbols and labels, making the main trends difficult to identify. I suggest using fewer representative benchmarks and clearer annotations to better illustrate the key findings
Questions for authors

See weaknesses above.

6 comments
AuthorsOpenReview ↗

Response to Reviewer 6GeX (Summary)

We thank the reviewer for the review. Our reply is long, so this comment summarises it. The details follow in three comments (W1, W2, and W3 to W5).

A. Two points in the review are already addressed in the submission

1. The review states that "It is not discussed in the paper how this distinguish from existing KV compression methods". Section 2 discusses it (lines 104–107).

In contrast, we compress once at the relay boundary, evicting a whole block rather than token by token during generation [...]. OBF also injects only the orthogonal residual and broadcasts one low-rank correction to all retained values rather than merging deleted tokens individually.

The contribution is this compensation, which is collective over the whole discarded block, orthogonal to what is retained, and broadcast to every retained value.

2. The review asks why some compressed methods exceed Full-KV. Section 6 states our hypothesis (lines 449–452). We keep it brief on purpose, because we have no experiment that isolates it.

These differences suggest that compression can remove useful context as well as content that does not help the receiving agent.

B. The first weakness concerns the LatentMAS paradigm, not this paper

1. "most existing frameworks communicate through text" is not a valid reason for being "not convinced of the necessity of latent communication in multi-agent systems". How many systems use a medium today does not show that an alternative is unnecessary.

2. Even if it were, establishing that necessity is not the responsibility of this paper. LatentMAS, a spotlight at ICML 2026 which we build on, has already done so, and none of our three stated contributions argues for it. We ask that the paper be judged against the claims it makes. The requests for other benchmarks and for single-agent baselines follow from the same premise.

C. The requests on motivation and on equations point to no specific step or equation

1. The Introduction and Section 2 argue the motivation step by step. Text is lossy, so work moved toward channels that keep more of the model's internal state, and LatentMAS completes this line by relaying the full KV cache. That fidelity is expensive, and this cost is what our paper addresses.

Text-based communication is lossy. The model produces a probability distribution over the vocabulary, but only one sampled token is transmitted, discarding all other plausible alternatives. (Section 1, lines 31–33)

All of these frameworks communicate through natural language, even though text is a lossy compression of each agent's internal state and is not strictly necessary for inter-agent exchange. This motivates relaying internal state without decoding it to text. (Section 2, lines 76–78)

If any step fails to explain the motivation of latent communication, we ask the reviewer to point to it.

2. Reducing the cognitive load of reading and understanding the method is always our priority. We introduce the method in three steps, one at a time. After each step we added a boxed "Why" paragraph that explains it in words (lines 263–265, 280–282 and 295–298), and Figure 3 draws the three steps geometrically. If the reviewer finds one of the seven numbered equations redundant, we would be glad to know which.

D. The evaluation concerns are answered in the general response

A real cross-device relay is a separate infrastructure project, and its result would depend on the link. The hierarchical setting in LatentMAS is still a sequential chain, so it would not test the operator differently. End-to-end time for every method is in Section 5.2 and Appendix I.

E. What we accept

We now give pooled significance and paired intervals (see the general response), and we will simplify Figure 4 and the definitions at the start of Section 3.

AuthorsOpenReview ↗

Response to Reviewer 6GeX (1/3)

W1. Why latent communication, the benchmark choice, and single-agent baselines

This weakness holds three requests, a stronger motivation for latent communication, benchmarks that show multi-agent collaboration is needed, and single-agent comparisons in accuracy and efficiency. We take them in that order.

1.a. We argue that "most existing frameworks communicate through text" is not a valid reason for being "not convinced of the necessity of latent communication in multi-agent systems". How many systems use a medium today describes present practice, not whether the medium is right or an alternative worth studying. Work on any new channel necessarily begins while the old one still dominates. Latent reasoning is the closest case. When it appeared most systems reasoned in plain text, and the same doubt applied to it. Yet the area now has two surveys from top universities and companies (https://arxiv.org/abs/2507.06203, https://arxiv.org/abs/2604.02029). The second has a section on collaboration, covers latent communication between agents, and cites LatentMAS. Both start from an observation similar to the one our introduction makes. Exploring a medium before it becomes standard is part of what research is for.

1.b. The Introduction and Section 2 already provide the motivation for latent communication (lines 28–39 and 72–93). We present the development of latent communication and the motivation and mechanism behind each stage. We also discuss information fidelity with respect to the communication medium. If any step fails to explain the motivation of latent communication, we would be glad to know which and will expand that step in the revision.

1.c. We argue that the necessity of latent communication should be, and is, established by LatentMAS rather than by this paper. This paper builds on LatentMAS, a spotlight at ICML 2026. The relay mechanism is the core of what we build on, and the preliminaries explain it thoroughly. Proving that a paradigm is worthwhile is the job of the paper that introduces it. A follow-up that adds a compression method takes the paradigm as given and is responsible for that method. None of the three contributions we state argues that agents ought to communicate in latent space. We ask that the paper be assessed against the claims it makes.

2. Inheriting the LatentMAS benchmarks is the conservative choice, and we state its limits in the paper. We use the same nine benchmarks as LatentMAS. Changing the benchmarks and the relay method at the same time would make any difference impossible to attribute. Latent communication is at an early stage, and our limitations already state that long-form collaboration, tool use and full agentic workflows remain untested (lines 481–483). Showing the operator on the benchmarks the paradigm was introduced with is the first step, and harder settings are the natural next one.

3. A single-agent baseline does not belong in this paper's comparison. A single agent relays nothing, so it has no channel for a compression operator to act on. Section 2 cites the evidence that multi-agent systems outperform single agents. Our reference point is Full-KV relay, against which every compressed row is measured. Whether the relay beats a single agent follows the same reasoning as 1.c, which we do not repeat for brevity.

AuthorsOpenReview ↗

Response to Reviewer 6GeX (2/3)

W2. Novelty, and whether the gains are real

This weakness raises three points, the novelty of OBF, the size and significance of its gains in Table 2, and why some compressed methods exceed Full-KV. We take them in that order.

1.a. We argue that the statement "It is not discussed in the paper how this distinguish from existing KV compression methods" is inaccurate. The related work names each compensation method and states how OBF differs from them (lines 99–107).

RazorAttention summarizes removed context through compensation tokens in non-retrieval heads, while CaM and WeightedKV merge discarded information into retained states instead of removing it outright. All of these target single-agent online inference, where compression interleaves with token generation rather than acting at an inter-agent boundary. (Section 2)

In contrast, we compress once at the relay boundary, evicting a whole block rather than token by token during generation, and ask only whether the result still serves the next agent. [...] OBF also injects only the orthogonal residual and broadcasts one low-rank correction to all retained values rather than merging deleted tokens individually. (Section 2, Positioning)

1.b. "combine existing KV eviction techniques with [...] compensation" applies to every compensation method. Compensation exists only because something was evicted, and this holds for CaM and WeightedKV as well. What distinguishes one compensation method from another is how it compensates, which is the subject of 1.c.

1.c. The "orthogonal projection and PCA" are exactly our contribution, and the paper explains why each is there. The design rests on three decisions, compensating collectively over the whole discarded block, injecting only the part orthogonal to what is retained, and broadcasting one correction to every retained value. The boxed "Why" paragraphs in Section 4.4 give the reason for each step, and Appendix D ablates each one. Without the projection, compensation does worse than eviction with no compensation at all.

2.a. No method in Table 2 wins more than two of the five task columns, and the methods compare at different compression ratios. Three of the four columns that L-OBF does not win go to int8 and int4 quantization, which compress the relay 1.9x and 3.7x against L-OBF's 4.6x. What the table shows is that L-OBF stays in the same accuracy range at a higher compression ratio, and that it combines with int8 quantization to reach 9.0x.

2.b. The gains are small on any one benchmark and consistent across them. Significance on a single benchmark is out of reach at three runs, as the general response explains, so we test pooled across the 21 model and benchmark cells. For L-OBF, the variant in Table 2, the gain rises with how much layerwise eviction lost, r = 0.66 with p = 0.001 (Figure 4). For H-OBF the same relation gives r = 0.78 with p < 0.0001. Pooled over the 21 cells, H-OBF is also higher than headwise eviction on average and wins 17 of them, a two-sided sign test at p = 0.007. Gains that were seed noise would not line up with how much eviction lost. The general response gives the paired difference with its 95% interval, and the same test on the fast path.

3.a. The paper states its hypothesis in Section 6 (lines 449–452). Eviction also removes content that does not help the receiving agent, which acts as denoising.

Preserving the entire cache is not always necessary for effective downstream continuation. At 4B, compressed variants perform better on coding tasks, while Full remains stronger on reasoning and QA benchmarks (Table 1). These differences suggest that compression can remove useful context as well as content that does not help the receiving agent. (Section 6)

We keep the statement brief on purpose. It is a hypothesis, we have not run an experiment that isolates it, and we decline to draw a conclusion from it.

3.b. On the reviewer's question, both are present, and we state the denoising effect as a hypothesis. Most of the gains over Full fall within experimental variability. On 4B HumanEval+, L-OBF reaches 69.11 against Full's 68.90 with seed deviations of 2.75 and 2.66. One case is larger than seed variation. The generation-only relay on 4B MBPP+, which drops all prompt states, is 3.53 pp above Full and consistent across seeds, as the hypothesis would predict. The significance tests are addressed in the general response, and we consider pooled significance the appropriate level of evidence for a three-seed protocol.

Reviewer_U5XEOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates KV-cache communication compression in sequential LLM-based multi-agent systems. Building on LatentMAS, the authors adapt attention-based cache eviction to inter-agent communication and propose Orthogonal BackFill (OBF), which attempts to preserve information from discarded KV states without increasing the retained cache size.

Specifically, OBF projects discarded value vectors onto the orthogonal complement of the retained-value subspace, extracts a low-rank representation using SVD, and injects an attention-weighted correction into the retained values. The method is evaluated on Qwen3-4B, 8B, and 14B across reasoning, question-answering, and coding benchmarks. The results demonstrate substantial reductions in relay volume, with OBF providing additional accuracy improvements over standard eviction in several settings. The paper also includes component ablations, budget and rank sensitivity analyses, and runtime measurements.

Critical strengths
  1. A relevant problem formulation with a practical compression mechanism.

The paper addresses an important efficiency bottleneck in latent multi-agent communication. Compressing KV states at agent boundaries is a meaningful extension of existing single-agent cache management techniques. The proposed OBF mechanism is relatively straightforward, training-free, and compatible with different eviction granularities. Its ability to improve downstream accuracy without increasing the number of retained KV positions is practically appealing.

  1. Broad experimental analysis within the evaluated setting.

The authors evaluate multiple backbone sizes and task categories, and provide useful ablations of the projection, low-rank filtering, and scaling components. The budget and PCA-rank sweeps offer additional insight into when compensation is effective. I also appreciate the explicit comparison between exact and approximate SVD, including the reported accuracy deviations and computational overhead.

Critical weaknesses
  1. The claimed accuracy-efficiency trade-off is not convincingly established under a consistent configuration.

A central concern is that the main accuracy results rely on exact SVD, whereas the most favorable end-to-end efficiency results use approximate SVD.

According to Table 13, exact OBF increases end-to-end execution time to approximately 1.28–1.29 times that of Full LatentMAS. Although the fast SVD implementation reduces this overhead to approximately 1.04–1.05 times, Appendix H reports a 4.27 percentage-point accuracy decrease for L-OBF on HumanEval+ compared with exact SVD. Combining this decrease with Table 1 suggests that fast L-OBF achieves approximately 64.84% on HumanEval+, below both Full LatentMAS (68.90%) and layerwise eviction without OBF (65.45%).

Consequently, the strongest accuracy results and the most favorable latency results do not correspond to the same implementation. The paper would benefit from a unified accuracy-latency comparison that makes this trade-off explicit.

Furthermore, all agents are simulated within a single process on one GPU, without actual inter-device communication. While the reduction in relay tensor size is clearly demonstrated, its benefit for distributed multi-agent systems remains unverified. This limitation is acknowledged by the authors, but it substantially affects the practical significance of the communication-efficiency claims.

  1. The robustness and attribution of OBF's improvements require stronger evidence.

Although OBF improves over eviction in many reported comparisons, the gains are not consistently substantial across models and benchmarks. For example, on Qwen3-8B, H-OBF has a slightly lower mean accuracy than headwise eviction alone across the seven non-AIME benchmarks. A similar pattern occurs for L-OBF on Qwen3-14B. On Qwen3-4B, H-OBF also remains 5.25 percentage points below Full LatentMAS on GPQA.

The reported 31 improvements out of 42 comparisons provide useful descriptive evidence, but do not establish statistical reliability. Most accuracy results are based on three random seeds, and the main comparison tables do not provide uncertainty estimates for the improvements. This is particularly relevant when the reported differences are small.

There is also an unresolved implementation detail concerning positional consistency. Appendix A.7 describes updating position cursors according to compressed cache lengths, but does not explicitly explain how the original RoPE positions of retained keys are handled after eviction. This is important because the retained keys may have been computed at different absolute positions from those implied by the compressed cache. The precise handling of this issue should be clarified before the method's soundness can be fully assessed.

Questions for authors

Q1. Accuracy-efficiency consistency. Can the authors provide a unified accuracy-versus-end-to-end-latency comparison for exact SVD, fast SVD, eviction-only baselines, and Full LatentMAS under identical evaluation settings? In particular, does fast OBF remain preferable to eviction alone when both accuracy and compression overhead are considered, especially on HumanEval+?

Q2. Positional consistency after eviction. In Appendix A.7, the position cursor advances according to the number of retained KV entries. Since retained keys may already contain rotary positional information associated with their original, uncompressed positions, are these keys re-rotated or otherwise adjusted after compression? How is positional consistency maintained when different layers or KV heads retain different token subsets? Additionally, how are attention scores aggregated from query heads that share the same KV head under grouped-query attention?

Q3. Statistical reliability and hyperparameter selection. Can the authors report per-seed results and paired uncertainty estimates for the main OBF-versus-eviction comparisons? How were the PCA ranks selected across backbone sizes, and was any held-out validation data used? These details would help establish whether the observed improvements are robust rather than influenced by sampling variability or hyperparameter selection.

Q4. Practical communication benefits and baseline competitiveness. Can the authors evaluate end-to-end performance under realistic inter-device or inter-node communication, or provide a bandwidth-sensitive latency analysis? It would also strengthen the contribution to compare OBF against more closely matched cache-merging alternatives, since the reported CaM-inspired baseline is not the original CaM implementation. Such evidence would help establish whether the additional orthogonal projection and low-rank computation provide a meaningful advantage over simpler compensation mechanisms.

2 comments

We thank the reviewer for the detailed and constructive review. We answer the four questions in order. Points shared with other reviewers are answered in full in the general response.

Q1. A unified accuracy against end-to-end latency comparison

The details and the unified table are in the general response under "Exact and fast SVD". The main observation is that the exact and fast paths almost match in accuracy. Two points specific to this question are below.

1. Fast OBF costs 6% to 9% more time per token than eviction alone. Whether that buys accuracy is the last column of that table. It does for H-OBF, and for L-OBF the two are not separable.

2. On HumanEval+, fast OBF is ahead of eviction in four of the six cells.

HumanEval+L evictL fastH evictH fast
4B65.4569.3162.4067.07
8B79.4781.1083.3381.30
14B85.5784.9684.7686.18

The two 4B gains are the large ones. HumanEval+ has 164 problems, so one problem is 0.61 pp, and the differences at 8B and 14B are within seed variation.


Q2. Positional consistency after eviction, and GQA aggregation

1. Retained keys are left completely unchanged. Yes, each key carries the rotary phase of its original position, and that makes keys very sensitive. In early experiments we tried modifying keys, and generation collapsed. OBF therefore writes to values only and never re-rotates or adjusts a key.

2. Because keys are unchanged, layers and heads that retain different subsets need no alignment. Every retained key still carries its own original absolute position. Eviction leaves gaps in the occupied positions and changes no relative distance between a retained key and a later query.

3. New tokens must therefore be placed at their absolute position, not at the position implied by the cache length. Since retained keys keep their original absolute positions, the next token has to continue that same timeline. For example, with 200 tokens of which 100 are evicted, the next token takes position 201, not 101. Otherwise generation breaks. This is what the cursor in Appendix A.7 does, and we will reword that paragraph to say so.

4. Under GQA, the attention scores of the query heads that share a KV head are summed, and those query heads share one retained set. The cache holds one key and value per KV head, so selection is made per KV head.


Q3. Statistical reliability and hyperparameter selection

1. Fast H-OBF is separable from headwise eviction, and fast L-OBF is not separable from layerwise eviction. The paired differences and their 95% intervals are in the last column of the table in the general response.

2. We will add per-seed results and standard deviations for every accuracy table in the appendix.

3. The PCA rank is fixed per variant and was chosen on Qwen3-4B alone, with no held-out validation data.

We fix pca_rank = 4 for L-OBF and pca_rank = 2 for H-OBF throughout the paper, chosen on Qwen3-4B before any Qwen3-8B or Qwen3-14B run [...]. (Section 5.1)

The 8B and 14B results therefore use a rank that was never tuned on them. Appendix G also shows that no single rank dominates within the range 2 to 8, so the result does not hinge on the exact value.


Q4. Practical communication benefits and baseline competitiveness

The full answer on communication is in the general response under "Real cross-device communication". The main reason is that cross-device does not name one setting. NVLink, cluster Ethernet and wireless links differ by orders of magnitude in cost per byte, so any single measurement would describe the link more than the method. Our answer on the baselines is below.

1. Our cache-merging baseline follows CaM's principle, with the adaptation that a relay requires. Like CaM, it recycles evicted values into retained slots and leaves keys untouched. CaM's merge target is the positions that follow the evicted entry, and at a relay those positions hold the latent reasoning states. Merging into them would overwrite the latent reasoning that the relay exists to pass on. We therefore route each evicted value to its most key-similar retained prompt slot, weighted by attention mass, and Appendix E states these differences. We do not expect a version that alters the latent states to do better.

2. Against this matched baseline, L-OBF has the better mean and the milder worst case. Table 2 compares the two at the same relay budget.

3. The paper already compares eight relay methods, and adapting each baseline to the relay boundary is our first stated contribution.

We give, to our knowledge, the first adaptation of KV-cache eviction to multi-agent latent relay [...]. (Section 1)

H2O's importance signal and CaM's merge target both assume a decoding stream that a relay does not have, so neither transfers unchanged. The comparison covers three model scales, nine benchmarks and three seeds, 1,418 runs in total.

Dear Reviewer,

Thank you again for your review. As the discussion period ends today, we would like to check whether our reply has addressed your concerns.

If it has, we would appreciate it if you could consider updating your score. If anything remains unclear, please let us know and we will answer right away.

Reviewer_5AEQOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies KV-cache compression for latent communication in multi-agent LLM systems. The authors propose Orthogonal BackFill (OBF), which selects important prompt KV entries based on attention scores and compensates for discarded information by extracting low-rank orthogonal residuals from evicted value states and injecting them into retained values. Experiments on various qwen models across multiple benchmarks show that OBF achieves approximately 4.7x KV relay compression and it maintains competitive downstream accuracy compared to full-cache communication.

Critical strengths
  1. OBF provides a training-free mechanism to compensate for information lost during KV eviction
  2. The paper evaluates multiple model sizes and benchmarks, with ablations examining the contributions of different OBF components
Critical weaknesses
  1. Although the paper discusses existing KV merging and compensation methods, the experimental comparisons are limited to a CaM-inspired implementation and quantization baselines. Stronger baseline comparisons are needed to establish the benefits of OBF
  2. Several relevant works on KV-based multi-agent communication and compression, such as KVComm (https://openreview.net/forum?id=F7rUng23nw) and CondenseFlow (https://aclanthology.org/2026.findings-acl.669), and more, are not adequately discussed
  3. Experiments are restricted to sequential multi-agent settings using Qwen3 models, without real cross-device communication measurements
  4. I have concern that the overhead of SVD has been too high for this method to work in practice because there are hundreds of SVD operations per sample
Questions for authors

Please check the weaknesses

3 comments

We thank the reviewer for the careful review and for pointing us to two works we should have engaged with. Questions shared with other reviewers are answered in full in the general response, and we summarise them here so that this reply reads on its own.

W1. Baselines limited to a CaM-inspired implementation and quantization

1. We already compare against four families of KV compression. The main table's H and Gen rows are H2O and StreamingLLM adapted to the relay boundary, and Table 2 and Appendix E add CaM-style merging and post-hoc quantization. Our adapted H2O is also close to SnapKV, because at a relay the selection is made once, from the latent block, rather than accumulated over decoding steps. The row names do not show this and we will annotate them.

2. Our comparison is also wider than the one in the concurrent work named. CondenseFlow's Table 5 compares three selection rules from one family, Random Pruning, H2O and SnapKV, against a dense baseline. Ours spans four families, and quantization does not appear in that comparison.

3. Each baseline here is one we adapted, because no prior work compresses latent communication in MAS. H2O's importance signal and CaM's merge target both assume a decoding stream that a relay does not have, so neither transfers unchanged.


W2. KVComm and CondenseFlow not adequately discussed

1. We will add CondenseFlow to the related work. It appeared two months before this submission and is concurrent. Two differences matter.

2. CondenseFlow has no latent reasoning. It compresses the cache an agent accumulates after ordinary text reasoning, while ours holds the 40 latent steps each agent performs before the hand-off, so compression acts on a different object.

3. CondenseFlow is not training-free. Its Latent Thought Condenser is a trained module, with probes learned under reconstruction, coverage and orthogonality losses, where OBF is a deterministic projection with no training and no per-model fitting.

4. KVComm addresses a different problem, and we will add it to Section 2 with this distinction. It reuses caches across agents whose prefixes diverge, so overlapping context is not re-prefilled. Its agents reason in text and it does not reduce what is relayed, so it does not give a baseline to compare against.


W3. Sequential settings, Qwen3 only, no real cross-device measurement

We answer the sequential and cross-device points in the general response. The short version is below.

1. Sequential setting. The hierarchical setting in LatentMAS is still a sequential chain, so it would not test the operator differently.

2. Qwen3 only. We follow LatentMAS in using the Qwen3 family, at three scales. Our limitations state that other model families are not yet tested.

3. Cross-device. A real cross-device relay is a separate infrastructure project, and its result would depend on the link. CondenseFlow is also measured on a single GPU.


W4. SVD overhead too high to be practical; hundreds of SVDs per sample

1. The count is right. On Qwen3-4B there are 36 layers times 8 KV heads, so 288 residual decompositions per relay boundary, and three boundaries per rollout.

2. An exact SVD is not required, because only the top two to four directions of the residual are needed. Section 4 uses a Gram-matrix subspace iteration, and Appendix H reports a 5.3x reduction in compression time for H-OBF.

3. With the fast path, OBF costs about the same per generated token as full relay, and its accuracy matches the exact path. See the general response for the numbers and the table.

4. The revision will report the fast path throughout.

Dear Reviewer,

Thank you again for your review. As the discussion period ends today, we would like to check whether our reply has addressed your concerns.

If it has, we would appreciate it if you could consider updating your score. If anything remains unclear, please let us know and we will answer right away.

Thanks for the response. Your description of cross-prefix cache reuse to avoid repeated prefilling appears to refer to the KVCOMM paper by Ye et al. The paper linked in my review is the KVComm paper by Shi et al. (ICLR 2026), which explicitly studies reducing KV transmission through layer selection.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General response to all reviewers

General response to all reviewers

We thank all reviewers. This note answers the questions that recur across the reviews.

Statistical significance (Reviewers 6GeX and U5XE)

1. Significance on a single benchmark is out of reach at three runs, and simply adding more runs would not reach it either. Separating a gain of this size on one benchmark needs about 20 seeds per cell, and each run takes about four A100-hours.

2. Our three runs strictly follow the LatentMAS protocol.

3. The only significance level we can give is pooled across benchmarks and scales. Pooled over the 21 model and benchmark cells, fast H-OBF beats headwise eviction in 15 of 20 cells with one tie (two-sided sign test, p = 0.041), and is +0.63 pp above it with a 95% interval of [+0.02, +1.25].


Real cross-device communication (Reviewers 5AEQ, 6GeX and U5XE)

1. Cross-device is a separate infrastructure project, not a missing experiment. LatentMAS provides no multi-device path at all, so measuring a real one means building the transport layer ourselves and running it on multi-node hardware.

2. Cross-device also does not name one setting. NVLink, cluster Ethernet and wireless links differ by orders of magnitude in cost per byte, so any single number characterises the fabric we used rather than the method.

3. What we do measure is device-independent. Relay volume per non-judger agent falls from about 290 MB to about 62 MB, a 4.7x reduction, and peak GPU memory by about 2 GB.

4. The concurrent work that Reviewer 5AEQ pointed us to is also measured on a single device. CondenseFlow reports end-to-end time with all agents on a single GPU (its Appendix D.2), and we hope the same standard can apply here.


Sequential setting only (Reviewers 5AEQ and 6GeX)

1. The hierarchical setting in LatentMAS does not implement a hierarchy. A hierarchy should mean that A, B and C each solve the question on their own and their latent reasoning is then passed to D, which answers. The implementation is still sequential. It only tells A, B and C in the prompt to solve independently, while B answers with A's full KV cache in context, and D decodes from the single cache accumulated along the chain rather than aggregating three.

2. Evaluating on it would not test the operator differently. It is the same chain under different prompts, so compression acts on the same relay.

3. Building a real hierarchy would mean designing part of LatentMAS, not evaluating our operator. It needs a new way to aggregate several caches, and the outcome would be unattributable. This is why we evaluate the sequential chain alone and state in our limitations that other topologies remain to be verified.


Exact and fast SVD (Reviewers 5AEQ and U5XE)

1. The submission mixed the two paths because the fast path was built after the accuracy experiments. We first tested whether the operator works, using the exact SVD, and optimised its speed later. The submission reported the fast path's cost and did not rerun accuracy on it.

2. The fast path now has its own accuracy on every cell, and it matches the exact path. We ran fast L-OBF and H-OBF on all 21 model and benchmark cells with three seeds, using 24 subspace iterations on 4B HumanEval+ and 12 elsewhere. Fast minus exact is -0.03 pp for H-OBF with a 95% interval of [-0.36, +0.30], and -0.11 pp for L-OBF with [-0.45, +0.24]. The one large gap in the submission was L-OBF on 4B HumanEval+, and raising the subspace iterations from 12 to 24 closes it, 69.31 against the exact path's 69.11.

3. The fast path removes the overhead of the exact SVD. Per generated token, exact OBF costs 1.27x and 1.30x of full relay. Fast OBF costs 0.97x and 1.00x, at a relay 4.7x smaller. The table is averaged over the three model scales, with seven benchmarks and three seeds each. The last column is the accuracy difference paired over the 21 model and benchmark cells, with its 95% interval.

methodcompress ss/1k tokaccuracyvs its eviction baseline
Full0.0019.21 (1.00x)78.95
L evict0.0617.53 (0.91x)77.67
L exact6.4524.48 (1.27x)77.92+0.25 [-0.21, +0.71]
L fast0.8918.66 (0.97x)77.81+0.14 [-0.37, +0.66]
H evict0.0717.61 (0.92x)77.60
H exact6.6724.91 (1.30x)78.26+0.66 [-0.08, +1.40]
H fast1.2419.21 (1.00x)78.23+0.63 [+0.02, +1.25]

4. The revision will report the fast path throughout. Accuracy and cost will then come from one implementation.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.