acceptodds
Under review as a conference paper at ICLR 2027

Receiver-Conditioned Latent Communication gives 94% CacheBack

Abstract

Multi-agent systems distribute large contexts across agents that communicate to solve a task. Text messages are compact but require decoding and may omit evidence the receiving agent needs. Recent latent communication instead transfers KV caches. This avoids text generation and can improve accuracy and latency. However, a full KV cache grows linearly with both the context an individual agent processes, and the number of agents that coordinate together. This raises memory and context costs, often far exceeding available GPU resources and context window sizes. Our key observation is that agents need only send what the receiving agent requires for its local task—which we call receiver-conditioned communication. The receiver agent passes the sender a small description of its information needs, which serves to filter and compress the sender agent's KV cache. CacheBack is a simple, robust, training-free instance of receiver conditioning based on the sender's attention weights. On FanOutQA, CacheBack with Qwen 3 removes 75% of the state the agent would otherwise receive, improving accuracy by 14.7 percentage points and reducing median task-completion latency by 3.2x relative to text communication. We show comparable improvements across model families that span dense Transformers, Mamba-attention hybrids, and sliding-window attention.

Primary area:
foundation or frontier models, including LLMs
Keywords:
receiver-conditioned communication, latent communication, latent collaboration, multi-agent systems, agent communication, KV-cache communication, long-context inference, long-context reasoning, latent reasoning, message selection, receiver-aware selection, attention-based selection, efficient LLM inference, distributed LLM systems, multi-agent reasoning, inter-agent communication, model communication, communication efficiency, inference efficiency, context management, scalable agent systems, hidden-state communication, continuous representations, task-conditioned communication, selective state transfer, LLM systems, KV Cache Compression

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_EQXBOpenReview ↗
Rating
2(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This work presents receiver-conditioned latent communication to mitigate the heavy memory and context overhead of full KV-cache transfer in multi-agent LLM systems. It proposes CacheBack, a training-free attention-driven selection mechanism that uses the receiver’s query to extract and transmit task-relevant sender states. Integrating query-aware position selection with compact state representations, CacheBack cuts communication cost while retaining critical information.

Critical strengths

This work features a formulation that addresses the scalability bottleneck of context re-expansion in full-KV-cache communication via receiver-conditioned selection without extra training. CacheBack adopts a lightweight, easy-to-implement design by correcting mean query attention to highlight highly attended positions, extending existing KV-cache selection strategies.

Critical weaknesses
  1. Methodological novelty is very constrained: CacheBack adapts existing KV-cache compression to multi-agent communication via receiver-query conditioning and heuristic attention reweighting. The paper needs clearer differentiation from prior query-aware KV-cache selection and stronger justification for the scoring function.

  2. Experimental scale and statistical validity are limited. Evaluations adopt merely 50 samples per benchmark, and only the Easy partition of LongBench v2 is tested due to text truncation issues, weakening generalizability. More difficult tasks, larger test sets and competitive baselines are required to solidify conclusions.

  3. Practical deployment constraints lack sufficient investigation. The method requires aligned sender-receiver representations and is validated mainly on high-bandwidth multi-GPU environments. Its performance under heterogeneous models, variable network conditions and dynamic receiver queries remains not clear.

Questions for authors

See Weakness.

2 comments
AuthorsOpenReview ↗

Reviewer 3 (1/2)

We thank the reviewer for a thoughtful review, and we appreciate the time and effort it takes to review papers. Below, we address the main concerns raised and will make the changes in the final version of the paper. Our main comment is that Cacheback is a simple, drop-in, training-free method that 1) addresses context-limit challenges in communication, and 2) outperforms existing methods across a variety of open attention-based model architectures.

Weakness 1: Methodological novelty and the case for attention correction

Receiver Conditioned Latent Communication (RCLC) addresses the context-window bottleneck created by combining the entire worker's state into the receiving agent, contradicting the context management benefit of delegation. Our key insight is that we can separate what we send versus how we represent it. What we send is therefore centered around the receiver’s information needs and context window (§2, Eqs. 1–2; Appendix A).

We then propose CacheBack, a training-free implementation of this formulation that builds on SnapKV’s attention-based position selection. We then hypothesize a specific failure of mean query attention where useful sender tokens that receive strong attention from an individual query token are evicted due to low values from other query tokens (Figure 3). CacheBack is designed to address this failure by upweighting concentrated attention scores. We test whether this correction improves recall on a heldout set of questions excluded from the end-to-end evaluation (§3.2; Appendix D).

To show the significance of this correction term, we ran an additional end-to-end ablation comparing mean query attention to cacheback on FanoutQA n=50 for Qwen 8B. Same-size text workers score 40.7%.

CompressionMean query attentionCacheBackDifference, pp [95% CI]
2×49.3%45.3%−4.0 [−11.3, +2.7]
4×52.0%55.3%+3.3 [−6.0, +12.7]
8×49.3%50.0%+0.7 [−8.0, +9.3]
16×36.0%46.0%+10.0 [+0.7, +20.0]
32×26.7%45.3%+18.7 [+10.0, +28.0]
64×20.0%30.0%+10.0 [+1.3, +19.3]
128×14.0%22.7%+8.7 [+2.0, +16.7]

At tested ratios from 16x compression onwards, we see CacheBack outperforms mean query attention, highlighting the significant impact of our correction method on downstream task accuracy. This now supplements the initial selector comparison shown in §3.2 and Appendix D.1, where we compare CacheBack with mean query attention, StreamingLLM, H2O, ChunkKV, and KVzip.

Weakness 2: Experimental scale and statistical validity

We agree that ideally the tests would have larger sample sizes to strengthen precision. We do, however, report confidence intervals for the n=50 results in Appendix E.3-E4, which show, at the selected FanoutQA operating points, Qwen 3 (+14.7 95% CI [1.3, 28.0]) and Ministral 3 (+20.7 [8.7, 32.7]) and an average gain across model families of (+13.5 [6.5, 20.3]; Table 8).

Moreover, across the full FanoutQA results, CacheBack achieves a higher mean strict accuracy and lower median TTEOA than its equivalent text worker at 4x, 8x, and 16x compression in all four model families (Tables 6- 7). The observed improvements are therefore shown across multiple compression ratios and model architectures.

As you mentioned, the Longbench V2 Easy choice is explained in §4.3 and Appendix F.3. However, easy remains challenging for the models of this size. This is shown by the fact that same-size text workers achieve 39% in both model families compared to chance at 25%.

Our baselines isolate the effect of receiver conditioning in communication. Text workers are given the same query; therefore, it is only in how the message is constructed (§4) and, where feasible, full context isolates selection (§4.2).

We agree that broader workflows would help strengthen application to agentic workflows. Our current benchmarks tests evidence aggregation across workers (FanOutQA) and preservation through successive handoffs (LongBench) (§4). We will discuss these evaluated workflows from the potential applications discussed in §5. If you would like we could run something closer to code based reasoning such as LOC Bench.

AuthorsOpenReview ↗

Reviewer 3 (2/2)

Weakness 3: Practical deployment constraints

Network costs

Network transfer costs in KV-based communication are one of the motivations for incorporating CacheBack in these multi-agent systems. CacheBack simultaneously reduces bytes over the wire compared to transmitting full KV caches (§1, §2) by reducing the number of positions sent and by sending one continuous input row per position instead of its KV position. This is discussed heavily in §3.1.2, where we show that for a Qwen 3 8B, transmitting a continuous input row is 18x smaller per position than a KV position. The receiver then reconstructs the activations through a fresh prefill.

Across 3.1.2 and Appendix C.3, we address communications over lower bandwidth networks. Across our Qwen experiments, the average size of a message is 25MB, which is negligible to decoding latency on high bandwidth networks (NVLink)with peak transfer times of 55.4us. Figure 8, shows that this would fall to 0.755us if replaced with tokenIDs where possible. We also see that with lower network speeds 300Mbits/s peak transfer time would fall from 664ms to 9.06ms under token IDs. Both transfer times are small when compared with the time spent generating text reports.

In §5 we also briefly discuss the tradeoff of CacheBack latency and how the advantage narrows when network transfer or receiver prefill costs more than the sender decoding avoided.

Heterogenous models

CacheBack uses LatentMAS for latent reasoning, which we keep fixed as stated in §2. LatentMAS was designed for the homogeneous setting stated in §1. This enables representation compatibility without a learned cross-model adapter. As §5 states, this allows us to isolate whether receiver-conditioned message selection can prevent context re-expansion without degrading communication quality and while reducing total serving costs.

However, we ran another ablation where we did not pass latent reasoning but used CacheBack to compress context (via indexing) between Nemotron and Qwen.

For 8x compression on FanoutQA (n=50)

Sender → ReceiverTextCacheBackΔ vs text [95% CI]
Qwen3-8B → Nemotron-12B0.38 (88 s)0.49 (29 s)+0.11 [0.00, +0.22]
Nemotron-12B → Qwen3-8B0.42 (65 s)0.55 (49 s)+0.13 [+0.02, +0.25]

Latency reflects prefill and decoding only, using the paper’s precomputed results for a fair per-message comparison. It does not include the total system time reported in the paper.

Dynamic Receiver queries.

We draw the reviewer’s attention to §5, where we explain why CacheBack can support different receiver queries at inference time. To test this on the 50 FanOutQA questions, we prefilled all of the required documents once for Qwen3 8B. Then ran 11 queries (the original plus 10 generated from the model) at CacheBack 8x compression.

The CacheBack selection algorithm is only 0.7s, therefore after we first prefill a sender’s context we can repeatedly re query the same cache cheaply. In our setup prefilling all 150k tokens takes 7.4s for the combined three senders. The optional latent steps add 5.4s and currently run outside the serving engine at batch size on. Forking vLLM and allowing cache mutation would drastically reduce this.

In this setup, Qwen 3 8B generated 10 independent queries for each question. The end-to-end accuracy showed no significant difference between the original (52.7) and the average of the 10 independent queries (51.5). The range across the 10 queries was 46.7 to 56.7.

Reviewer_v1ZFOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper presents CacheBack, which utilizes the attention score of the receiver's questions and additional correction to select the important context among agents. It uses the Latent representation to send to receiver to reduce the communication and restoration cost. The method is proved to be more efficient than the pure context communication in both accuracy and latency performance.

Critical strengths

The methods presented in this paper are well motivated and the workload and scenario of the multi-agent system seem relatively realistic and practical for me. The method provides a straightforward but efficient way to solve this critical problem. The evaluation is performed on FanOutQA and LongBench with both serial and concurrent workflow, and present it with systematic analysis.

Critical weaknesses
  1. The core method of this paper is about dynamic selection utilized the attention scores, which is similar to the idea of sparse attention, the experimental results should include accuracy or efficiency comparison recent dynamic sparse attention works with the same latent representation.

  2. The contribution of the latent presentation is not clearly presented, as the gains may mainly come from selecting relevant source text and avoiding report generation

Questions for authors
  1. How much do latent states contribute beyond selecting relevant source text? Could you compare CacheBack with a baseline that sends the same selected source positions without latent states, reporting accuracy and end-to-end latency, to evaluate the additional benefit of latent communication?

  2. Comparisons with existing KV-selection methods mainly report evidence recall. Could you compare their end-to-end accuracy and latency under the same message budget to clarify the benefit of CacheBack’s scoring mechanism?

  3. Each benchmark uses only 50 tasks, and some accuracy gains have confidence intervals that include zero. Could you evaluate on more tasks, including harder LongBench v2 questions, to verify whether the gains are consistent? Besides the agentic workflow evaluation would be better if it could be expanded to other workflow such as realistic MAS trace.

2 comments
AuthorsOpenReview ↗

Reviewer 2 (1/2)

We thank the reviewer for a thoughtful review, and we appreciate the time and effort it takes to review papers. Below, we address the main concerns raised and will make the changes in the final version of the paper.  Our main comment is that Cacheback is a simple, drop-in, training-free method that 1) addresses context-limit challenges in communication, and 2) outperforms existing methods across a variety of open attention-based model architectures.

Weakness 1: Comparison with sparse attention

We agree that receiver conditioning and CacheBack is related to this line of literature, such that it applies a function to a subset of information  via query aware attention building on SnapKV (§3.1.1).

Sparse attention methods could be an alternative method to CacheBack via evicting the context window before the message is sent. Dynamic sparse attention methods such as Quest (Tang et al., 2024, §2.2)* and DeepSeek sparse attention (DeepSeek et al., 2025)** keep every token and select a subset at each decoding step. This reduces decoding costs however, does not reduce the total number of positions sent to the receiving agent. Only sparse attention methods such as Nvidia’s RocketKV (Behnam et al., 2025, §§3.3)*** use permanent cache eviction and could thus replace CacheBack as a method for preventing context re-expansion. However, for the eviction stage of RocketKV, they adopt SnapKV. Our naive selector, mean query attention is our communication adaption of SnapKV’s scoring (§3.1.1; App. C.1) and CacheBack adds its query token correction to that method.

In §3.2 and Appendix D.1 we show CacheBack recall outperforms mean query attention. We also ran an end-to-end ablation on Qwen 8B FanoutQA (n=50) strict-accuracy setting comparing mean query attention to CacheBack. CacheBack outperforms mean query attention from 4x compression onwards, including non-zero confidence intervals (95%) from all tested ratios at 16x compression onwards. At 32x compression CacheBack increases strict accuracy from 26.7% to 45.3% (+18.7 points; paired 95% CI [10.0, 28.0]) compared to mean query attention.

Therefore, sparse attention methods could be used to speed up decoding and work complementary to CacheBack, but is not a baseline for comparing communication quality.

We will cite Quest, Deepseek Sparse Attention and RocketKV in the final version.

** https://arxiv.org/abs/2512.02556**

*** https://proceedings.mlr.press/v267/behnam25a.html***

Weakness 2 / Question 1: Core Contribution and Contributions of Latent State

This paper studies how receiver conditioned communication can prevent context re-expansion (where sending the sender’s KV cache will overwhelm the receiver’s context) while preserving the accuracy and latency improvements of latent communication (§5). We use LatentMAS’s latent steps and hold it fixed in the main experiments (§2). The experiments therefore isolate whether communicating every sender position is necessary for downstream accuracy.

Generating text messages prevents context re-expansion but constructing them requires autoregressive decoding. CacheBack instead uses the receiver’s query to score and select positions from the sender. This sends a message with a greater number of positions however, our approach moves the communication bottleneck from autoregressive message generation to network and prefill costs. This is the intended benefit of CacheBack. Appendix A formalises this separation of what information we send and how it is represented.

We ran your proposed latent steps ablations on Qwen 3 8B on FanoutQA. The accuracy difference table represents the 40 latent steps (L40) minus 0 latent steps (L0).

BenchmarkDifference (L40 − L0), pp [95% CI]
FanOutQA−0.2 [−2.8, +2.4]
LongBench v2 Easy+0.2 [−4.2, +4.7]

We find there are no detectable benefits in our benchmarks for LatentMAS’s latent steps. Though, LatentMAS itself does show across its six general-task benchmarks Qwen3 8B averages approximately a +0.3 accuracy gain in hierarchical settings and +1.1 in sequential settings in addition to 59%-88% fewer output tokens  (Zou et al., 2026, Tables 1 and 3). Therefore, the supported benefits of Latent steps would sit inside our sample confidence interval. We therefore credit the CacheBack’s selection algorithm as the primary source of gains in our paper.

AuthorsOpenReview ↗

Reviewer 2 (2/2)

Question: downstream accuracy and latency of the selection mechanism

We have run an ablation to compare CacheBack vs mean query attention on Qwen 8B on the FanoutQA benchmark. We report TTEOA % change, however selection remains insignificant for TTEOA latency (under 1% of median TTEOA, about 0.7s per task) so this is model generation noise.

CompressionMean query attentionCacheBackDifference, pp [95% CI]CacheBack faster than Mean Query attention (%)
2×49.3%45.3%−4.0 [−11.3, +2.7]-0.6%
4×52.0%55.3%+3.3 [−6.0, +12.7]+9.4%
8×49.3%50.0%+0.7 [−8.0, +9.3]+7.5%
16×36.0%46.0%+10.0 [+0.7, +20.0]+8.2%
32×26.7%45.3%+18.7 [+10.0, +28.0]-1.6%
64×20.0%30.0%+10.0 [+1.3, +19.3]-2.3%
128×14.0%22.7%+8.7 [+2.0, +16.7]-7.0%

Latency: (mean query attention - CacheBack) / CacheBack

At test ratios from 16x compression onwards we see CacheBack outperforms mean query attention highlighting the significant impact of our correction method on downstream task accuracy. CacheBacks see significant increase in end-to-end accuracy under higher levels of compression compared to mean query attention. The ability to have higher compression rates enables CacheBack to have a greater number of agents fan-in to one agent.

Selection remains insignificant for TTEOA latency (under 1% of median TTEOA, about 0.7s per task). Section 4.1 and Figure 6 give the end-to-end latency breakdown including but not limited to message construction, transfer and prefill.

This will be added to the final version of the paper to supplement the initial selector comparison shown in §3.2 and Appendix D.1 where we compare CacheBack with mean query attention, StreamingLLM, H2O, ChunkKV and KVzip.

Question: evaluation scale and additional workloads

We agree that ideally the tests would have larger sample sizes to strengthen precision. We do, however, report confidence intervals for the n=50 results in Appendix E.3-E4, which show, at the selected FanoutQA operating points, Qwen 3 (+14.7 95% CI [1.3, 28.0]) and Ministral 3 (+20.7 [8.7, 32.7]) and an average gain across model families of (+13.5 [6.5, 20.3]; Table 8).

Moreover, across the full FanoutQA results, CacheBack achieves a higher mean strict accuracy and lower median TTEOA than its equivalent text worker at 4x, 8x, and 16x compression in all four model families (Tables 6- 7). The observed improvements are therefore shown across multiple compression ratios and model architectures.

LongBench V2 Hard: Qwen 32-B

MethodAccuracyFleet TTEOA p50vs Text [95% CI]
No context34.724 s—
Text notes34.7791 s—
CacheBack46.0407 s+11.3 [−3, +25]

Presentation

Relating the presentation of the paper, we would appreciate further guidance and specific examples on sections that were difficult to follow. This clarity will help us make these changes in the final version of the paper.

Reviewer_e44tOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes a more efficient mechanism in agent communication: the sender agents send KV cache to the receiver, but not all the KV caches, only the part relevant to the receiver's request. This is demonstrated to reduce communication latency and improve task accuracy.

Critical strengths
  1. The proposed method is simple, clear, and intuitive. Its assumption is that communicating in cache but selectively can get a better tradeoff in speed/latency and/or accuracy, compared to text summary and/or full cache communication. This sounds reasonable and the experiments demonstrated this.

  2. The problem tackled is a very practical one in multi-agent systems. How subagents communicate with the main agent is a central problem where lots of performance / efficiency can be gained.

Critical weaknesses
  1. The core mechanism for selecting the relevant KVs is SnapKV, which is a query-aware KV compression method from previous work. This is good integration work in practical context but does not bring new methods.

  2. The scale of the experiments is small. Each benchmark has 50 problems only, and the largest model is 14B. It is hard to judge the effectiveness in general.

  3. The baseline comparison with text messages is not controlled for the same budget for latents. This makes it an unfair comparison.

  4. No empirical comparison with several related methods KVComm/Q-KVComm/Cache-to-Cache.

Questions for authors

Please see weaknesses.

2 comments
AuthorsOpenReview ↗

Response to Reviewer 1 (1/2)

We thank the reviewer for a thoughtful review, and we appreciate the time and effort it takes to review papers. Below, we address the main concerns raised and will make the changes in the final version of the paper. Our main comment is that Cacheback is a simple, drop-in, training-free method that 1) addresses context-limit challenges in communication, and 2) outperforms existing methods across a variety of open attention-based model architectures.

Question / Weakness: “The core mechanism is SnapKV; this does not bring new methods.”

Receiver Conditioned Latent Communication (RCLC) addresses the context-window bottleneck created by combining the entire worker's state to the receiving agent, contradicting the context management benefit of delegation. Message construction is therefore centered around the receiver’s information needs and context window (§2, Eqs. 1–2; Appendix A).
This paper then introduces CacheBack, a communication compression method that builds on SnapKV by applying a correction term. Its correction term isolates a failure in mean query attention where sender tokens that receive strong attention from an individual query token are evicted due to low values from other query tokens. CacheBack also differs from text in a few practical ways. It copies spans from the sender’s context instead of writing new text, so most of the message can be read back and checked against the source. This also means it can’t make up evidence the way a text summary can.
To show the significance of this correction term, we ran an additional end-to-end ablation comparing mean query attention to cacheback on FanoutQA for Qwen 8B. Same-size text workers score 40.7% strict.

Compression RateMean query attention (SnapKV)CacheBackDifference, pp [95% CI]
2×49.3%45.3%−4.0 [−11.3, +2.7]
4×52.0%55.3%+3.3 [−6.0, +12.7]
8×49.3%50.0%+0.7 [−8.0, +9.3]
16×36.0%46.0%+10.0 [+0.7, +20.0]
32×26.7%45.3%+18.7 [+10.0, +28.0]
64×20.0%30.0%+10.0 [+1.3, +19.3]
128×14.0%22.7%+8.7 [+2.0, +16.7]

At test ratios from 16x compression onwards, we see CacheBack outperforms mean query attention with large effect size and statistical significance (see 95% CI). We will update the initial selector comparison shown in §3.2 and Appendix D.1, where we compare CacheBack with mean query attention, StreamingLLM, H2O, ChunkKV, and KVzip.

Question / Weakness: “The largest model is 14B.”

We have also added a Qwen 32B end-to-end ablation for FanoutQa to supplement the evidence-recall analysis in §3.2, Figure 4, and Appendix D.1. We use strict accuracy, which counts an answer only if every part is right. Loose accuracy gives partial credit and inflates scores, since the receiver can fill in some parts from parametric memory: even with no handoff, the 32B receiver gets 22.7% strict accuracy.

MethodStrict accuracyFleet TTEOA p50 (s)vs Text [95% CI]
No context23——
Text53735s—
CacheBack 4×60.7610s+7.3 [−2.7, +18.0]
CacheBack 8×57.3534s+4.0 [−5.3, +14.0]
CacheBack 32x56.0522+2.7 [−8.0, +13.3]

At 32B, CacheBack retains higher accuracy point estimates than text: 60.7% versus 53.3% strict accuracy at 4× compression.
However, we also note that 14B in cache communication literature 14B is the upper scale evaluated in prior work such as LatentMAS and C2C. C2C primarily focuses on a 0.6B receiver, KVComm: 3–8B models, and Q-KVComm: 1.1–1.5B models. Our 14B evaluation therefore matches the upper scale of the related methods.
In addition to varying model sizes, we also studied four model families, covering dense Transformers, sliding-window attention, and Mamba-attention hybrids.

AuthorsOpenReview ↗

Response to Reviewer 1 (2/2)

Question / Weakness: “Text and latent messages do not have the same budget.”

Text communication requires each additional message to be generated autoregressively, then prefilled at the receiver. A key benefit of CacheBack is the ability to send more evidence without generating a long message (§4.2; §5). By selecting positions that are already computed, we transfer generation costs of a message to prefill costs (which is not autoregressive), moving the bottleneck to a cheaper computation.

Matching message lengths would constrain the amount of evidence each method can send without equalizing the computational cost of producing them. We therefore fix the available hardware and measure accuracy and end-to-end latency including message construction, transfer, and prefill (§4.1; Figure 6). The larger message CacheBack can communicate for a given hardware setup is one of the main benefits we evaluate.

To give a sense of scale, a CacheBack message at 64× compression is about as long as a text message from the same model. So a 32× message is roughly twice the length of text, and a 4× message roughly 16 times the length. At 32×, CacheBack is more accurate than text on FanOutQA for Qwen 3 (45% vs 41% strict), Ministral 3 (59% vs 39%), and Nemotron (44% vs 39%), with Gemma 4 as the exception (45% vs 52%). It is also faster for all four families, by 1.6× to 8.0× in median task-completion time. CacheBack remains faster than text even with order of messages orders of magnitude larger by replacing the generation cost with a prefill one.

Question / Weakness: “Missing KVComm, Q-KVComm and Cache-to-Cache comparisons” As we noted in the introduction and §2, Eqs. 1–2, it is infeasible to run LatentMass, KVComm, and Q-KVComm in settings where the senders read large documents that fill their context windows. In particular, fanout settings where K senders each read N positions would fill KN positions on the receiver, likely exceeding its context window (or use so much context to degrade model performance). We were not able to use these methods for Qwen and Nemotron settings. Thus, receiver-conditioned selection such as Cacheback is arguably necessary.

For Cache-to-Cache, the method uses cache fusion where both the sender and receiver have a shared context and fuse a residual per position. Therefore, it is not applicable to work in settings where the context between agents differs, which is the setting we study.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Additional Experimental Results

We thank all the reviewers for their thoughtful reviews and questions raised about the work. In addition to providing direct responses to each reviewer. We are providing this additional comment containing all of the further experiment run.

LongBench v2 hard: Qwen3-32B

To test the effect of CacheBack on harder benchmarks and larger models we test Qwen 32B on LongBench V2 Hard with CacheBack compression set to 8x.

MethodAccuracyFleet TTEOA p50vs Text [95% CI]
No context34.724 s—
Text34.7791 s—
CacheBack46.0407 s+11.3 [−3, +25]

TTEOA: Time till end of answer

CacheBack at 8x compression scores 46% compared to 34.7% for text and with 1.9x lower median TTEOA. This observed result suggests as we increase model sizes from 8B to 32B and the task difficulty itself, the gains of CacheBack still remain.

FanoutQA: Qwen3-32B

To test larger models we ran Qwen 32B on our paper FanoutQA (n=50) setting:

MethodStrict accuracyFleet TTEOA p50 (s)vs Text [95% CI]
No context23——
Text53735s—
CacheBack 4×60.7610s+7.3 [−2.7, +18.0]
CacheBack 8×57.3534s+4.0 [−5.3, +14.0]
CacheBack 32x56.0522+2.7 [−8.0, +13.3]

Here we see the observed CacheBack performance remains both faster and more accurate than text-based communication with larger models.

Selector: mean query attention vs CacheBack

To show that the CacheBack method improves end-to-end downstream accuracy on FanoutQA N=50 compared to mean query attention, we run an ablation for all compression ratios.

CompressionMean query attentionCacheBackΔ (pp) [95% CI]TTEOA change
2×49.3%45.3%−4.0 [−11.3, +2.7]−0.6%
4×52.0%55.3%+3.3 [−6.0, +12.7]+9.4%
8×49.3%50.0%+0.7 [−8.0, +9.3]+7.5%
16×36.0%46.0%+10.0 [+0.7, +20.0]+8.2%
32×26.7%45.3%+18.7 [+10.0, +28.0]−1.6%
64×20.0%30.0%+10.0 [+1.3, +19.3]−2.3%
128×14.0%22.7%+8.7 [+2.0, +16.7]−7.0%

TTEOA change: (mean query attention − CacheBack) / CacheBack.

At tested ratios from 16x compression onward, CacheBack outperforms mean query attention with statistical significance (see 95% CI). We will update the initial selector comparison shown in §3.2 and Appendix D.1, where we compare CacheBack with mean query attention, StreamingLLM, H2O, ChunkKV, and KVzip.

Heterogeneous with no Latent Steps

To show how CacheBack can be used in the heterogeneous case, we remove latent steps and pass only the compressed subset to the receiving agent.

FanOutQA dev50 · 8× compression · no latent steps

Sender → ReceiverTextCacheBackΔ vs text [95% CI]
Qwen3-8B → Nemotron-12B0.38 (88 s)0.49 (29 s)+0.11 [0.00, +0.22]
Nemotron-12B → Qwen3-8B0.42 (65 s)0.55 (49 s)+0.13 [+0.02, +0.25]

Without latent steps, CacheBack achieved higher observed accuracy than text in both tested heterogeneous cases.

Latent Steps

To show the effects of CacheBack selection vs latent steps from Latent Mass on Qwen 3 8B on FanoutQA. We ran CacheBack without any latent steps (L0) and compared it to our paper's fixed 40 latent steps (L40).

BenchmarkDifference (L40 − L0), pp [95% CI]
FanOutQA−0.2 [−2.8, +2.4]
LongBench v2 Easy+0.2 [−4.2, +4.7]

We do not observe a detectable benefit in our benchmarks for adding 40 latent steps.. Though LatentMAS itself does show across its six general-task benchmarks that Qwen3 8B averages approximately a +0.3 accuracy gain in hierarchical settings and +1.1 in sequential settings, in addition to 59%-88% fewer output tokens (Zou et al., 2026, Tables 1 and 3). Therefore, the supported benefits of Latent steps fall within our sample confidence interval. These results suggest that CacheBack’s selection method contributes to the observed gains in our benchmarks.

Dynamic Queries

For each of the 50 FanOutQA questions, we prefilled all of the required documents once for Qwen3 8B. Then ran 11 queries (the original plus 10 generated from the model) at CacheBack 8x compression.

The CacheBack selection algorithm is only 0.7s. Therefore, after we first prefill a sender’s context, we can repeatedly re-query the same cache cheaply. In our setup, prefilling all 150k tokens takes 7.4s for the three senders combined. The optional latent steps add 5.4s and currently run outside the serving engine at batch size one. Forking vLLM and allowing cache mutation would reduce this.

In this setup, Qwen 3 8B generated 10 independent queries for each question. The end-to-end accuracy showed no significant difference between the original (52.7) and the average of the 10 independent queries (51.5). The range across the 10 queries was 46.7 to 56.7.

We will add all these results to the paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.