acceptodds
Under review as a conference paper at ICLR 2027

KVTrans: Towards Seamless Cross-Context KV Cache Reuse for Efficient Agentic Communication

Abstract

LLM-based multi-agent systems repeatedly re-encode received messages, even though their key–value (KV) states have already been computed during sender generation. Reusing these states can reduce redundant computation, but differences in agent instructions, interaction histories, and token positions make direct cross-context reuse unreliable. We introduce **KVTRANS**, a **training-free** framework for **adapting sender-generated KV caches to receiver contexts** among agents sharing the same model. KVTRANS combines rotary position alignment, receiver-referenced key and value rescaling, and dynamic modulation of receiver-instruction keys during decoding. Across four models on MMLU debate, KVTRANS improves macro-averaged accuracy from 59.61% under direct reuse to 65.44%, compared with 64.88% for text communication, while reducing receiver-side prefill tokens by **59.14%**. Timing measurements on four matched role pairs show a **57.04%** reduction in prefill time, including pre-decoding cache-adaptation overhead. In WebSailor experiments on BrowseComp-EN and BrowseComp-ZH, KVTRANS partially recovers the quality degradation caused by direct reuse while processing only **53.9%** and **62.3%** of the text baseline’s receiver-side prefill tokens, respectively. These results demonstrate that lightweight receiver-side adaptation can improve the quality–efficiency trade-off of cross-context KV reuse without retraining model weights.

Primary area:
foundation or frontier models, including LLMs
Keywords:
KV Cache, Multi-Agent System, Large Language Models, Efficiency

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.20 over 5 reviews, above 49% of the others.
Reviewer_tXAXOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies reuse of key-value caches generated by a sender agent when the receiver agent shares the same model weights and tokenizer but has a different role instruction, interaction history, and token position. Direct reuse avoids re-encoding received messages but is reported to degrade accuracy in a way that varies across models and role pairs. The paper attributes this to three mismatches, namely position shift, magnitude misalignment, and history information conflict, and proposes a training-free framework with three components. RoPE Alignment re-rotates transferred keys to receiver positions, L2-Rescaling scales keys and values toward receiver-local norm statistics, and Dynamic Attention Amplification scales the receiver's instruction keys in dynamically selected heads during decoding. The method is evaluated in a multi-agent debate setting across several models and role pairs, and in a long-horizon deep research agent on two browsing benchmarks. Compared with direct reuse, it recovers accuracy to a level reported as comparable to text communication in the debate setting, and it partially recovers the degradation on the browsing benchmarks, while reducing receiver-side prefill tokens and measured prefill time.

Critical strengths

S1. The paper addresses a practical efficiency problem with a training-free method and measures cost carefully. Repeated re-encoding of received messages is a real cost in multi-agent pipelines, and adapting existing caches without modifying weights is easy to adopt in principle. The cost accounting separates processed prefill tokens from cache hits, includes the adaptation overhead in prefill time, and reports end-to-end timing in the appendix, including a pair where the method is slower. This makes the efficiency claims more interpretable, which matters because the contribution is ultimately a quality-efficiency trade-off.

S2. The evaluation extends beyond controlled debate to a long-horizon agentic setting and includes an honest analysis of its limits. The appendix uses teacher-forced probes and partial recomputation to show that the discrepancy is present before trajectories diverge and is not confined to the end of the message. This clarifies where cache adaptation can and cannot help, and supports a more realistic reading of the results than the debate setting alone.

Critical weaknesses

W1. The positioning against closely related training-free methods is not supported empirically, and the novelty of individual components is unclear. The central claim concerns a better quality-efficiency trade-off for cross-context KV reuse, yet the main comparison includes only text communication and unadapted direct reuse. KVCOMM (Ye et al., 2025) addresses the same problem of reusing KV caches under diverging agent prefixes without training, but it is not evaluated as a baseline. RelayCaching (Geng et al., 2026) also reuses decoding-phase KV caches of a previous agent in the next agent's prefill, relying on selective recomputation, and is not cited. Its observation that prefix-induced deviations are localized appears to differ from the finding in Appendix J that tail recomputation gives only partial recovery, which makes a direct comparison informative. As a result, the limitations of prior work are shown only against a naive baseline, and it is not clear whether the proposed adaptation improves over existing alternatives. In addition, the components resemble existing ideas that are not discussed. L2-Rescaling is related to the scaling and temperature alignment used for reused KV states in APE (Yang et al., 2025), and Dynamic Attention Amplification is related to instruction-focused attention steering in selected heads such as PASTA (Zhang et al., 2024) and SpotLight (Venkateswaran & Contractor, 2026). The three mismatches are motivated mainly by a schematic (Figure 3), and Appendix C states that the diagnostic identities characterize possible sources rather than measured contributions. Together, these points make it hard to judge what is new and what is responsible for the reported gains, which is central to the significance of the contribution.

W2. The evidence for quality comparable to text communication is weaker than the headline numbers suggest. The macro-averaged accuracy of 65.44% versus 64.88% for text appears to be driven largely by one model, where the proposed method improves over text by 5.33 points (72.54% versus 67.21%). On two of the other three models the method remains below text (57.46% versus 58.39%, and 51.68% versus 54.36%), and on WebSailor it stays below text on both BrowseComp-EN and BrowseComp-ZH (28.33% versus 34.67%, and 33.33% versus 40.00%). The evaluation scale also limits how reliably such differences can be interpreted. Each ordered role pair uses 50 questions, the component ablation and the head-selection comparison (Tables 3 and 4) rely on 4 to 12 role pairs, and the gain of dynamic selection in Table 4 comes mostly from two pairs. The hyperparameters are tuned on a development set whose size and overlap with the few-shot examples are unclear. Since the paper's conclusion rests on recovering text-level quality while saving prefill cost, these issues make it difficult to tell whether the reported recovery is robust or specific to certain models and role pairs.

Questions for authors

Q1. Direct reuse matches or exceeds text communication on Ministral-3-3B in many role pairs (Figure 2), whereas it degrades on the other models, and Appendix H does not analyze this model. I wonder what accounts for this difference. It would be helpful to see the rate of invalid final answers, which are counted as incorrect, for each method and model, and whether the longer messages implied by the text prefill counts for this model play a role. I also noticed that the Direct KV accuracy in Table 5 appears identical for Qwen2.5-3B and LLaMA3.2-3B, and it would be good to confirm that this is not a copy error.

Q2. The proposed method exceeds text communication in some settings where direct reuse does not degrade (e.g., the Ministral results and Table 4), which makes me wonder whether Dynamic Attention Amplification repairs cache mismatch or mainly increases attention to instructions in general. It would be helpful to see Dynamic Attention Amplification and L2-Rescaling applied to text communication where no foreign cache is inserted. It would also be helpful to report the fraction of heads selected per layer under the chosen threshold, the sensitivity to the gain and threshold, and the data split used to tune them. Since the amplification scales signed logits (Appendix D), I am curious how often it lowers attention for the selected heads because the query-key product is negative.

Q3. The three mismatches are motivated mainly by the schematic in Figure 3. I wonder whether they can be connected to measured quantities, for example by relating the degradation of Direct KV to the sender-receiver prefix length difference, and by measuring the norm gap relative to receiver-native re-encoded message states rather than receiver prompt statistics. The benefit of L2-Rescaling differs substantially across models, so I am curious whether the magnitude gap itself differs across models and whether architectural details such as query-key normalization or projection biases are relevant.

Q4. The method assumes that the sender and receiver share identical weights and tokenizer. I wonder how common this setting is in practical multi-agent deployments, where agents are often backed by different model sizes or fine-tuned variants of a shared base model. It would be helpful to see whether the proposed components remain effective for fine-tuned variants of the same base model, and to discuss what would be required to extend the approach beyond a single model. This is offered as a suggestion rather than a requirement.

Q5. The introduction describes the message-prefill computation as reduced from quadratic to linear in the number of agents, but the RoPE Alignment and L2-Rescaling steps appear to be applied per message and receiver pair. I am curious how the adaptation overhead scales with the number of agents in the fully connected setting. In addition, the prefill time reduction in Table 2 shrinks considerably in the end-to-end timing of Appendix E.2, with one pair becoming slower, and it would be helpful to know how the timed pairs were selected and to see end-to-end timing for the WebSailor experiments.

Reviewer_s6wvOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies cross-context KV-cache reuse among agents sharing the same language model. The proposed KVTrans framework combines RoPE alignment, receiver-referenced L2 rescaling, and dynamic modulation of receiver-instruction keys during decoding. Experiments cover multi-agent debate on MMLU and collaborative web research with WebSailor. The results show improved task performance over direct cache reuse while reducing receiver-side prefill computation.

Critical strengths
  1. The paper addresses a practical problem. Avoiding repeated encoding of exchanged messages could reduce computation in multi-agent systems, while differences between sender and receiver contexts make reliable cache reuse challenging.

  2. The proposed framework is lightweight and requires no model training. Its components are straightforward to implement, and KVTrans improves over direct reuse across all four models in the main MMLU evaluation.

  3. The evaluation includes both controlled debate and a more complex tool-using workflow. The authors also report end-to-end timing and discuss the remaining degradation on BrowseComp, providing useful context for the method’s practical limitations.

Critical weaknesses
  1. The experiments do not sufficiently isolate cache-specific recovery from general attention steering. Dynamic Attention Amplification modifies how the receiver attends to its system instructions, which could improve performance even under ordinary text communication. A Text + Dynamic Attention Amplification control is therefore important for interpreting the gains. Tab. 3 provides sequential ablations, but these do not establish the independent contribution of each component or whether all three are necessary. This matters particularly because the attention intervention accounts for a substantial part of the reported recovery.

  2. The method matches incoming message norms to statistics from the receiver’s existing prompt and history. However, these reference states need not have the same distribution as the message states obtained through receiver-side re-encoding. Differences in norm could reflect content type rather than contextual incompatibility. It would strengthen the method’s motivation to show that rescaling actually reduces discrepancies from receiver-native message representations, and to examine sensitivity to the choice of reference positions.

  3. On page 21, Fig. 5 is described as showing receiver-instruction preference scores across layers and heads, but the plotted panels are labeled as K/V relative L2-difference histograms. Similarly, Fig. 7 on page 22 is described as comparing attention patterns under Text, Direct KV, and KVTrans, while the panels again show K/V relative L2-difference histograms without an identifiable comparison among these methods. This may be a figure-placement or caption error, but it currently prevents verification of the associated mechanism claims.

  4. The evidence for preserving task quality is mixed and lacks uncertainty estimates. The MMLU macro average is close to Text, but Tab. 5 shows that KVTrans remains below Text on Qwen2.5 and LLaMA, while the larger improvement on Ministral offsets these losses. On BrowseComp, the method improves substantially over Direct KV but remains below Text in both languages. Confidence intervals would help assess these differences, accounting for questions reused across role pairs and repeated rollouts of the same question. In addition, the BBH experiment reports token savings without task accuracy, limiting its support for quality generalization.

Questions for authors
  1. How does Text communication perform with the same Dynamic Attention Amplification intervention? Could the authors also provide component-removal ablations to clarify which operations are necessary?

  2. Which receiver positions are used for norm calibration in each experimental setting? Does rescaling reduce the distance to the corresponding receiver-native message KV states, beyond matching reference norms?

  3. Could the authors clarify the apparent discrepancies between Fig. 5 and 7 and their captions, and provide the intended head-selection and attention-distribution analyses?

Reviewer_YZTdOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a practically useful and well-evaluated KVCache reuse framework, based on a novel observation on performance heterogeneity under direct prefill cache reuse. The author identifies three key aspects, including differences in agent instruction, interaction history difference, and mismatched token positions, that potentially undermines the prefill cache reuse. The authors then address each problem with rotary position alignment, receiver referenced key and value rescalling and dynamic modulation of reciver instruction keys during decoding. Empirical evidence suggests the effectiveness of these approaches in multi-agent system under the shared base model.

Critical strengths

The paper addresses an important efficiency bottleneck in multi-agent communication. The proposed framework is training-free and relatively lightweight, with experiments demonstrating a favorable quality-efficiency trade-off across different models and agentic settings.

The paper is well written, with a clear and logical progression from problem motivation to method design and empirical evaluation. The experimental results and supporting analyses are also well organized.

Critical weaknesses

While the observation of heterogeneity is interesting, it primarily serves as a motivation for the proposed heuristic solutions. The paper does not sufficiently establish the connection between this observation and the three identified sources of mismatch. For instance, it remains unclear why different sender-receiver pairs exhibit substantially different sensitivities to direct KV reuse, or to what extend these differences can be explained by the proposed mismatch mechanisms. Although the component ablations demonstrate empirical effectiveness, they do not fully explain the underlying causes of the observed heterogeneity or how each component specifically addresses them. This gap between the motivating observation and the proposed design remains my primary concern.

The method is calibration based, so the generality is not guaranteed, despite the empirical evidence supports current claim.

Questions for authors

I would like to know if there is any quantitative or causal evidence connecting the observed performance heterogeneity to the three proposed mismatch mechanisms? For example, can the authors explain why certain models benefit substantially more from L2-Rescaling than others, and whether these differences correlate with measurable properties of their cross-context KV representations?

Could the authors provide a more detailed breakdown of the computational overhead introduced by L2-Rescaling calibration?

Reviewer_SFjcOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

KVTrans reuses the KV a sender produced while generating its message, so a receiver running the same model can skip re-prefilling it. The method is training-free: RoPE re-rotation, per-head L2 norm matching, and a decode-time scaling of the receiver's system-instruction keys in selected heads. On MMLU debate with four 3–4B models, macro accuracy is 65.44 vs 64.88 for Text and 59.61 for direct reuse, with 59% fewer receiver prefill tokens. On WebSailor/BrowseComp it partially recovers the drop from direct reuse.

Critical strengths

The problem is well motivated, and Fig. 2 shows how much direct reuse varies across pairs. The probe in App. J is useful negative evidence: recomputing a 128-token tail still gives only 81.9% top-1 agreement. The paper is also honest that end-to-end time drops only 20.9%.

Critical weaknesses
  1. There is no comparison with methods that address the same problem. Sec 7 itself describes KVCOMM as handling training-free cross-context reuse with positional correction. Yet Table 1 only compares against Text and Direct KV, and Direct KV doesn't even re-rotate RoPE. There is no CacheBlend/EPIC-style selective recompute and no tail-recompute baseline, so the improvement is shown only against a weak baseline.

  2. It is unclear where the gain comes from, and the parity claim is weak. DAA only changes the receiver's own system keys, which is close to PASTA-style attention steering. Ours often beats Text (Ministral 72.54 vs 67.21; HotPotQA F1 42.93 vs 40.76). Without a Text+DAA control, we can't tell how much of the gain is cache repair. Macro parity relies on Ministral, where Direct KV already beats Text. On Qwen2.5 (57.46 vs 58.39) and LLaMA (51.68 vs 54.36), KVTrans stays below Text, and on BrowseComp it reaches 82–83% of Text. BBH reports token counts but no accuracy.

Questions for authors
  1. How do KVCOMM, CacheBlend-style recompute and Direct KV + RoPE perform at the same prefill budget?
  2. What is the accuracy of Text+DAA?
  3. What is the accuracy on BBH?
  4. How were the ablation pairs selected, and what development data was used to tune γ and τ?
  5. How many tokens does each method generate in Table 6, and what is DAA's per-token decode overhead?
Reviewer_1SK4OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper studies cross-context KV reuse among agents that share one model but differ in prompts, histories and positions. Training-free KVTrans re-rotates transferred keys to receiver RoPE positions and rescales message keys and values per layer and KV head to receiver-local mean norms. During decoding, Dynamic Attention Amplification (DAA) scales system-prompt keys by γ in the heads whose queries lean towards them. On a two-expert-plus-judge MMLU debate (56 role pairs, 50 questions each, four 3–4B models), macro accuracy rises from 59.61% (Direct KV) to 65.44% (Text: 64.88%) with 59.14% fewer receiver prefill tokens, and prefill time on four pairs falls by 57.04%. In WebSailor, BrowseComp-EN/ZH Pass@1 is 28.33%/33.33% (Direct KV 10.00%/26.67%, Text 34.67%/40.00%) at 53.9%/62.3% of Text prefill tokens.

Critical strengths
  1. The problem is relevant and the characterization is useful. Figure 2 and Table 5 show that direct reuse is strongly model- and pair-dependent: Qwen2.5 keeps 77% of Text accuracy, while Ministral beats Text. Appendix J's teacher-forced probe isolates cache mismatch from trajectory divergence: foreign summary KV matches Text's top-1 token 76.9% of the time, against 98.1% for receiver-local reconstruction, and recomputing the last 128 tokens reaches only 81.9%. That is a valuable negative result for KV-passing agents.

  2. The method is cheap, training-free and fully specified (Eqs. 3–6, Appendix D). It beats Direct KV in every setting tested: all four MMLU models, both BrowseComp languages, and HotPotQA/GAM (F1 15.57 to 42.93).

Critical weaknesses
  1. The baselines are only Text and Direct KV. The paper describes KVCOMM as training-free cross-context reuse with positional correction (Section 7), but does not run it. Nor is there CacheBlend-style selective recompute or partial recompute at a matched prefill budget (Table 12 is a 160-position probe). There is no Text+DAA control, and Ours beats Text by 5.3 points on Ministral and by 5 points in Table 4, so part of the gain may be generic instruction steering. RoPE alignment adds only 1.8/0.34 points (Table 3), so the novelty rests on norm matching and logit scaling of system keys; as far as I remember, the latter resembles post-hoc attention steering such as PASTA (Zhang et al., ICLR 2024), which the paper does not discuss.

  2. The quality results carry no uncertainty estimates. MMLU has 400 unique questions per model and one greedy run, and the 0.56-point macro lead over Text is about 16 of 2,800 judge decisions per model. Ours trails Text on Qwen2.5 (57.46 vs 58.39) and LLaMA (51.68 vs 54.36). Parity comes from Ministral, where even Direct KV beats Text by 3.5 points, which suggests a noisy protocol. Table 3 uses undefined "common pairs" on which LLaMA beats Text, unlike over all 56 pairs. Table 4 rests on 200 questions from an unstated model.

  3. The efficiency evidence is thin. The 57.04% figure comes from four pairs, an unstated model, and a Hugging Face backend on one A100. In Table 6, prefill explains only 2.64 s of the 20.58 s end-to-end saving. The remaining ~18 s presumably reflects different generation lengths, a behavioural confound; Economics–Philosophy is even slower end to end (102.33%). DAA's per-step cost and fused-kernel compatibility are not reported. WebSailor costs are logical tokens only: Text counts are reconstructed while KV counts are recorded, and KV export is excluded (Appendix K).

  4. Some claims go beyond the evidence. On BrowseComp, KVTrans closes 74% (EN) and 50% (ZH) of the gap from Direct KV to Text. The abstract says "partially" and the conclusion says "substantially". The "BBH results" are only prefill-token counts, which are identical for Direct KV and Ours by construction. Figures 5–7 all show the same "K/V Matrix Relative L2 Diff" histograms at different layers. None shows head-selection scores or attention under Text, Direct KV and Ours, so the Appendix H claims built on them are unsupported.

Questions for authors
  1. Can you add KVCOMM, CacheBlend-style selective recompute at KVTrans's prefill budget, and Text+DAA on MMLU and BrowseComp? If KVTrans still wins, I would raise my rating.

  2. Please give paired bootstrap CIs for Ours vs Text per model and for Tables 3–4, the pair-selection rule and model for Tables 2–4 and 6, and why Table 9's science columns are identical for Direct KV and Ours. How many BrowseComp-ZH rollouts were scored (all three ZH values are multiples of 1/15), and was the manual correction blind to method?

  3. Which development set did you use for the 66-point (γ, τ) grid? As far as I know, MMLU's dev split has 5 items per subject, which is your few-shot pool. Which γ, τ and reference set did you use for WebSailor and GAM?

  4. Does Text re-encode each expert's own Stage-1 history, and does WebSailor's Text use SGLang prefix caching? Please report generated tokens per method in Table 6 and measured WebSailor wall-clock including DAA and KV transfer.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.