acceptodds
Under review as a conference paper at ICLR 2027

Can a Main Agent Reuse Its Sub-Agents' KV Caches? A Systematic Empirical Study

Abstract

Multi-agent systems incur substantial inference costs, yet agent harnesses typically operate without awareness of the underlying inference engine. We investigate a simple question: Can a main agent reuse the KV cache of its sub-agents? We build a cross-layer system that enables this reuse and supports an empirical study spanning content reconstruction, retrieval-augmented question answering, and multi-agent trajectories. Our study suggests that conclusion-token KV states retain information about both the conclusion and its preceding context. Reusing these states can preserve answer quality in document question answering and limited-fan-out agent workflows while avoiding redundant prefill computation. However, accuracy degrades as the number of contributing sub-agents increases, exposing limits to direct cache reuse. These findings identify both opportunities and challenges for cross-agent KV sharing, motivating a broader shift toward co-designing agent harnesses and LLM inference systems, with workload-aware coordination across their traditional boundary.

Primary area:
infrastructure, software libraries, hardware, systems, etc.
Keywords:
Agent, KV Cache, LLM Inference

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_9yyYOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies whether a main agent can directly reuse the KV cache that a subagent computed for its returned conclusion, framed as partial-segment position-independent caching (PIC). The method builds a vLLM-based serving system that grafts the subagent response KV into the main agent's prompt with RoPE rebasing and without recomputation, based on the observation that an interior span carries no attention sink. The paper evaluates reuse at three levels: a Retell test that reproduces the grafted notes, a QA diagnostic on the final chunk of a long context, and a deep-research workflow with multiple subagents per turn. The results show that single-span reuse stays within single-digit F1 of full prefill, while quality degrades as the number of reused spans grows.

Critical strengths
  1. The paper addresses a concrete and practically relevant question. Since subagent outputs are produced by the same model, their KV cache already exists, and the paper builds an end-to-end path to examine its reuse in a real agent loop. The exact-prefix boundary tracking, which keeps grafted approximate states from being treated as exact prefix hits in later rounds, is a careful design detail for cross-agent reuse.
Critical weaknesses
  1. (Originality / Quality) The central explanation that an interior span needs no recomputation because it has no attention sink is not directly validated, and its relation to prior work on the same setting could be clarified.

    a. Reusing a sender's generated-response KV has been studied in KVCOMM [1] and RelayCaching [2], both of which introduce correction because direct reuse affects the receiver's quality. It would be helpful to discuss why the conclusion here differs.

    b. The role of the attention sink is not isolated, for example by comparing the same span with and without its leading tokens, or against EPIC/CacheBlend-style repair at matched cost. KV Repair is disabled without reported numbers.

    c. The high single-span scores may also be related to the probe design, since Retell is close to verbatim copying and the QA questions concern the span's own content.

    d. In Agent-Trace, the gap reaches 7.98–13.14 pp at k=9k=9 for the Qwen models. Since each grafted span's KV is computed without access to the other spans or the main agent's context, this degradation appears close to the missing cross-chunk attention addressed by CacheBlend [3], rather than a still-uncharacterized obstacle. The conclusion that it is not caused by the sink would be more convincing once the sink explanation of the single-span case is established.

  2. (Quality) Some experiment settings may not fully reflect the intended counterfactual, and a few settings are difficult to reconcile with the described system.

    a. In KVCache-QA, the reuse condition keeps only the final chunk's KV while the baseline prefills all CC chunks. Since a main agent would prefill only the returned text, the gap may reflect both context removal and cache mismatch.

    b. In Agent-Trace, the number of subagents kk also changes the fraction of the prompt occupied by approximate states, which makes it difficult to separate an aggregation effect from the amount of reused KV.

    c. The appendix states that one delegation is executed per main turn, while the main results vary kk up to 9 subagents per turn.

    d. The appendix describes the backend as supporting dense Qwen3 with full attention, while the results include MoE (Qwen3-30B-A3B) and MLA (GLM-4.7-Flash) models, and Figure 4 notes missing or mismatched GLM records. Clarifying these points would help confirm the reported numbers.

[1] H. Ye et al. KVCOMM: Online Cross-context KV-cache Communication for Efficient LLM-based Multi-agent Systems. NeurIPS, 2025.

[2] Y. Geng et al. RelayCaching: Accelerating LLM Collaboration via Decoding KV Cache Reuse. arXiv:2603.13289, 2026.

[3] J. Yao et al. CacheBlend: Fast Large Language Model Serving for RAG with Cached Knowledge Fusion. EuroSys, 2025.

Questions for authors
  1. (ref Weakness 1) Could the authors test the attention-sink explanation more directly, for example by reusing the same span with and without its leading tokens, or by applying EPIC/CacheBlend repair at matched cost? How do the findings relate to KVCOMM and RelayCaching in the same setting?

  2. (ref Weakness 1) Would cross-span recomputation, as in CacheBlend, recover the multi-span gap? This would help clarify whether the aggregation obstacle is indeed a new phenomenon.

  3. (ref Weakness 2) What is the score of a KVCache-QA baseline that prefills only the final chunk with the question?

  4. (ref Weakness 2) Could the authors clarify how kk subagents per turn are executed given the one-delegation-per-turn description, which backend produced the MoE and MLA results, and whether per-query records for GLM are available?

  5. (ref Weakness 2) Does the multi-span gap remain when the total number of grafted tokens is fixed while kk varies?

  6. (Clarification) Since segmented prefill issues one internal request per gap, how much end-to-end prefill time is saved compared with exact-prefix reuse and suffix prefill?

  7. (Presentation, minor) It would help to describe how RoPE rebasing is applied under MLA, where only the decoupled RoPE key is rotated. The description of the null KV Repair stage is also repeated across Sections 3 and 4.

1 comment

Thank you for the response and the clarifications.

The concerns on the probe design and the amount of approximate KV are partially addressed, as the supporting evidence comes from single-span settings that do not test the multi-span confound. The clarifications on the k-delegation workflow, the RoPE treatment under MLA, the GLM records, and the repeated text are helpful.

Some of my main concerns still remain, since the response narrows the central claims without providing new evidence. If the attention sink is presented as a hypothesis, the conclusion that the aggregation gap is not caused by the sink may also need to be softened. Similarly, attributing the gap to missing cross-span attention, together with the observation that RelayCaching's Zero Reuse results show a similar pattern, suggests that the contrast between single-span and multi-span reuse may have been partly reported before. It would be helpful to clarify what new understanding the paper adds beyond this. In addition, the final-chunk-only baseline and the fixed-token control are not yet provided. Without them, it remains somewhat difficult to attribute the KVCache-QA gap and the Agent-Trace degradation. Since the subagent also receives task context from the main agent (Appendix A.1), the k=1 case may not fully separate attention to the receiving context from attention between spans. Lastly, as latency is not measured, the efficiency motivation of the paper could be better supported with wall-clock results.

I appreciate the authors' clarification, but I will keep my current assessment.

Reviewer_Xx8dOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies whether a main agent can reuse the KV states of a subagent's response when both agents share the same model but have different contexts. The authors implement cache grafting with position adjustment, without recomputing the matched response spans. They evaluate three models through response retelling, question answering using a cached suffix, and an iterative multi-agent workflow. The reported results suggest that individual reused spans retain substantial usable information, while configurations combining multiple subagent responses show larger quality losses. The main contribution is an empirical investigation of the feasibility and limitations of this form of cache reuse.

Critical strengths

The paper makes a useful distinction between accessing information from one reused span and combining several reused responses in an agent workflow. Successful reconstruction alone would not establish that the approach supports downstream task completion. Evaluating both is therefore sensible, and the reported deterioration in multi-response settings is an informative finding that deserves further investigation.

Critical weaknesses
  1. The practical benefit remains unquantified. The paper motivates reuse as a way to reduce inference cost, but Section 5 reports only answer quality. There are no measurements of latency, GPU time, throughput, or net savings after cache transfer and segmented execution, even in the favorable single-subagent setting. Meanwhile, quality losses can be substantial: with nine subagents per turn, Qwen3-30B-A3B drops from 67.62 to 54.48 F1 (Section 5.4). What is gained in return is unclear. I recognize that this is an empirical study and that feasibility findings can be valuable on their own. However, the claim in Section 6 that this is a practical way to reduce cost needs evidence beyond avoiding some prefill computation.

  2. The stronger conclusions need better-controlled and more complete evidence. In Section 5.3, questions are generated from the final chunk, but the reuse condition retains only that chunk's KV while the baseline receives all chunks. This provides evidence of information accessibility, but it does not isolate the effect of grafting versus recomputation under identical receiving inputs. In Agent-Trace, execution trajectories can also differ, as the authors acknowledge. These observations do not establish the claimed causal role of attention sinks in single-span success or exclude their contribution to multi-span degradation. Separately, the Figure 4 caption reports missing per-query records for GLM at k > 1 and a mismatch between the updated reuse summary and archived records at k = 1. These issues limit confidence in the conclusion across all three models.

Questions for authors
  1. Evidence for the cost claim: What net latency or GPU-time savings does the implementation achieve over the reported baseline, including cache movement and segmented execution? Results for the single-subagent setting, alongside F1 and the actual fraction of tokens reused, would be particularly helpful.

  2. Evidence for the interpretation: Do the quality findings hold when the returned text and receiving prompt are fixed, changing only whether the response spans are grafted or recomputed? Is there a controlled ablation supporting the attribution to attention sinks, beyond observing good single-span performance?

  3. Clarification of reported results: Can the Figure 4 summaries be reconciled with complete per-query records, particularly for GLM? Please clarify failure counts and whether a common failure-handling rule changes the comparison.

Reviewer_DhBkOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates whether a main agent can directly reuse the KV caches generated by its subagents instead of recomputing their textual outputs during prefill. The authors implement a KV grafting mechanism on top of vLLM that supports cross-agent cache transfer, positional rebasing, and segmented prefill without recomputing the reused spans. The proposed system is evaluated through three experiments: KVCache-Retell, which measures content reconstruction fidelity; KVCache-QA, which examines information accessibility from reused segments; and Agent-Trace, which evaluates downstream task quality in an iterative multi-agent research workflow. The results suggest that single-span KV reuse can preserve much of the original information with relatively small performance degradation compared to full prefill, whereas the performance gap becomes substantially larger when multiple subagent responses are combined. The paper highlights the potential of direct KV cache transfer between agents while identifying multi-span aggregation as a major unresolved challenge.

Critical strengths

1. A practically relevant problem supported by a concrete system implementation. The paper addresses an important inefficiency in multi-agent LLM systems: redundant prefill computation when a main agent consumes responses already generated by its subagents. The authors implement a vLLM-based system supporting cross-agent KV grafting, positional rebasing, and segmented prefill. Although cross-agent KV reuse has been explored in prior work, the implementation provides a concrete platform for evaluating the feasibility and limitations of directly reusing subagent-generated KV states without recomputation. This is valuable because it connects a practically motivated serving optimization with systematic measurements of downstream model behavior.

2. A structured empirical investigation of single-span fidelity and multi-span degradation. The evaluation separates content reconstruction, information access, and end-to-end agent execution, providing complementary perspectives on the behavior of reused KV states. The results show that individual cached spans can often be reused with limited quality degradation, while combining multiple spans leads to much larger performance gaps. For example, Agent-Trace reports F1 decreases of 7.98 and 13.14 percentage points relative to full prefill at nine subagents per turn for Qwen3-8B and Qwen3-30B-A3B, respectively. Although related failure modes have been observed in previous studies, characterizing this behavior in a hierarchical research-agent workflow provides useful empirical evidence about the limitations of cross-agent cache reuse.

Critical weaknesses

1. Limited novelty and insufficient differentiation from closely related work. The central problem formulation substantially overlaps with prior work, particularly RelayCaching [1], which already investigates directly reusing decoding-phase KV caches generated by upstream agents during downstream prefill. RelayCaching further characterizes prefix-dependent representation mismatches and proposes selective recomputation to preserve accuracy while improving inference efficiency. While the present work focuses on zero-recomputation reuse, its relationship to existing cross-agent KV reuse methods is not sufficiently established.Furthermore, Liang et al. [2], which the paper already cites, demonstrate that KV cache reuse can weaken cross-candidate interactions and disrupt downstream decision-making in multi-agent systems. Although the evaluation settings differ, the present work does not clearly establish whether its multi-span aggregation failure represents a distinct phenomenon or an extension of previously identified cross-segment interaction problems. The supplementary material provides useful implementation details and experimental artifacts, but does not resolve this central question of scientific novelty.

2. Insufficient causal analysis of the main empirical findings. A central observation is that single-span KV reuse closely approximates full prefill, whereas combining multiple reused spans leads to substantial degradation. However, the experiments do not adequately isolate the mechanisms responsible for this difference. In Agent-Trace, reuse and full-prefill conditions can follow different execution trajectories, making it difficult to determine whether the observed degradation arises directly from combining approximate KV states or indirectly from changes in downstream reasoning and delegation. Additionally, the paper attributes successful single-span reuse to the absence of an attention sink, but does not establish this explanation through a controlled ablation. Although the supplementary material includes KV repair implementations and diagnostic tests, these do not provide a systematic causal analysis of the reported aggregation failure. Consequently, the paper identifies an interesting limitation of cross-agent KV reuse but provides insufficient evidence explaining its underlying mechanism, reducing the significance of its principal empirical contribution.

References

[1] Yingsheng Geng, Yuchong Gao, Weihong Wu, Guyue Liu, and Jiang Liu. RelayCaching: Accelerating LLM Collaboration via Decoding KV Cache Reuse. Proceedings of the 43rd International Conference on Machine Learning (ICML), PMLR 306:34524–34543, 2026. https://proceedings.mlr.press/v306/geng26a.html

[2] Sichu Liang, Zhenglin Wang, Jiajia Chu, Pengfei Xia, Hui Zang, and Deyu Zhou. When KV Cache Reuse Fails in Multi-Agent Systems: Cross-Candidate Interaction is Crucial for LLM Judges. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (ACL), pages 7205–7230, 2026. https://aclanthology.org/2026.acl-long.327/

Questions for authors
  1. How does the proposed zero-recomputation KV grafting approach differ fundamentally from RelayCaching (ICML 2026), which directly reuses decoding-phase KV caches across agents and selectively repairs prefix-dependent deviations? In particular, which findings in this paper are not already captured by RelayCaching's analysis of cross-agent KV reuse?

  2. How does the multi-span aggregation failure observed in this paper differ from the cross-candidate attention degradation characterized in When KV Cache Reuse Fails in Multi-Agent Systems (ACL 2026)? Does the present study identify a distinct mechanism, or does it extend the same underlying phenomenon to a different multi-agent workflow?

  3. Could the authors evaluate multi-span KV reuse with fixed subagent responses and identical main-agent prompts, varying only the number of grafted spans?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.