acceptodds
Under review as a conference paper at ICLR 2027

Is Cross-Context KV Cache Reuse Always Reliable and Harmless in Agent Execution?

Abstract

Cross-context KV cache reuse can reduce repeated prefill for reusable agent context, such as skill documents, tool schemas, and policy instructions. However, preserving task success does not necessarily preserve agent execution: KV reused from a different context can substantially alter an agent’s responses and actions, while incurring additional execution costs or unwanted workspace changes even when the task is completed successfully. This paper comprehensively explores the cross-context KV cache reuse mechanism. We trace KV cache reuse from cache differences within model to agent behavior and task outcomes. Further, we characterize how reuse induces behavioral divergence, explain how divergent executions can still preserve task success, and establish a deployment-level criterion for determining when such reuse remains permissible under execution requirements. Overall, this paper identifies and theorizes the impact of cross-context KV cache reuse on agent execution, guiding the reliable deployment of cross-context KV cache reuse in agent systems.

Primary area:
foundation or frontier models, including LLMs
Keywords:
KV cache reuse, LLM agents, behavioral divergence, mechanism analysis, inference efficiency

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.20 over 5 reviews, above 49% of the others.
Reviewer_yaEMOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies whether or not an agent can safely reuse a model's KV cache built from a different context, going in depth characterizing what happens and why. It finds that reuse can change agent responses and actions, affecting execution cost and workspace state. The authors argue that using task success alone to judge cache reuse safety may not be enough, and propose a criterion for deciding when cross-context cache reuse is permissible.

Critical strengths
  • Useful analysis beyond task success: The paper shows that under cache reuse, an agent can still complete the required task while taking different actions, changing execution cost and workspace state, highlighting that using success only as a judge may hide problems with cache reuse.

  • Careful behavioral analysis: By exhaustive comparisons between current and stale caches on the same tasks, the paper examines how cache reuse can lead to different responses and actions. The authors comprehensively describe and characterize this behavior.

Critical weaknesses
  • Proposed criterion is post-hoc: It assesses whether a cache-reused execution met the requirements, but does not predict in advance which reuse cases are safe. Its value as a deployment safeguard is therefore limited.

  • The practical impact of cache reuse remains unclear: The experiements test full-span cache substitution rather than a more realistic selective-reuse policy. They show that reuse can change behavior in some unintended ways, but not how often this can happen in practical deployments or whether reuse still saves time overall.

Questions for authors
  1. Can the proposed criterion identify safe cache reuse before execution, or is it only an after-the-fact check?

  2. Do the behavioral effects persist under selective reuse, and does that approach still reduce end-to-end latency?

Reviewer_XeK5OpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies replacing a repeated span's current-context KV cache with KV computed for identical span tokens under a different preceding context. It connects realized logit changes and token margins to the first divergent token, then studies how retained KV, changed history, and environment feedback affect subsequent agent actions. Controlled tasks show frequent behavioral divergence despite preserved task success; public benchmarks show setting-dependent introduced failures and gained successes. The paper also defines an execution-level audit that combines success with process-cost, workspace, and action constraints, emphasizing that preserved aggregate success does not imply harmless reuse.

Critical strengths
  1. The paired intervention design is careful and informative. Current and stale conditions load caches through the same pathway with matched visible inputs, while recomputation provides a numerical-control baseline. Token exchanges, partial cache interventions, and direction controls go beyond observing a correlation between cache changes and output changes. Appendix B.2 also checks request replay and treats tasks, rather than repeated greedy seeds, as the resampling unit.

  2. The results expose effects that aggregate accuracy can hide. In Table 2, Ministral on SkillsBench has 17 introduced failures and 10 gained successes, while 24 of 41 current-success tasks remain successful under stale KV. The controlled audit additionally rejects 28 successful Qwen pairs for process or workspace differences. This distinction between overall success rate and paired execution reliability is useful for evaluating approximate reuse.

Critical weaknesses
  1. The deployment criterion is an ex-post audit, and its practical scope needs tighter framing. Section 6 requires both complete executions and preselected budgets; Lemma 3 follows from imposing coordinatewise bounds with nonnegative weights. It does not predict whether a cache can safely be reused before running the task, as Section 7 acknowledges. Moreover, requiring matching final workspaces can reject benign alternative artifacts, and the reported prohibited-pair count depends on the chosen budgets. The contribution is strongest as an empirical diagnostic framework rather than a new predictive safety guarantee.

  2. The public-task reliability result needs sensitivity analysis before it can be generalized. Appendix D defines success as passing at least half the tests, uses one shared seed per task, and reconstructs SWE-Skills-Bench repositories without task-specific commits. The models also have different turn/compression limits. Within-model paired comparisons remain meaningful, but the numerical baseline is substantial: on SkillsBench, only 37 of 46 Ministral recomputation successes remain successes under current-KV loading. The full-stale intervention and FP8 setting therefore do not by themselves establish the risk of practical selective-recomputation reuse systems.

Questions for authors
  1. How do introduced failures, gained successes, and absorption change under the benchmark's native success criterion or an all-tests-pass criterion? Please report paired uncertainty at the task level and distinguish missing test results from executed test failures.

  2. How much of the excess failure and execution-cost change persists with a higher-precision or numerically better-matched current-cache baseline, and with one practical partial-recomputation reuse setting?

  3. How sensitive are the 28 prohibited Qwen pairs to process budgets and to allowing semantically equivalent final artifacts? Please clarify which constraints are application requirements and which are illustrative audit choices.

Reviewer_BLqeOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how cross-context KV cache reuse affects the reliability of LLM agents. It considers repeated text segments, such as skill documents, tool schemas, and policy instructions, whose cached KV states were computed under a different preceding context. Although the visible prompt remains unchanged, reusing these states can alter token predictions and subsequent agent actions. The paper examines how these changes arise, why divergent execution paths can still complete the same task, and whether successful executions incur additional costs or workspace changes.

The authors conduct controlled experiments using OpenHands and a modified vLLM engine with Qwen3-14B and Ministral-3-14B, supplemented by evaluations on SkillsBench and SWE-Skills-Bench. Their analysis connects KV perturbations to token-score margins and investigates subsequent behavior through token interventions, cache and history restoration, and feedback ablations. The controlled experiments show frequent response and action divergence despite preserved task success, while the public benchmarks reveal that success itself is not always preserved. The paper also introduces a criterion for permissible reuse that considers task completion alongside execution costs, workspace changes, and action requirements. Its main contribution is an empirical and mechanistic account of cross-context KV reuse in interactive agents, emphasizing that task success alone provides an incomplete measure of its effects.

Critical strengths
  1. The paper examines an important limitation of task success as the sole measure of safe cache reuse. Its controlled experiments show that agents can complete the same task while taking different actions, incurring additional execution costs, or leaving different workspace states. Extending the evaluation to these aspects of interactive execution is consequential: preserving benchmark success does not necessarily satisfy the operational requirements of an agent system. This gives the study practical relevance beyond measuring output accuracy.

  2. The experimental design provides useful evidence about how cache differences affect agent behavior. The authors compare current and stale KV under identical visible inputs, include controls for numerical effects introduced by cache loading, and examine native token logits. Interventions on perturbation direction, token selection, retained state, and feedback further investigate how divergence develops and when task success survives. These controls and ablations make the mechanistic analysis a substantive contribution and strengthen confidence that the observed effects warrant further study.

Critical weaknesses
  1. The experiments do not establish whether the findings apply to practical cross-context reuse systems. The evaluation centers on stale-KV substitution without the selective recomputation or content corrections used by methods such as CacheBlend and EPIC. Although these systems are cited, they are not evaluated. It therefore remains unclear whether the observed execution costs and workspace changes persist under their reuse policies, or how those effects trade off against latency savings. This limits the practical significance of the paper’s conclusions about reliable deployment.

  2. The evidence supporting successful absorption comes from a narrowly selected task distribution. The controlled tasks are bounded file-editing problems retained only after a pilot agent succeeds twice and encounters corrective feedback. This selection favors solvable tasks with opportunities for recovery, limiting what can be inferred from the observation that all controlled stale-KV executions succeed. Several important interventions, including alternative-token and feedback experiments, also use only eight tasks, while the broader study covers two 14B models under greedy decoding in one agent framework. The results establish that absorption can occur in this setting, but do not sufficiently establish its boundaries in longer, less recoverable executions. Broader mechanism-level validation is needed to support the paper’s general account of agent behavior.

Questions for authors
  1. The experiments use stale-KV substitution without the partial recomputation employed by methods such as CacheBlend and EPIC. Which findings do the authors expect to remain valid when these corrections are applied, and what evidence supports that expectation? If any existing experiments include selective recomputation, do the observed task failures, additional execution costs, or workspace differences persist?
  2. The margin analysis characterizes when the first token changes. What evidence justifies extending this explanation to the eventual success of the resulting trajectory? In particular, does the existing analysis identify properties of natural KV differences that explain successful absorption, or should this connection be understood as an empirical observation limited to the evaluated tasks?
Reviewer_YRx7OpenReview ↗
Rating
2
Confidence
2(−1)
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies how cross-context KV cache reuse affects LLM agent execution. The authors show that reusing KV caches from different contexts can change token predictions, agent actions, and execution paths, even when the final task succeeds. They analyze how these differences arise and introduce a criterion for deciding whether cache reuse is acceptable based on task success, execution cost, and workspace changes. Experiments are conducted using Qwen3-14B and Ministral-3-14B on controlled tasks and two public agent benchmarks.

Critical strengths
  1. The paper studies an important problem in KV cache reuse, especially its impact on agent execution beyond final task accuracy.

  2. The experiments are carefully designed, with several controlled comparisons and interventions to understand how stale KV affects agent behavior.

  3. Considering execution costs and workspace changes in addition to task success is useful for evaluating the reliability of agent systems.

Critical weaknesses
  1. My main concern is that the theoretical contributions are somewhat limited. Proposition 1 mainly follows from the argmax condition, while Decomposition 2 and Lemma 3 are relatively straightforward probability and inequality results. It is unclear how much new theoretical insight these results provide.

  2. The controlled tasks seem too simple to evaluate the risks of stale KV reuse. All tasks succeed under both current and stale caches, so these experiments mainly demonstrate behavioral differences rather than actual failures. Also, the 144 pairs come from 72 tasks with two seeds, which produce nearly identical trajectories under greedy decoding.

  3. The results on public benchmarks are not very convincing. Table 2 shows both introduced failures and gained successes from stale KV reuse. For example, Qwen on SkillsBench actually gains more successful tasks than it loses. Since the experiments use only a single seed, it is difficult to draw strong conclusions about the reliability impact.

  4. The practical value of the proposed criterion is unclear. It requires comparing completed executions under current and stale KV caches, so it cannot determine whether reuse is safe before execution. The paper also does not evaluate the actual prefill savings or compare against cache reuse methods with selective recomputation.

Questions for authors
  1. How do the public benchmark results change across multiple random seeds?

  2. Can the proposed criterion be used to decide whether KV reuse is safe before execution?

  3. How much inference time does stale KV reuse save, and how does it compare with selective recomputation methods?

Reviewer_41hbOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper shows that reusing KV caches across different preceding contexts (stale KV) in LLM agents can alter attention outputs and flip near-tied token decisions, leading to divergent multi-turn execution trajectories even when the final task succeeds. To analyze and evaluate this phenomenon, the authors trace how stale KV perturbations propagate into trajectory divergence, examine how valid alternative paths and corrective environment feedback absorb these changes, and define a post-hoc permissibility criterion (BzB_z) that accounts for process overhead and workspace side effects. To demonstrate their findings, the authors perform experiments using Qwen3-14B-FP8 and Ministral-3-14B-FP8 across 72 controlled tasks as well as SWE-Skills-Bench and SkillsBench.

Critical strengths

(S1) Stepwise analysis of trajectory divergence and side effects under stale KV reuse: Rather than only checking terminal task success, the authors trace how stale KV representations flip near-tied token rankings to trigger multi-turn action divergence. They also show empirically that even when divergent trajectories still pass the task through alternative paths or corrective feedback (absorption), stale KV reuse can increase token/turn costs and leave unintended workspace modifications.

Critical weaknesses

(W1) Baseline distortion from cache-loading noise and inconsistent reference baselines: According to Section 4, Appendix B.4, and Table 9 (Appendix C.2), simply loading the exact same current-context cache (Current KV vs. Recomputation) already alters tokens in 71/72 executions (98.6%) and 130/144 parsed actions on Ministral-3-14B-FP8 (and drops SkillsBench successes from 46 to 41 in Table 17). Although the authors compare Stale KV against Current KV to control for loading noise, a Current KV baseline that diverges from standard recomputation 98.6% of the time no longer represents the model's true trajectory, and Table 17 further confounds the two effects by reporting process cost changes as stale-minus-recomputation rather than stale-minus-current.

(W2) Selection bias from pre-filtering controlled tasks for guaranteed feedback recovery: While ensuring baseline solvability (YI=1Y_I=1) is reasonable, Appendix B.1 reveals that the 72 controlled tasks (trivial single-field edits) were explicitly filtered using a Qwen3-8B pilot agent to retain only candidates that triggered at least one corrective-feedback event and still succeeded in two consecutive runs. Pre-selecting tasks specifically for reliable self-correction artificially inflates the controlled absorption rate to 100% (140/140 and 144/144 in Table 2), which contrasts sharply with the unfiltered SkillsBench tasks where conditioning on Current KV success (YI=1Y_I=1) yields only 58.5% absorption (24/41, with 17 induced failures) on Ministral.

(W3) Evaluating only naive full substitution while ignoring partial recomputation: Practical cross-context KV reuse systems such as CacheBlend (Yao et al., 2025), EPIC (Hu et al., 2025), and CacheSlide (Liu et al., 2026b) specifically mitigate stale-cache errors by selectively recomputing a small fraction of high-deviation or chunk-boundary tokens under the current prefix. In Section 3, however, the authors explicitly exclude these partial recomputation mechanisms and only test naive 100% full stale substitution (KsS\mathcal{K}_s^S for KsI\mathcal{K}_s^I), which overstates the severity of the problem compared to realistic deployments.

(W4) Lack of practical utility in the post-hoc permissibility criterion: The proposed permissibility rule (BzB_z in Section 6, Equations 12–13, and Appendix A.3, Equation 29) requires executing both the current-KV reference trajectory (τI\tau^I) and the stale-KV trajectory (τS\tau^S) to completion in order to compare their task outcomes, cost deltas (ΔCz\Delta C_z), and workspace side effects (ΔXz\Delta X_z). Because the entire purpose of KV cache reuse is to avoid running the full reference computation, a post-hoc criterion that cannot predict permissibility before execution (as acknowledged in Section 7) cannot be used for online serving decisions.

References

  • Junhao Hu, Wenrui Huang, Weidong Wang, Haoyi Wang, Tiancheng Hu, Qin Zhang, Hao Feng, Xusheng Chen, Yizhou Shan, and Tao Xie. EPIC: Efficient position-independent caching for serving large language models. In International Conference on Machine Learning, pp. 24391–24402, 2025.
  • Yang Liu, Yunfei Gu, Liqiang Zhang, Chentao Wu, Guangtao Xue, Jie Li, Minyi Guo, Junhao Hu, and Jie Meng. CacheSlide: Unlocking cross position-aware KV cache reuse for accelerating LLM serving. In Proceedings of the 24th USENIX Conference on File and Storage Technologies (FAST), pp. 83–99, 2026b.
  • Jiayi Yao, Hanchen Li, Yuhan Liu, Siddhant Ray, Yihua Cheng, Qizheng Zhang, Kuntai Du, Shan Lu, and Junchen Jiang. CacheBlend: Fast large language model serving for RAG with cached knowledge fusion. In Proceedings of the Twentieth European Conference on Computer Systems (EuroSys), pp. 94–109, 2025.
Questions for authors

Please see the critical weaknesses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.