- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper explores the idea of reducing the cost of agent context compaction by reusing the retained KV cache before compaction. The idea is that during agent compaction, while some histories are summarized, a notable number of messages are retained with their KV cache readily available and thus can be reused. The problem is that the KV cache of the same texts before and after compaction differs, leading to different subsequent agent behaviors. The paper then studies the efficiency as well as behavior differences of this reuse mechanism compared to the no-reuse baseline.
Critical strengths
- The paper is well motivated by identifying that retained messages account for a median of 85.2% of the compressed observed-message input. This makes reusing the existing KV cache of these messages reasonable.
Critical weaknesses
- My main concern is that the paper provides very limited evidence (46 tasks) on whether the KV cache reuse hurts the agent's task success rate. I think this is what the authors should put more effort into, rather than emphasizing too much on how the reuse changes the semantics or behavior without relating them to how they change the task success rates. It's quite obvious that the model's outputs will change given different KV cache input. What matters is to what extent the reuse hurts utility.
- The reported speedup is measured with the reusing cache prepared, with some potential reuse-specific overhead excluded.
Questions for authors
- Can the authors provide more convincing evidence on the extent to which KV cache reuse hurts its utility?
- Can the authors provide an end-to-end comparison of speedup?
2 comments
Thank you for the response. However, I don't find the additional explanation sufficient to address the concerns without further empirical evidence. I am keeping my initial rating.
Functional utility, full-data reference, and cost accounting
Thank you for bringing the discussion back to task utility. We agree with your central point: showing that cached representations change an agent's outputs is not enough to tell a reader how much useful performance is lost. This prompted us to put official test outcomes at the center of the revised evaluation and to revisit how we present the task statistics.
What we can say about utility
The controlled three-boundary study now measures local recovery and official tests on the same Mistral executions. REUSE reduces local recovery time at all three boundaries (1.56×–8.56× versus FRESH-1). Final official-test counts agree with FRESH on Astropy and Sympy, but Seaborn has seven additional preservation-test failures relative to both FRESH repeats (86/94 versus 93/94 passing). All three boundaries remain unresolved within the fixed budget. The practical finding is therefore a local saving with a measured functional risk, rather than a task-success improvement. Your broader question about the frequency and average size of this risk still calls for a larger controlled evaluation from the start of each task.
The new strict-configuration Astropy comparison also executes selective 10%, FRESH and REUSE for ten calls each. All end at 0/4 target and 63/68 preservation tests, versus 68/68 preservation initially. It is one additional comparison on an existing boundary, not three independent tasks or new successful solves. Its 2199.2 s of arm spans and 248.8 s of scoring are reported separately; task-prefix and initial-summary costs remain missing.
Your emphasis on utility also changes how we interpret differing actions. In the strict Astropy comparison, different edit-argument forms produce the same first patch and tool result: a verified case where output disagreement does not mean a different state change. Later tool feedback could also compensate for an earlier divergence, but we have not established that mechanism. Conversely, equal test counts can hide different failures, and all three arms here remain unresolved. We use this trace to explain why behavioral disagreement alone is a poor utility metric, not to claim equal task quality.
Across the 54 historical task pairs analyzed in the revision, REUSE solves 68.5% (37/54), compared with 70.4% (38/54) for FRESH: a difference of one task, or −1.85 percentage points. Appendix B documents the analysis set, inclusion decisions and complete reference results. We treat these rates as descriptive evidence, rather than a non-inferiority result.
The prompt audit identified differences in system text after the initial branch request, so this historical comparison does not isolate the effect of KV reuse. Normalizing workspace paths removes the inspected text differences, but their effect on task outcomes was not measured. We therefore cannot attribute the observed gap solely to reuse or dismiss it as a prompt artifact. This is why the new controlled study fixes system prompts and paths throughout and reports its results separately. It provides the cleaner test of the mechanism; the historical rates provide broader, less controlled task coverage.
The two Django exclusions have separate protocol grounds: an invalid second FRESH backend boundary and an out-of-contract shared-summary fallback. The audit appendix retains these decisions, unscored attempts and original scores. Four earlier short continuations preserve already-passing fork states; their 8/8 passing arms are not new task solves and do not enlarge the success-rate denominator.
Accounting for the work around recovery
Your concern about prepared-cache timing is also well taken. We have made the starting state and excluded work explicit. The twelve-boundary 10.98× result starts with complete old KV and a shared summary, charging rendering/alignment, selection/copying, summary prefill, correction, assembly and host-visible first-token extraction. It includes more recovery work than the original prepared-cache measurement, but still excludes summary generation, model loading and task-prefix work. The new study adds actual continuation wall time and official utility, while explicitly retaining its narrower local timer.
Historical logs recover partial backend costs for 99 of 102 attempts; three lack the necessary logs. Summary time is a subset of generation time, and tool, transport, disjoint KV preparation and full-path timestamps are unavailable. We leave those quantities missing rather than treating them as zero; the available logs cannot support a complete-task speedup claim. Three qualification failures and two numerical diagnostics cost 637.1 s including startup and cleanup, retained despite missing scientific outcomes.