- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes AsynCompact, a framework for reducing the latency of context compaction in long-horizon LLM agents. It has two parts:
- Asynchronous compaction. The history summary is generated in the background while the agent keeps working.
- KV grafting. When the agent switches to the compacted context, its KV cache is built by reusing and re-positioning cached states from both branches instead of recomputing them.
The method is implemented in SGLang and evaluated on agentic coding and terminal benchmarks. The paper reports less compaction blocking and lower switch latency, with task performance comparable to synchronous compaction.
Critical strengths
-
Practical problem. The paper studies a practical problem and runs experiments on realistic agentic benchmarks.
-
Solid engineering for KV grafting. The authors implement grafting inside SGLang's radix and paged KV cache. It supports several attention mechanisms (full attention, hybrid linear attention, and sparse attention) and speculative decoding through MTP draft-KV transfer. Supporting all of these is nontrivial and useful to practitioners.
Critical weaknesses
W1. Limited novelty in asynchronous compaction.
The async protocol is not a novel design. Two papers the authors already cite do something very similar:
- Slipstream summarizes in the background while the agent continues on the full context, then swaps the summary in.
- SmoothAgent runs summarization as a background "lookahead" operation. CoMem (in ICML 2026) likewise has a async compaction mechanism that leverages a separate model to summarize the history in the background while the agent keeps working. This async compaction mechanism is also already used in popular agent harnesses, such as Copilot and Codex.
The paper does not identify a substantive difference from these designs, so this part reads as reimplementation rather than a new contribution.
W2. Limited novelty in KV grafting.
This leaves KV grafting as the main contribution, but the mechanism also lacks novelty. Grafting concatenates cached KV segments and moves their keys to new positions by RoPE rotation, with no recomputation (Eq. 6). This operation is well established:
- "Position re-encoding" in Block-Attention (Sun et al., ICLR 2025, §2.3) and KVLink (Yang et al., 2025, §2.2).
- "Full KV reuse" in Prompt Cache (Gim et al., 2024) and CacheBlend (Yao et al., 2025, §3.3 and App. A). CacheBlend shows it is insufficient without selective recomputation.
- The "Naive" baseline in EPIC (Hu et al., ICML 2025, §2.3), described as "too rudimentary to be classified as [a] proper algorithm" and shown to cause substantial accuracy loss.
- llama.cpp's KV context shift (PR #3228, 2023), which re-rotates cached keys in place after dropping tokens.
The new element is applying this to agent compaction. There, the reused KV states were computed while attending to the history that compaction removes, rather than lacking preceding context. That does not change the mechanism. The paper's own fidelity analysis (about 89% top-1 agreement, Sec. 3.2.2) shows the familiar deviation from a full prefill.
Besides, the work is also suffered from limited generality of the evaluation and integration. All experiments use a minimal custom coding harness (a shell tool plus a context manager). It is unclear whether the results transfer to widely used harnesses such as Hermes Agent, Claude Code or Codex.
Questions for authors
-
Could the authors explain the technical novelty / contribution relative to the works mentioned in the weaknesses section?
-
Could the authors please expand the experimental analysis with further harnesses e.g. CC or Codex?