AsynCompact: Efficient Context Compaction for Long-Horizon LLM agents

Context compaction supports long-horizon LLM agents but can interrupt execution during summary generation and KV-cache reconstruction. We present a compaction framework that reduces these interruptions through asynchronous execution and optional KV grafting. The agent continues its task while a background branch generates a summary, then adopts the compacted history at a safe step boundary. For self-hosted inference, KV grafting further reuses cached states for the summary and retained context, with positional rebasing, instead of fully recomputing the compacted context. This reuse is approximate and requires no model retraining. Experiments on three benchmarks show broadly comparable task performance. Controlled boundary replays show that asynchronous compaction reduces mean explicit summary blocking by $73.7$–$84.5\%$. In a separate Qwen3.8-27B replay with one active request per replica, KV grafting reduces mean post-compaction resumption latency by $53.2\%$, including graft-specific preparation costs. These results demonstrate reduced waiting at compaction boundaries, supporting more responsive agent execution without relying on consistent full-task speedups.