Progressive Hierarchical Context Compaction for LLM Agents

Long-horizon LLM agents accumulate increasingly large interaction histories, making context management essential for both inference efficiency and task performance. An important challenge in context compaction is how to maintain and progressively update historical information at different levels of abstraction as an agent executes. We introduce a progressive hierarchical context-compaction framework that maintains an agent's working context as an ordered sequence of blocks at three abstraction levels: trajectory, subtask, and state. At each compaction checkpoint, an LLM compactor examines the current working context and chooses an abstraction level. The selected level determines how much of the recent history is rewritten, producing an updated working context. Repeated compaction progressively produces a hierarchical representation in which different portions of interaction history are maintained at different abstraction levels. The framework further supports a plug-and-play compactor that can be trained independently of the task-performing agent. Experiments on BrowseComp-Plus and NL2Repo-Bench demonstrate substantial improvements in task performance and the context processed per task-model call over existing compaction methods. Training a smaller compactor using decisions generated by a stronger teacher yields further performance improvements.