- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
The paper proposes RECAP, an RL framework for jointly training context compaction and agent policies in long-horizon LLM agents. The central idea is that an effective summary should preserve the agent's behavior induced by the original full history. The authors design a behavior distortion objective based on the KL divergence between the policies conditioned on the original full history and compacted contexts.
Experiments on BrowseComp-Plus and SWE-bench demonstrate improvements from RECAP.
Critical strengths
-
Context compaction has been demonstrated useful in long-horizon task. The paper aims to simultaneously optimize the context compaction and agent policy. The paper has clear motivation.
-
The experiments demonstrate improvements across both deep-research and software-engineering tasks.
-
The paper is well written and easy to follow.
Critical weaknesses
- My main concern is the assumption that the full-context policy provides the desired behavior that the compacted policy should imitate. Context compaction is not necessarily just an information-preservation mechanism. In long-horizon agent tasks, the full interaction history may contain redundant observations. By removing such distracting information and highlighting task-relevant states, a compacted context may actually enable the same model to make better decisions than when conditioned on the full context.
In other words, the proposed self-distillation objective encourages behavior preservation, whereas the ultimate goal of context management should be improving task performance, which does not always require preserving the original behavior. The paper does not sufficiently investigate when these two objectives are aligned or conflicting.
- The experimental results show that self-distillation improves overall performance, which supports the effectiveness of the proposed method. However, these results do not establish that imitating the full-context policy is always beneficial. In particular, it remains unclear what happens when the compacted-context policy already outperforms the full-context policy at a given decision state. Does self-distillation still improve subsequent decisions, or does it pull the compacted policy toward an inferior behavior?
Questions for authors
See weaknesses.
5 comments
Many thanks to authors' detailed response. I still have two questions:
-
What is the detailed setting for Compacted policy?
-
Have you evaluated task success rates across different task lengths (e.g., different numbers of turns)? Does the full-context policy consistently outperform the compacted policy on longer tasks?
Thank you for your the kind follow-up questions:
-
Setting for compacted policy: We use 32K context window with context compaction. When the active context reaches 32k token limit, the model generates a handoff summary using the prompt specified in Appendix E.1. The agent then resumes the same task with the original task instructions, the summary, and the most recent action and its observation ( on both benchmarks). We allow up to three compactions per task.
-
Task success rates across different task lengths: We thank the reviewer for the great suggestion. We group the tasks by their average number of steps and compare the performance of full-context policy and compacted policy below
| Task | Model | Avg. steps | Full context (%) | Compacted (%) | Gap (pp) |
|---|---|---|---|---|---|
| BrowseComp-Plus | Qwen3.5-4B | 86.9 | 82.1 | +4.8 | |
| 92.4 | 84.8 | +7.6 | |||
| 32.3 | 22.9 | +9.4 | |||
| BrowseComp-Plus | Qwen3.5-4B Trained | 91.9 | 94.2 | −2.3 | |
| 82.9 | 85.5 | −2.6 | |||
| 31.5 | 31.2 | +0.3 | |||
| SWE-bench Verified | Qwen3.5-4B | 60.8 | 56.5 | +4.3 | |
| 50.5 | 45.3 | +5.2 | |||
| 40.7 | 33.8 | +6.9 | |||
| SWE-bench Verified | Qwen3.5-4B Trained | 64.4 | 64.0 | +0.4 | |
| 48.3 | 47.2 | +1.1 | |||
| 40.0 | 34.8 | +5.2 |
Both policies generally achieve lower success rates on longer tasks. For the base model, the full-context policy consistently outperforms the compacted policy, with the performance gap increasing as task length increases on both environments. After training, the gap reduces for BrowseComp-plus, while still persists for SWE-bench Verified. Overalll, these results suggest that the advantage of full-context policy is more pronounced on the challenging long-horizon tasks.
We thank the reviewer for the invaluable comments. We’re happy that the reviewer thinks our paper is clearly motivated, well written and easy to follow. We address the reviewer’s concern below.
We thank the reviewer for the important question. RECAP uses the full-context policy as a self-teacher for training the compacted policy of the agent. To test whether the full-context policy serves as a good teacher, we compare the performance of full-context and compacted policies in different scenarios, as shown below.
Table 1: Task success rate (%) on BrosweComp-Plus.
Table 2: Task success rate (%) on SWE-bench Verified.
The full-context policy performs better in 5 out of 8 settings, with an average advantage of 3.1 points. For Qwen3-Coder, task success rates were similar across the two settings, as approximately 90% of trajectories terminated without triggering context compaction. The only exception is the trained Qwen3.5-4B on BrowseComp-Plus (-1.0%), where the difference is minor. The results strongly indicate that the full-context policy generally provides a strong reference for the compacted policy.
Finally, we appreciate the reviewer’s insight that context management can remove redundant observations while preserving task-relevant information. RECAP is compatible with this form of selective compression. Redundant information can be discarded without penalty as long as its removal does not affect subsequent behavior, whereas omitting information that matters for future decisions increases behavioral distortion and is penalized. Thus, the distortion objective provides a functional criterion for what should be preserved and what can be safely removed. Importantly, behavior preservation remains an auxiliary regularizer alongside the task reward, allowing task optimization to favor compacted behaviors that differ from the full-context behavior and potentially achieve better task performance. We’ll include the discussion in the revised manuscript.
In response to the reviewer’s suggestion, we take the trained Qwen3.5-4B whose compacted policy behaves slightly better than the full-context policy (62.8% vs. 61.8% in the table above). We continue training for 10 steps using a frozen full-context teacher without task-success rewards. We separately optimize the summary reward (gsum), post-compaction action distillation (gpost), or both. The results are summarized below:
*Table 3: Task performance after additional training using behavior distortion. *
Interestingly, the summary reward gsum alone improves the performance, while self-distillation on actions with gpost reduces the task accuracy. These results show that behavior preservation alone does not always improve the task performance, and support the design choice of RECAP which takes the distortion objective as an auxiliary regularization objective alongside the task success reward. We will clarify this in the revised manuscript.