- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces PACE, a reinforcement learning framework designed to train a memory compactor for language model agents. PACE optimizes the compactor by comparing an agent's future performance when given a complete history versus a compressed one from the same state. The objective incorporates three key factors: one-sided downstream regret, task-state preservation, and memory compactness, using recursive training for multi-step compression. Experiments on HotpotQA and MuSiQue with Qwen3.5-4B as both the compactor and agent demonstrate the effectivness of PACE against several compression baselines.
Critical strengths
- A fresh, useful perspective on learning agent memory. PACE frames compression as preserving future decision utility and makes this concrete through boundary-local comparisons between full and compressed histories. Its distinctive emphasis is on turning the harm caused by individual compression decisions into a direct training signal for the compactor.
- Strong results within the evaluated setting. On 8K HotpotQA, PACE achieves 81.90 F1 using approximately 135 memory tokens, compared with 73.06 F1 and 251 tokens for RECOMP-Ext. Direct transfer to MuSiQue and the rolling-compression results provide encouraging evidence that the learned compactor preserves useful information beyond its training setting.
Critical weaknesses
- The contribution over closely related approaches is insufficiently established. ACON already optimizes compression through downstream outcomes and full-versus-compressed trajectory comparisons, while Context Folding/FoldGRPO learns context management through RL. PACE’s boundary-local counterfactual objective is a plausible distinction, but its advantages are neither clearly motivated nor experimentally demonstrated against these approaches. Moreover, the terminal-reward ablation uses a synthetic scalar policy, and the LLM regret-weight study excludes a zero-regret condition, leaving the contribution of counterfactual feedback unclear.
- Limited evaluation scope. The main experiments use constructed multi-document QA histories from HotpotQA and MuSiQue, with Qwen3.5-4B as the sole backbone. One interactive evaluation, such as AppWorld or a BrowseComp search-agent setup, would more directly test these claims. LongMemEval would provide complementary evidence on long-term memory, while an additional model classes (e.g., DeepSeek-V4-Flash) would help assess model generality.
Questions for authors
-
What specific limitation of ACON’s contrastive guideline optimization or FoldGRPO’s process-reward training motivates PACE’s approach? Can a controlled comparison establish the benefit of boundary-local counterfactual feedback over one of these alternatives?
-
Could the authors provide evaluation on further benchmarks (agentic workflows or LongMemBench-like tasks) and model families?