acceptodds
Withdrawn from ICLR 2027

ScrubAgent: Context Scrubbing for Scalable Long-Horizon LLM Agents

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Large language model agents trained with reinforcement learning have achieved strong results on complex tasks requiring extended interaction. Recent work addresses the challenge of growing context by compressing or folding interaction history. These methods reduce context length but do not help agents locate critical information. Valuable discoveries remain mixed with failed attempts and repeated searches. Agents face disorganized history when making decisions. This motivates us to examine how humans process long content such as movies. Rather than passively receiving every frame, viewers actively construct structured memory. They note key plot points and skim over routine scenes. Important moments are highlighted and become easier to recall later. \ourmethod brings this approach to LLM agents. Through five operations, agents learn to compress routine steps, preserve important exploration and store discoveries as retrievable keyframes. We propose ScrubGRPO with process rewards that guide compression and memory decisions. On BrowseComp-Plus and SWE-Bench Verified, \ourmethod achieves 63.3% and 59.2% pass@1 respectively with over 65% context reduction. It outperforms full-context baselines and generalizes effectively to trajectories longer than those seen during training.

Primary area:
reinforcement learning
Keywords:
LLM agents, long-context reasoning, memory, reinforcement learning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_ehHzOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies context and memory management for long-horizon LLM agents. ScrubAgent integrates task actions with operations for protecting selected context spans, compressing recent interactions, storing discoveries as retrievable keyframes, and recalling relevant evidence. ScrubGRPO trains this policy with outcome rewards and process penalties targeting context pressure, unused keyframes, and information loss. The paper evaluates the approach on a 150-question BrowseComp-Plus split and 500 SWE-bench Verified instances, and reports component ablations, memory-retrieval analyses, context-budget sweeps, and computational costs. Its intended contribution is to combine selective preservation of detailed context with directly retrievable task evidence under a bounded working context.

Critical strengths
  1. A practically relevant problem and a concrete, reusable interface. Long-horizon agents must retain decisive evidence while controlling the cost of accumulated tool observations. The proposed combination of protected context spans and retrievable keyframes provides an explicit interface for addressing this tension. Its use in both information-seeking and software-repair tasks suggests potential applicability across different tool environments, although this does not establish zero-shot cross-domain generalization.

  2. Useful diagnostic analyses beyond aggregate task scores. The manuscript includes component ablations, retrieval variants, context-budget sweeps, and cost accounting. Subject to clarification of the evaluation protocol and reported numbers, these analyses can help explain when memory preservation and retrieval are useful and what costs they incur. In particular, the distinction between active-context compression and end-to-end latency is useful for assessing practical benefits.

Critical weaknesses
  1. Limited demonstrated novelty relative to closely related work. Several central ideas have close precedents. AgeMem integrates storage, retrieval, summarization, and discarding into the agent's policy and trains unified memory management with step-wise GRPO. AgentFold jointly generates context-folding directives and tool actions at each step, preserving useful details through multi-scale condensation. Context Folding/FoldGRPO already learns active context management through end-to-end RL with process rewards. Cognitive Scaffold converts context into structured event snapshots and proactively retrieves precise evidence. These methods differ in their memory representations, training procedures, and task settings, so I do not consider them identical to ScrubAgent. Nevertheless, the manuscript does not sufficiently establish why its particular combination of focus windows, task-local keyframes, and process penalties constitutes a substantial methodological advance. The paper compares against Context Folding and adapted memory methods, but lacks direct comparisons with important close alternatives such as AgeMem and AgentFold. Component-removal ablations establish usefulness within the proposed system, rather than novelty over existing combinations. Matched comparisons against an existing folding agent augmented with the same keyframe store would be especially informative.

  2. Limited evidence on model-size scaling and a missing matched control. Increasing the backbone from Qwen2.5-7B-Instruct to 14B improves BrowseComp-Plus from 54.8% to 56.4%, but SWE-bench Verified only from 53.6% to 53.8%. The manuscript itself reports that the latter difference is not statistically significant (p = 0.34). However, it does not report corresponding 14B ReAct/standard-GRPO baselines, making it impossible to determine whether the small gain is intrinsic to this task and training setting or is associated with the proposed memory policy. This is particularly relevant because increasing the 7B working context from 32K to 64K yields larger reported gains of 3.9 and 2.2 percentage points, respectively. A matched model-size-by-method comparison, together with a memory/context-capacity analysis, is needed to establish whether the method preserves the benefits of a stronger backbone. This concern limits the evidence for parameter scaling; it does not by itself refute the separate claim of supporting longer interaction horizons.

  3. Unresolved inconsistencies in the reported evaluation and abstract claims undermine confidence in the empirical evidence. Appendices H/I.4/K specify three independent seeds, 150 BrowseComp-Plus tasks, one greedy episode per task, and binary correctness, with scores averaged over seeds. Under this protocol, the mean must be 100k/450 percent for an integer k. However, the reported 54.8% is not attainable after rounding to one decimal: 246 successful episodes yield 54.7%, whereas 247 yield 54.9%. The 54.3% FoldGRPO baseline has the same incompatibility. These observations do not identify whether the error lies in implementation, aggregation, reporting, or the protocol description, but they prevent verification of the headline result and comparisons; the associated confidence intervals and significance tests also require reconciliation with the underlying data. Separately, the supplied OpenReview abstract reports 63.3% / 59.2% without specifying model size, whereas the manuscript abstract reports 54.8% / 53.6% at 7B and 56.4% / 53.8% at 14B. The OpenReview abstract explicitly claims effective generalization to trajectories longer than those seen in training, while the manuscript abstract only states that trajectory-length and retrieval analyses are reported. The core method remains consistent and neither abstract states a formal theoretical guarantee, so this should not be characterized as a change in algorithm or a conflicting theorem. Nevertheless, the authors need to identify the authoritative result version, reconcile the evaluation and aggregation protocol, and clarify the scope and evidence for the generalization claim. Until then, the reliability and scope of the empirical contribution remain uncertain.

  4. Insufficient grounding of the human-cognition motivation. The abstract, introduction, and Figure 1 repeatedly motivate the design through claims that movie viewers skim routine content, preserve important scenes at higher resolution, and later recall bookmarked scenes directly. The manuscript does not cite cognitive-science evidence for these specific claims or clearly distinguish established findings from an engineering analogy. Research by Baldassano et al. and Ben-Yakov and Henson on event segmentation and episodic memory supports structured encoding of continuous experience, but does not directly establish the proposed combination of adaptive resolution and explicitly addressable keyframes. The component ablations demonstrate utility for the agent, rather than correspondence with human memory mechanisms. The authors should either substantiate the mapping between cognitive findings and design choices or frame the movie example explicitly as an analogy and ground the technical motivation in measurable evidence-retention and retrieval failures. I view this as a supporting concern about the argument and positioning, rather than a standalone reason to reject an otherwise strong engineering contribution.

Questions for authors
  1. Clarification of evaluation aggregation. How can the 54.8% ScrubAgent score and 54.3% FoldGRPO score arise under the stated protocol of three seeds, 150 tasks, and one binary-scored episode per task? Please provide all 150 task IDs and the 3-by-150 binary outcome matrices for the compared methods, per-seed success counts and denominators, and the exact aggregation code. If additional runs, weighting, exclusions, non-binary scores or a different estimator were used, please state this explicitly and correct the protocol description. Please also provide the bootstrap inputs and explain how seed and task variability were handled, then recompute affected means, deltas, confidence intervals and p-values as necessary.

  2. Clarification of abstract versions and claim scope. Which model, checkpoint, evaluation split and protocol produced the OpenReview abstract's 63.3% / 59.2%, and how do those results relate to the manuscript's 7B and 14B scores? Do the two abstracts describe the same implementation and experiment version? Does the paper retain the OpenReview abstract's explicit claim of effective generalization beyond training trajectory lengths, and which controlled experiment supports it? The supplied manuscript groups tasks by each method's own executed length; please distinguish this conditional analysis from a comparison on a common fixed task set or a controlled horizon-extrapolation experiment. This is a request to identify the intended claim and its existing evidence, not an assumption that the method has changed.

  3. Clarification of the evaluation setup and unusually strong base-agent scores. Table 1 reports 38.0% and 48.8% on SWE-bench Verified for short- and long-context Qwen2.5-7B-Instruct ReAct without the listed RL stage. Please identify the exact checkpoint and any prior training, and release the prompts, environment configuration, predictions, and complete evaluation reports for all 500 instances. Please explain the handling of timeouts, failed patch applications and test failures, and how hidden evaluation artifacts are isolated from the agent. For BrowseComp-Plus, please provide the 680/150 split IDs, split provenance, corpus configuration and scorer, and indicate which reference-model rows were rerun under the identical setup versus imported from other sources. Different setups may explain the absolute scores; the current description is insufficient to verify this explanation.

  4. Evidence needed to assess scaling. Can the authors provide 14B ReAct/standard-GRPO results under the same data and evaluation budgets, and establish whether expanding the working context or keyframe capacity changes the 7B-to-14B gain? The matched baseline is central to interpreting the current scaling result; the capacity sweep would help diagnose its cause.

  5. Evidence needed to assess the additional contribution. What benefit remains over AgeMem or AgentFold, or over a folding agent equipped with the same keyframe storage and retrieval module? Please distinguish any existing evidence from proposed additional experiments. Matching the maximum 32K context alone does not match actual inference cost: Table 2 reports 210.5K prefill tokens for ScrubAgent versus 78.6K for FoldGRPO. A comparison at comparable actual cost would materially affect the assessment of the additional mechanism.

Reviewer_jVgiOpenReview ↗
Rating
2
Confidence
4
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces ScrubAgent, a framework for learning context management in long-horizon LLM agents. The agent jointly produces task actions and memory operations, allowing it to preserve detailed observations, compress routine interactions, and store important discoveries as retrievable keyframes. The authors further propose ScrubGRPO, which combines task outcome rewards with process rewards for context efficiency and information preservation. Experiments on BrowseComp-Plus and SWE-Bench Verified demonstrate improved task performance over several context-management baselines using Qwen2.5 backbones.

Critical strengths
  1. The paper is clearly written and well organized. The motivation, memory operations, and training procedure are easy to follow, and the figures help explain the overall framework.
  2. The combination of adaptive context resolution and retrievable keyframes is an interesting approach to agent memory. The operation interface is relatively independent of individual tasks and has the potential to serve as a general context-management mechanism.
  3. The work presents a coherent pipeline connecting memory operations, context construction, and reinforcement learning. Evaluation across research and software-engineering tasks, together with component ablations and retrieval analyses, provides a reasonably complete examination of the proposed system.
Critical weaknesses
  1. The trade-off between inference cost and performance needs stronger justification. Relative to Context Folding with FoldGRPO, ScrubAgent increases cumulative prefill on BrowseComp-Plus from 78.6K to 210.5K tokens, approximately 2.7×, while improving pass@1 by only 0.5 percentage points, a statistically insignificant difference. The improvement on SWE-Bench Verified is 2.2 points. Given the additional summarization and retrieval components, it remains unclear whether these gains justify the added cost relative to this strongest baseline. A direct comparison of end-to-end latency and total inference compute would make this trade-off clearer.
  2. The evidence for generality across backbone models is limited. The proposed method is evaluated only on Qwen2.5-7B and Qwen2.5-14B. Extending the experiments to additional model families or generations, such as Qwen3 or Qwen3.5, would strengthen the paper. Stronger models may have different abilities to identify important evidence and manage their own context, so it is unclear whether the observed benefits persist as the backbone becomes more capable.
  3. Training-free applicability to stronger models remains unexplored. The paper includes a no-RL experiment with Qwen2.5-7B, but this does not establish whether a stronger model, such as Claude, could effectively use the proposed operations through prompting alone. If provided with the same memory tools and suitable instructions, could such a model organize its context without additional training? This comparison would help distinguish the benefits of the operation design from those of ScrubGRPO.
  4. The relationship to agent-managed file memory needs clarification. A related harness design allows agents to write, update, and consult Markdown notes while solving complex tasks. This appears to overlap with the motivation behind keyframe writing and retrieval. Appendix D includes a fixed 512-token notepad baseline, but it does not directly address a more flexible setup in which the agent decides when and how to maintain external notes. The authors should clarify the conceptual differences and, ideally, compare against such a baseline under matched model and resource budgets to better isolate the contribution of the proposed memory structure and training method.
Questions for authors

See weakness above. I appreciate the novelty of the work and the thoughtful design of the proposed framework. However, the experimental evaluation is missing several important results needed to address the concerns above. I would be willing to raise my score if the authors can adequately address these concerns.

Reviewer_DzhtOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes ScrubAgent, a long-horizon LLM agent that manages its own context through learned memory operations: it keeps important steps at high resolution, compresses routine ones, and stores key discoveries as retrievable keyframes. The policy is trained with ScrubGRPO, a GRPO variant with process rewards for these operations. With Qwen2.5-7B, the paper reports 54.8% and 53.6% pass@1 on BrowseComp-Plus and SWE-Bench Verified, with 65% context compression.

Critical strengths
  1. Well-motivated design. Treating compression and memory writes as actions of one RL-trained policy is a natural way to keep key evidence retrievable.
  2. Broad evaluation. The same operations work across deep research and software engineering, with extensive appendix analyses.

Given the concerns below, these do not change my recommendation.

Critical weaknesses
  1. The experimental results cannot be verified as independent runs. The Qwen2.5-7B baselines in Table 1 are almost exactly a constant fraction of the Seed-OSS-36B results of Sun et al. (2025): SWE-Bench Verified pass@1 is ≈0.885× and tool calls ≈0.91× the 36B values in nearly every shared setting, a pattern far tighter than the paper's own seed variance (Table 18). The GPT-5, DeepSeek-V3.1 and GLM-4.5-Air rows also match Sun et al. without attribution. Since every main claim rests on these comparisons, I cannot assess them without per-seed, per-task outputs and logs.

  2. Limited novelty. The closest prior work is Context Folding / FoldGRPO (Sun et al., 2025), which ScrubGRPO closely follows: the policy is trained under the managed context, token-level process rewards enter the advantage, uncompressed tokens are penalized once the context exceeds 50% of the budget, and an LLM judge adds a further penalty. Apart from the keyframe-utility term, the algorithmic difference is small, yet the paper never discusses FoldGRPO as a precursor. The two memory mechanisms also have direct precedents: keyframe/recall mirrors MemGPT's archival memory insert/search (Packer et al., 2023), and compress_recent mirrors AgentFold's retrospective folding of multiple steps into one summary (Ye et al., 2025).

  3. The comparisons are not strong enough.

    • Against the closest baseline, ScrubAgent ties Context Folding + FoldGRPO on BrowseComp-Plus (54.8 vs. 54.3, p = 0.24) while using 2.7× its prefill (210.5K vs. 78.6K tokens).
    • The gains from RL and from the memory design are not separated. Most baselines receive no RL, and Summary Agent + RL is missing. Without RL, ScrubAgent (38.9/44.5) falls below the tuned hand-crafted policy (41.5/46.8, Table 8), whose largest gap to ScrubAgent is on easy tasks (59.4 vs. 78.4), where context management should matter least.
    • CLEX (a RoPE extension) and InfLLM (inference-time KV retrieval) are not agent context-management methods, the closest learned context-editing methods (AgentFold, MemAct) are not compared, and the 14B model has no baselines.
Questions for authors
  1. How were the Qwen2.5-7B baselines in Table 1 obtained? Were ReAct, Summary Agent and Context Folding (with GRPO/FoldGRPO) trained and evaluated on Qwen2.5-7B as described in Sec. 5.1 and Appendices A and K? If so, please explain why their SWE-Bench Verified pass@1 and tool calls are, in nearly every shared setting, ≈0.885× and ≈0.91× of the Seed-OSS-36B values in Sun et al. (2025), and how the 327K long-context ReAct was run on a model that supports 32K natively and 128K with YaRN.
  2. How were the BrowseComp-Plus numbers computed? With 150 tasks, one greedy episode per task and three seeds (Appendix I.4), pass@1 must be a multiple of 1/450, yet several reported values (e.g., 54.8 and 54.3) are not. Please report per-seed scores, and the number of tasks on which ScrubAgent and FoldGRPO disagree behind the [−0.3, +1.3] interval in Table 18.
  3. Where do the reference rows come from? The GPT-5, DeepSeek-V3.1 and GLM-4.5-Air rows match Sun et al.'s Table 1 exactly. For Claude 4 Sonnet and Gemini 2.5 Pro, please give the source and setting, since the official BrowseComp-Plus results for these models (36.8% and 28.7% with the Qwen3-Embedding-8B retriever) are far lower than those reported here.

If these questions can be answered clearly and unambiguously, I would rate the paper higher and reassess it on its remaining merits.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.