acceptodds
Under review as a conference paper at ICLR 2027

Learning to Manage Context: Training Agents with Meta-Tools Beyond a Single Context Window

29 views

Abstract

An agent that continually accumulates interaction history eventually exhausts its context window, thereby limiting the task horizons over which they can be trained. We introduce the Concurrent Agent Rollout Process (CARP), which brings context-management mechanisms into a common framework for agent execution and training. CARP encapsulates these mechanisms as meta-tools, allowing agents to replace accumulated histories with summaries or delegate work to other agent instances. By retaining each generated token’s conditioning context, we can compute its log-probability for gradient-based training without requiring the complete execution to fit within one context window. We address obstacles which prevent agents from reasoning about context capacity. Using Group Relative Policy Optimisation (GRPO), agents learn to solve synthetic counting tasks whose data exceeds their context window, making context management necessary. Different reward and prompt conditions produce different strategies, including parallel task decomposition when the reward accounts for parallel computation savings. These results lay the groundwork for end-to-end training of agents on tasks whose execution extends beyond a fixed context budget.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLMs, Agents, Meta Learning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_aiVUOpenReview ↗
Rating
2(+1)
Confidence
2
Informativeness
2
Presentation
1
Soundness
2(+1)
Read review
Paper summary

The paper introduces CARP, an execution model that treats an agent as a collection of threads and offers context management through two meta-tools, Compact, which swaps a thread's history for a summary, and Delegate, which launches a subagent. During training the input and output of every model call are stored, so GRPO can assign the episode advantage to each sampled token without the full execution having to fit in a single window. Under a 4,096-token window, Qwen3.8-27B is trained for at most ten steps on a synthetic table-counting task and afterwards tested on OOLONG-synth.

Critical strengths
  1. The formulation is tidy and broadly applicable. A single interface handles history replacement as well as delegation, and the training requirement is formulated without reference to a particular RL estimator, so the peak memory during training is set by the largest individual model call instead of by the entire execution (Section 4).
Critical weaknesses
  1. The work offers little novelty, since the training construction already exists and the new part, putting compaction and delegation in one process, is not shown to matter. The paper itself lists MemAct, MemSearcher, Context-Folding and CompactionRL as training policies under changing contexts, and RAO and RLM as training delegation from task outcomes (Section 3). Closer works go uncited. MemAgent (arXiv 2507.02259) reads a long input chunk by chunk in a small window with an overwritten memory, the role Compact serves here, and applies RL over independent contexts. ReSum (arXiv 2509.13313) propagates the final advantage over segmented trajectories as in Section 4.1. The experiments do not test the two meta-tools together, since the counting task is solvable without compaction (Appendix C) and the parallel arm nearly stops calling Compact (Figure 1).

  2. There is no evidence that the agent learns to reason about its context capacity, the problem the abstract sets out. All counting files have one token length just above the window (Appendix A), the trained policy is not tested on other sizes, and OOLONG success drops to 7.8% at eight times the window and to zero at 32 times (Section 5.4). Real long-horizon agent tasks with unknown input size are not tested.

  3. The appendices do not back the abstract's credit of parallel task decomposition to a reward for parallel savings. Parallel fan-out shows up only with a rewritten prompt that requests less work (Appendix D.2), and with the standard prompt and the same α, parallel episodes fall to zero by step 8 (Appendix F.1). The two rewards also differ in the margin between success and failure, a confound the paper admits (Appendix E).

  4. The untrained policy is the sole baseline, so what RL and policy-chosen timing contribute is unclear. The paper distinguishes itself from CompactionRL by this timing (Section 3) but does not compare with threshold-triggered compaction on the same model, a fixed chunked map-reduce over subagents, training with one meta-tool, or an agent in the style of MemAgent or RLM.

Questions for authors
  1. Could the authors list each counting run with its reward, prompt, seed and final success, and specify which run Figure 1 shows? The paper variously calls these experiments three models, two seeds each, four runs, and one run each (Sections 1 and 5.3). The two task-only seeds are 43 points apart at step 10 (Appendix F), and Figure 1 matches the stronger seed of each reward (Table 4).

  2. Is the OOLONG gain due to table-counting training, or to any starting policy that sometimes succeeds on over-window documents? The base model solves no over-window question before OOLONG training (Table 1), so under group-centred advantages it gets no reward signal there, unlike the table-counting policy. Each initialisation has one run, and the table-counting run restarts the optimiser (Section 5.4). A short-to-long curriculum on OOLONG can separate the two explanations.

4 comments

Thank you for the detailed review and constructive suggestions. We have uploaded a revised manuscript addressing your comments, with changes highlighted in blue. We respond to each point below.

New experiments. We completed 42 table-counting runs with five combinations of meta-tools and automatic compaction: automatic compaction alone; Delegate; Delegate with automatic compaction; Compact and Delegate; and all three. We use three seeds per tested reward/combination. Rewards are outcome-only and cost-based with α = 1 or 0.5; automatic compaction alone omits α = 0.5 because it equals α = 1 without subagents.

Novelty and joint tool use. Prior work establishes training across changing contexts and delegation. CARP provides a common execution model and training representation covering history replacement, subagent execution, and their compositions. Compact and Delegate are minimal examples of these mechanisms. We added MemAgent and ReSum, and Section 4 now explains how memory updates, runtime-triggered summaries, and branch/return workflows fit within CARP. Our new agents compose the tools: delegated subagents call Compact in 29% of held-out episodes, or 63% with automatic compaction, and make 95–99% of all Compact calls (Table 10).

Context capacity and document length. Counting files now vary from 1–2× the 4,096-token window; evaluated OOLONG documents span 1-8×. Counting questions reveal neither row count nor file length. With Delegate and the outcome reward, success is 90% at 1–1.5× and 87% at 1.5–2×, against 9% and 7% before training (Table 7). For Delegate and Compact + Delegate with α = 0.5, three completed OOLONG-training seeds each give 64–65% success at 1–4× and 37–38% at 4–8×, against less than 1% before training (Table 8).

We also measure explicit context-related reasoning accompanied by a meta-tool call. An untrained Qwen3.8-27B judge labels whether each response discusses remaining capacity (majority of five samples); we check whether that response calls Compact or Delegate. With the outcome reward, episodes containing such a response rise from 28–41% to 98–100%, and success among those episodes rises from 11–34% to 71–89% (Figure 1).

Tasks with unknown input size. All counting-trained runs are evaluated on BrowseComp-Plus: 830 deep-research questions over 100,195 web documents, with no training on this benchmark. At an 8,192-token window, success rises from 0.8–4.2% before training to 4.6–22.9% afterward, across reward/combination means (Table 1).

Reward and prompt confounds. Within each tool combination, rewards share the same prompt, data, seeds, and training settings. The prompt states that tool calls can run concurrently, but does not ask for parallel work or less work. With Delegate, parallel fan-out rises from 22% to 98–99% under every reward (Table 9). We removed the cost-aware prompt arm and the claim that cost rewards cause parallel decomposition. The results show no consistent increase from pricing critical-path cost. The success–failure reward gap still differs, so we do not isolate cost shaping from reward scale.

Baselines and RL. We added trained automatic-compaction and single-meta-tool baselines on the same model and data. Automatic compaction triggers at 75% of the window; we train it alone and with the meta-tools. These are mechanism baselines, not full reproductions of RLM or MemAgent or a fixed map-reduce workflow. Every trained agent is compared with its untrained counterpart using the same tools. With Delegate and the outcome reward, counting success rises from 7.8% to 88.2%. On BrowseComp-Plus at 4,096 tokens, adding automatic compaction raises outcome-reward success from 0.9% to 6.5% with Delegate, and from 1.2% to 4.5% with Compact + Delegate (Table 1).

Run reporting. Table 3 lists all 42 counting runs and their seeds, rewards, tools, and scores. Each is scored at step 20, with no seed or checkpoint selection. Table 1 reports three-seed means and standard deviations; Figure 1 uses outcome-reward runs. Final OOLONG results are currently available for 22 runs; Tables 5–6 mark available seed counts and resumed jobs. The remaining OOLONG results will be included shortly.

OOLONG initialization. Our direct transfer evidence precedes OOLONG training: counting training raises OOLONG success from 0.2% to 21.2% with Delegate under the outcome reward (Table 1). Further OOLONG learning may benefit from these initial successes. We have not compared with a short-to-long OOLONG curriculum and do not claim superiority to it.

We would appreciate your reassessment of the paper in light of these revisions and new experiments.

Thank you for the revision. The complete run table and the removal of the parallel-reward claim address my reporting concerns, so I raise my score. However, the new results weaken the main claims. On BrowseComp-Plus, threshold-triggered compaction without meta-tools gives the best result (22.9%). On table counting, adding Compact to Delegate lowers success from 88–89% to 62–81%. Comparisons with MemAgent/RLM-style agents are still missing. I raised my score to 2.

Thank you for reconsidering the paper and acknowledging the reporting changes. Please can you clarify which main claim you consider weakened by these comparisons?

Our intended claim is that CARP provides a common execution and training framework covering history replacement, delegation, and their compositions. We do not claim that exposing more decisions to the model, or supplying both tools, must outperform prescribed control. Automatic compaction is itself a CARP setting, and the results show that agents using different settings can be trained within the same framework.

The revision already includes representative comparisons of the memory-replacement and delegation mechanisms used by MemAgent and RLM. We use the same model, data, and task tools to isolate execution-management choices. The differences in interfaces and update rules are deliberate controls, as explained below.

  • MemAgent-style memory management: the model learns what information to retain, while the runtime determines when memory is replaced. Our automatic-compaction baseline tests this arrangement using a context threshold rather than replacement after each input chunk.
  • RLM-style delegation: RLM uses a persistent Python REPL through which generated code constructs subtask inputs and invokes submodels. Our Delegate tool exposes subagent creation directly through tool calls, including recursive delegation. We retain Read-based task access across our comparisons to hold computational capabilities fixed. Unrestricted Python could solve some tasks directly, without requiring the model to process the rows in its context.

The revised results support our central claim: automatic compaction, delegation, and their combinations are all CARP settings, and agents improve through training in each. Different settings performing best on different tasks is consistent with the framework’s intended scope. We would appreciate understanding whether your remaining concern is about this distinction in the claims, or about the novelty and usefulness of the common framework itself.

AuthorsOpenReview ↗

More Evidence of Generalised Context Management

Apologies for the relatively late comment, but we have had some more results come in that you might find interesting. The key result is that by training with just delegation, evaluation without delegation and with compaction shows transfer in performance. And training with autocompaction alone (like MemAgent and CompactionRL) shows no transfer to settings without autocompaction. Indicating failure of the model itself to learn context management.

We trained agents in Table Counting tasks with the following set-ups:

  • auto only: No metatools, just autocompaction enabled.
  • Delegate: Only the delegate metatool, no compact or autocompaction.
  • Delegate + auto: Only the delegate metatool, and autocompaction enabled.
  • Compact, Delegate: Both metatools available, no autocompaction.
  • Compact, Delegate + auto: Both metatools available, and autocompaction enabled.

Tables

In the follow tables we show evaluations of agents trained on one set-up (and the untrained model), and tested each other considered set-up, and additionally an agent with only the compact metatool. Note: the untrained model has almost no performance under Compact only, so we did not train as there would be little learning signal.

The first table shows evaluations on held-out Table Counting tasks, the second shows evaluations on OOLONG (for the same models trained on Table Counting).

Trained ↓ / Evaluated →auto onlyDelegateDelegate + autoCompact, DelegateCompact, Delegate + autoCompact only
untrained4.17.811.31.86.70.1
auto only54.00.060.70.069.00.0
Delegate1.888.286.383.783.922.7
Delegate + auto1.087.686.284.885.026.7
Compact, Delegate0.169.370.370.374.116.4
Compact, Delegate + auto6.475.977.981.483.640.0

OOLONG (no OOLONG training)

Trained ↓ / Evaluated →auto onlyDelegateDelegate + autoCompact, DelegateCompact, Delegate + autoCompact only
untrained0.80.21.00.10.90.0
auto only6.00.012.10.021.50.0
Delegate7.321.222.818.521.46.6
Delegate + auto7.621.729.920.229.86.1
Compact, Delegate9.514.818.912.519.98.5
Compact, Delegate + auto14.610.326.39.225.98.9

We bring your attention to the bolded results in both tables:

  • When we train with auto only (in the style of CompactionRL and MemAgent), the agent fails every time auto compaction is disabled, and cannot use the metatools. It has not learned to manage its own context when given the tools to do so.
  • When trained with only the delegate metatool, the agent successfully generalised to using compact only, despite never seeing that tool during training.
Reviewer_CNSKOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces a "Concurrent Agent Rollout Process" (CARP) framework, which models an agent’s execution as a dynamically evolving set of agent instances (or "threads") and defines context management mechanisms as "meta-tools" capable of modifying the execution state: Compaction: Replacing a thread’s accumulated history with a concise summary; Delegation: Spawning a new agent instance with its own independent history, while blocking the caller until the sub-agent returns a result.

The training process preserves the conditional context for every generated token. These recorded contexts are organized into a token tree (a prefix tree), allowing for the calculation and accumulation of log-probability gradients during each model invocation. Consequently, the entire execution process does not need to fit within a single context window; peak memory usage during training depends on the scale of a single model evaluation rather than the total number of tokens. Furthermore, the paper defines token-based computational costs—encompassing both total workload and "critical path" workload—and incorporates these into the reinforcement learning (RL) reward function.

Key Contributions: CARP: A general-purpose process for agent execution that supports a dynamically changing set of instances. Meta-tools: Encapsulating "compaction" and "sub-agent delegation" as operations that modify the execution state. End-to-end gradient-based training via CARP: Eliminates the requirement for the entire execution process to reside within a single context window. Identification and mitigation of obstacles in learning context management through RL (by introducing explicit token counters and guided prompts; otherwise, the initial policy would never invoke meta-tools, nor would it be able to assess remaining context capacity). Introduction of computation-aware rewards: Utilizing total token workload and critical-path costs for evaluation. Demonstration that agents can learn to solve tasks exceeding their context window limits solely through reinforcement learning—without requiring demonstrations of context management strategies—with validation on the OOLONG benchmark.

Verification Results: Table-counting task (handling files that exceed the 4,096-token context window limit): The success rate on the test set improved from approximately 5% (pre-training) to 48%–84% (depending on the specific training run). Different reward conditions drive the agent to adopt distinct strategies: outcome-only rewards—or rewards with α=1\alpha=1—tend to favor "compression" operations, whereas cost-sensitive rewards with α=0.5\alpha=0.5 (which account for critical-path time savings gained through parallelization) encourage the agent to decompose tasks into parallel "sub-agents."

Transfer to OOLONG-synth (a cross-document aggregation benchmark): Without additional training, policies trained on the "counting" task increased the overall success rate from 5.7% to 16.2%, and the success rate for "cross-window" problems from 0% to 8.7%. With further training on OOLONG, models initialized via the "counting" task achieved a 34.3% success rate on "cross-window" problems, compared to 0% for the base model (a difference of Δ=34.3\Delta = 34.3 percentage points; 95% confidence interval: 26.5%–42.5%).

Limitations: Success rates decline as input length increases (dropping to 7.8% at 8 times the context budget, and to 0% at 32 times the budget). Furthermore, the current comparison—based on two random seeds—has yet to definitively confirm the advantages of incorporating "computational cost" into the reward function.

Critical strengths
  1. A principled architecture that decouples execution length from training feasibility The core technical insight is that evaluating the log-probability of a given token requires only the conditioning context present at the time of that token’s generation. This insight enables end-to-end training that spans beyond a single context window. By organizing recorded contexts into "token trees" that share common prefixes, gradient terms can be computed and accumulated in batches; consequently, the entire execution process does not need to be loaded into the memory of an accelerator (such as a GPU) all at once. This is a universal result: the execution topology—how threads are created, merged, or delegated—does not constrain the optimization process. Furthermore, the peak memory footprint during training depends solely on the scale of a single model evaluation, rather than the total number of generated tokens. This characteristic distinguishes CARP from methods tied to specific reinforcement learning (RL) estimators or particular context-editing operations, theoretically supporting training across execution processes of arbitrary length.
  2. Clear empirical validation: Policies can emerge spontaneously from reward signals alone and demonstrate cross-task transferability Experiments show that, without any provided examples of context-management strategies, GRPO can guide an agent to solve tasks where the input length exceeds the context window. In a "table counting" task, the model’s success rate on an unseen test set improved from approximately 5% to 48%–84%. Strong evidence for the framework’s expressive power lies in the fact that different reward criteria lead the model to acquire vastly different strategies—for instance, when the reward mechanism accounts for "shortening the critical path via parallelization" (\alpha=0.5), the model spontaneously evolves a strategy of "decomposing tasks into parallel sub-agents." This achievement extends beyond synthetic tasks: a policy trained on a "counting" task can be transferred to the OOLONG-synth dataset; with further training, its success rate in scenarios exceeding the context window rises from 0% to 34.3% (whereas the base model achieves a 0% success rate in the same scenario). This combination of "strategies that emerge based on outcomes" and "cross-task transfer capability" provides compelling evidence that an agent can genuinely learn to manage context, rather than simply following pre-programmed strategies.
Critical weaknesses
  1. The validation process lacks sufficient statistical rigor, and the claims regarding the benefits of "computation-cost rewards" remain unproven. The paper itself acknowledges that "comparisons based on only two random seeds are insufficient to definitively demonstrate that computation-cost rewards yield actual benefits." The core experiments utilize only two random seeds per condition; furthermore, the critical comparisons on the OOLONG transfer tasks are based on just a single full run for each initialization configuration (with the "counting" task’s initialization run even involving an optimizer restart, which further complicates the attribution of the results). Given that one of the study’s primary contributions is the "computation-aware reward"—and its purported ability to elicit qualitatively distinct strategies (such as parallel task decomposition when \alpha=0.5)—the absence of statistically rigorous comparisons (i.e., the lack of multiple random seeds, unreported variance, and failure to control for confounding factors) means that the findings regarding the reward mechanism design should be viewed merely as "heuristic suggestions" rather than established conclusions.
  2. The core motivation—"training beyond the context window"—is validated only within a very narrow scope, and the learning process relies heavily on extensive, manually constructed scaffolding. While the framework aims to enable "arbitrarily long execution" for agents, the empirical evidence supporting this is weak: the success rate drops to 7.8% when the input length reaches 8 times the context window limit, and falls to 0% at 32 times the limit. Consequently, the claim of "transcending a single context window" is validated only within a marginal range beyond the window limit—and exclusively for a simple, synthetic counting task. The transfer gains on the OOLONG tasks are also quite limited (with zero-shot success rates increasing from 5% to 13%; out of 144 experimental segments that exceeded the context window limit, only 9 were successfully resolved). Moreover, this learning capability does not emerge spontaneously: the initial strategy is entirely incapable of utilizing "meta-tools" or reasoning about remaining capacity. It is only through the provision of explicit, tokenizer-supplied counters and guided prompting that the agent learns to manage its context. This form of scaffolding significantly undermines the claim of "end-to-end learning without demonstrations"—while the agent does indeed learn to use tools, this is contingent upon the paper providing key signals beforehand, which render tool usage discoverable and straightforward to implement. In summary, although the paper proposes a promising training architecture, the evidence remains quite limited regarding its robust scalability, transferability, and whether the "computational cost reward" mechanism truly plays a pivotal role.
Questions for authors

By organizing the context of the sequences into "token trees" that share common prefixes, gradient terms can be computed and accumulated in batches; consequently, the entire execution process does not require the full context to be loaded into the memory of an accelerator (such as a GPU) all at once. The question is: do these "token trees" themselves require additional memory, and what is their memory footprint?

1 comment

Thank you for the careful summary and clear questions. We have uploaded a revised manuscript addressing your comments, with changes highlighted in blue. We respond to each point below.

New experiments since submission. We completed 42 table-counting runs with five combinations of meta-tools and automatic compaction: automatic compaction only; Delegate; Delegate with automatic compaction; Compact and Delegate; and Compact and Delegate with automatic compaction. Each tested reward/tool combination has three seeds. We use an outcome-only reward and two cost rewards, with α = 1 and α = 0.5; automatic compaction alone omits α = 0.5 because it equals α = 1 without subagents. Counting files vary from 1–2× the 4,096-token window, and the evaluated OOLONG documents span 1-8×. We evaluate all 42 counting-trained checkpoints on held-out table counting, OOLONG, and BrowseComp-Plus, which is never trained on. We also report the available final results after further OOLONG training.

The validation process lacks sufficient statistical rigor...

All 42 table-counting runs are complete. Table 1 reports the mean ± standard deviation over three seeds for every evaluated reward/configuration, and Table 3 lists every per-seed result. All counting runs are scored at step 20, with no selection of seeds or checkpoints. Within each tool configuration, the reward conditions share the same system prompt, training data, seeds, and training settings; only the reward differs. Final OOLONG-training results are currently available for 22 of the 42 runs. Tables 5 and 6 report those available results, with reduced seed counts marked and resumed jobs identified. The remaining OOLONG results will be included shortly.

The revision no longer claims that the cost reward produces distinct strategies. Parallel decomposition, where one response starts several sub-agents, emerges under every reward: with Delegate, it occurs in 98–99% of held-out episodes after training under each reward, against 22% before training. The cost reward is one of three reward conditions, and the main results hold under all three.

Range of input sizes. Every counting file and OOLONG document exceeds the context window, and we report success by document length. With Delegate and the outcome reward, counting-trained agents succeed on 90% of tables at 1–1.5× the window and 87% at 1.5–2×, against 9% and 7% for the untrained model (Table 7). With the α = 0.5 cost reward, Delegate and Compact + Delegate both have three completed OOLONG-training seeds. They succeed on 64–65% of OOLONG questions at 1–4× the window and 37–38% at 4–8×, against less than 1% for the corresponding untrained models in every length band (Table 8). The evaluated documents reach 7.3× the window; we do not claim results beyond 8×.

Tasks beyond synthetic counting. We now evaluate on BrowseComp-Plus: 830 deep-research questions answered by searching a corpus of 100,195 web documents, where the agent does not know in advance how much it must read. At an 8,192-token window, agents trained only on table counting reach 4.6–22.9% success, against 0.8–4.2% for the untrained model with the same tools. Table-counting training also transfers to OOLONG before any OOLONG training: success rises from 0.1–1.0% to 6.0–40.6%. These ranges are the reward/configuration means reported in Table 1.

Scaffolding. The counters report resource usage, and the prompt asks the agent to consider its remaining capacity. Neither supplies demonstration trajectories, a fixed compaction schedule, a task decomposition, or instructions about which information to retain. These provisions are identical before and after training, so the comparison measures what RL adds beyond the supplied guidance. Identifying obstacles in the initial policy and interventions that support learning is part of the empirical contribution. We also measure explicit context-related reasoning accompanied by a meta-tool call: with the outcome reward, the share of episodes containing such a response rises from 28–41% before training to 98–100% afterward (Figure 1). With Delegate, task success per attempt rises from 7.8% to 88.2% under that reward (Table 1).

Do these “token trees” themselves require additional memory, and what is their memory footprint?

The token tree is stored in host (CPU) memory, never on the GPU. It holds one node per distinct token prefix, so shared prefixes are stored once. In the two example training steps reported in Section 4, an episode’s tree has a median of 14,412 and 27,990 nodes, against 35,422 and 84,519 tokens summed over its model calls. All trees for one step (192 episodes) occupy 339 and 672 MiB of host memory, respectively. For training, the tree is expanded into sequences no longer than the 4,096-token window and packed into micro-batches of at most 4,096 tokens, so GPU memory is bounded by one micro-batch, whatever the length of the episode.

Reviewer_wFaVOpenReview ↗
Rating
2
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The work proposes CARP that treats "compact" and "delegate" as tools the agent can call it self and run st he agent as a changing set of threads. The work used GRPO to train the whole execution.

Critical strengths

The idea is clean and interesting. It treats compaction and delegation as operations on a set of threads.

Critical weaknesses
  1. I think the idea here is covered in a number of related studies that treat context managements as a step of the model and use RL to train. It will help the work if it can better clarify its novelty.
  2. As a followup I think the work will benefit from some comparison with existing baselines.
Questions for authors

None

1 comment

We thank the reviewer for finding the idea clean and interesting. We have revised the manuscript in response to the reviewers’ comments. Changes are highlighted in blue.

New experiments since submission. We completed 42 table-counting runs across five combinations of meta-tools and automatic compaction: automatic compaction only; Delegate tool only; Delegate tool with automatic compaction; Compact and Delegate tools; and Compact and Delegate tools with automatic compaction. Each reward/configuration has three seeds. We use an outcome-only reward and two cost rewards, with α = 1 and α = 0.5; automatic compaction alone omits α = 0.5 because it equals α = 1 without subagents. Counting files vary from 1–2× the 4,096-token window, and the evaluated OOLONG documents span 1–8×. We evaluate all 42 counting-trained checkpoints on held-out table counting, OOLONG, and BrowseComp-Plus. We also report the available final results after further OOLONG training. The remaining OOLONG results will be included shortly.

We address each of your comments below, making reference to the revised manuscript and the new results.

I think the idea here is covered in a number of related studies that treat context managements as a step of the model and use RL to train. It will help the work if it can better clarify its novelty.

Prior work establishes that context management and delegation can be learned. CARP provides a common execution model and training representation covering history replacement, subagent execution, and their compositions. Compact and Delegate are minimal examples of these mechanisms. The revised Section 4 explains how memory updates, runtime-triggered summaries, and branch/return workflows fit within CARP. The new experiments compare policy-chosen and automatic compaction under matched training conditions, and show the mechanisms being composed in practice: delegated subagents compact their own histories in 29% of held-out episodes, rising to 63% with automatic compaction. Subagents make 95–99% of all Compact calls (Table 10). The revised related-work section discusses the relevant precedents, including MemAgent and ReSum.

As a followup I think the work will benefit from some comparison with existing baselines.

We added trained automatic-compaction and single-meta-tool baselines, using the same model, data, and training settings:

  • Automatic threshold-triggered compaction: the agent writes a summary when its context reaches 75% of the window, and continues from it.
  • Delegate alone: the agent hands parts of the task to subagents.
  • Combinations with automatic compaction and the model-selected Compact tool.

Every configuration is also compared with the untrained model using the same tools. Averaged over reward conditions and seeds, table-counting success per attempt rises from 7.8% to 88.5% with Delegate, and from 11.3% to 87.9% with Delegate and automatic compaction. Trained agents also improve on OOLONG and BrowseComp-Plus (Table 1).

Given the new baselines, repeated runs, and broader evaluations, we would appreciate your reassessment of the paper in light of this evidence.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.