- Rating
- 2(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2(+1)
Read review
Paper summary
The paper introduces CARP, an execution model that treats an agent as a collection of threads and offers context management through two meta-tools, Compact, which swaps a thread's history for a summary, and Delegate, which launches a subagent. During training the input and output of every model call are stored, so GRPO can assign the episode advantage to each sampled token without the full execution having to fit in a single window. Under a 4,096-token window, Qwen3.8-27B is trained for at most ten steps on a synthetic table-counting task and afterwards tested on OOLONG-synth.
Critical strengths
- The formulation is tidy and broadly applicable. A single interface handles history replacement as well as delegation, and the training requirement is formulated without reference to a particular RL estimator, so the peak memory during training is set by the largest individual model call instead of by the entire execution (Section 4).
Critical weaknesses
-
The work offers little novelty, since the training construction already exists and the new part, putting compaction and delegation in one process, is not shown to matter. The paper itself lists MemAct, MemSearcher, Context-Folding and CompactionRL as training policies under changing contexts, and RAO and RLM as training delegation from task outcomes (Section 3). Closer works go uncited. MemAgent (arXiv 2507.02259) reads a long input chunk by chunk in a small window with an overwritten memory, the role Compact serves here, and applies RL over independent contexts. ReSum (arXiv 2509.13313) propagates the final advantage over segmented trajectories as in Section 4.1. The experiments do not test the two meta-tools together, since the counting task is solvable without compaction (Appendix C) and the parallel arm nearly stops calling Compact (Figure 1).
-
There is no evidence that the agent learns to reason about its context capacity, the problem the abstract sets out. All counting files have one token length just above the window (Appendix A), the trained policy is not tested on other sizes, and OOLONG success drops to 7.8% at eight times the window and to zero at 32 times (Section 5.4). Real long-horizon agent tasks with unknown input size are not tested.
-
The appendices do not back the abstract's credit of parallel task decomposition to a reward for parallel savings. Parallel fan-out shows up only with a rewritten prompt that requests less work (Appendix D.2), and with the standard prompt and the same α, parallel episodes fall to zero by step 8 (Appendix F.1). The two rewards also differ in the margin between success and failure, a confound the paper admits (Appendix E).
-
The untrained policy is the sole baseline, so what RL and policy-chosen timing contribute is unclear. The paper distinguishes itself from CompactionRL by this timing (Section 3) but does not compare with threshold-triggered compaction on the same model, a fixed chunked map-reduce over subagents, training with one meta-tool, or an agent in the style of MemAgent or RLM.
Questions for authors
-
Could the authors list each counting run with its reward, prompt, seed and final success, and specify which run Figure 1 shows? The paper variously calls these experiments three models, two seeds each, four runs, and one run each (Sections 1 and 5.3). The two task-only seeds are 43 points apart at step 10 (Appendix F), and Figure 1 matches the stronger seed of each reward (Table 4).
-
Is the OOLONG gain due to table-counting training, or to any starting policy that sometimes succeeds on over-window documents? The base model solves no over-window question before OOLONG training (Table 1), so under group-centred advantages it gets no reward signal there, unlike the table-counting policy. Each initialisation has one run, and the table-counting run restarts the optimiser (Section 5.4). A short-to-long curriculum on OOLONG can separate the two explanations.
4 comments
Thank you for the revision. The complete run table and the removal of the parallel-reward claim address my reporting concerns, so I raise my score. However, the new results weaken the main claims. On BrowseComp-Plus, threshold-triggered compaction without meta-tools gives the best result (22.9%). On table counting, adding Compact to Delegate lowers success from 88–89% to 62–81%. Comparisons with MemAgent/RLM-style agents are still missing. I raised my score to 2.
Thank you for reconsidering the paper and acknowledging the reporting changes. Please can you clarify which main claim you consider weakened by these comparisons?
Our intended claim is that CARP provides a common execution and training framework covering history replacement, delegation, and their compositions. We do not claim that exposing more decisions to the model, or supplying both tools, must outperform prescribed control. Automatic compaction is itself a CARP setting, and the results show that agents using different settings can be trained within the same framework.
The revision already includes representative comparisons of the memory-replacement and delegation mechanisms used by MemAgent and RLM. We use the same model, data, and task tools to isolate execution-management choices. The differences in interfaces and update rules are deliberate controls, as explained below.
- MemAgent-style memory management: the model learns what information to retain, while the runtime determines when memory is replaced. Our automatic-compaction baseline tests this arrangement using a context threshold rather than replacement after each input chunk.
- RLM-style delegation: RLM uses a persistent Python REPL through which generated code constructs subtask inputs and invokes submodels. Our Delegate tool exposes subagent creation directly through tool calls, including recursive delegation. We retain Read-based task access across our comparisons to hold computational capabilities fixed. Unrestricted Python could solve some tasks directly, without requiring the model to process the rows in its context.
The revised results support our central claim: automatic compaction, delegation, and their combinations are all CARP settings, and agents improve through training in each. Different settings performing best on different tasks is consistent with the framework’s intended scope. We would appreciate understanding whether your remaining concern is about this distinction in the claims, or about the novelty and usefulness of the common framework itself.
More Evidence of Generalised Context Management
Apologies for the relatively late comment, but we have had some more results come in that you might find interesting. The key result is that by training with just delegation, evaluation without delegation and with compaction shows transfer in performance. And training with autocompaction alone (like MemAgent and CompactionRL) shows no transfer to settings without autocompaction. Indicating failure of the model itself to learn context management.
We trained agents in Table Counting tasks with the following set-ups:
- auto only: No metatools, just autocompaction enabled.
- Delegate: Only the
delegatemetatool, nocompactor autocompaction. - Delegate + auto: Only the
delegatemetatool, and autocompaction enabled. - Compact, Delegate: Both metatools available, no autocompaction.
- Compact, Delegate + auto: Both metatools available, and autocompaction enabled.
Tables
In the follow tables we show evaluations of agents trained on one set-up (and the untrained model), and tested each other considered set-up, and additionally an agent with only the compact metatool. Note: the untrained model has almost no performance under Compact only, so we did not train as there would be little learning signal.
The first table shows evaluations on held-out Table Counting tasks, the second shows evaluations on OOLONG (for the same models trained on Table Counting).
| Trained ↓ / Evaluated → | auto only | Delegate | Delegate + auto | Compact, Delegate | Compact, Delegate + auto | Compact only |
|---|---|---|---|---|---|---|
| untrained | 4.1 | 7.8 | 11.3 | 1.8 | 6.7 | 0.1 |
| auto only | 54.0 | 0.0 | 60.7 | 0.0 | 69.0 | 0.0 |
| Delegate | 1.8 | 88.2 | 86.3 | 83.7 | 83.9 | 22.7 |
| Delegate + auto | 1.0 | 87.6 | 86.2 | 84.8 | 85.0 | 26.7 |
| Compact, Delegate | 0.1 | 69.3 | 70.3 | 70.3 | 74.1 | 16.4 |
| Compact, Delegate + auto | 6.4 | 75.9 | 77.9 | 81.4 | 83.6 | 40.0 |
OOLONG (no OOLONG training)
| Trained ↓ / Evaluated → | auto only | Delegate | Delegate + auto | Compact, Delegate | Compact, Delegate + auto | Compact only |
|---|---|---|---|---|---|---|
| untrained | 0.8 | 0.2 | 1.0 | 0.1 | 0.9 | 0.0 |
| auto only | 6.0 | 0.0 | 12.1 | 0.0 | 21.5 | 0.0 |
| Delegate | 7.3 | 21.2 | 22.8 | 18.5 | 21.4 | 6.6 |
| Delegate + auto | 7.6 | 21.7 | 29.9 | 20.2 | 29.8 | 6.1 |
| Compact, Delegate | 9.5 | 14.8 | 18.9 | 12.5 | 19.9 | 8.5 |
| Compact, Delegate + auto | 14.6 | 10.3 | 26.3 | 9.2 | 25.9 | 8.9 |
We bring your attention to the bolded results in both tables:
- When we train with auto only (in the style of CompactionRL and MemAgent), the agent fails every time auto compaction is disabled, and cannot use the metatools. It has not learned to manage its own context when given the tools to do so.
- When trained with only the
delegatemetatool, the agent successfully generalised to usingcompactonly, despite never seeing that tool during training.
Thank you for the detailed review and constructive suggestions. We have uploaded a revised manuscript addressing your comments, with changes highlighted in blue. We respond to each point below.
New experiments. We completed 42 table-counting runs with five combinations of meta-tools and automatic compaction: automatic compaction alone; Delegate; Delegate with automatic compaction; Compact and Delegate; and all three. We use three seeds per tested reward/combination. Rewards are outcome-only and cost-based with α = 1 or 0.5; automatic compaction alone omits α = 0.5 because it equals α = 1 without subagents.
Novelty and joint tool use. Prior work establishes training across changing contexts and delegation. CARP provides a common execution model and training representation covering history replacement, subagent execution, and their compositions. Compact and Delegate are minimal examples of these mechanisms. We added MemAgent and ReSum, and Section 4 now explains how memory updates, runtime-triggered summaries, and branch/return workflows fit within CARP. Our new agents compose the tools: delegated subagents call Compact in 29% of held-out episodes, or 63% with automatic compaction, and make 95–99% of all Compact calls (Table 10).
Context capacity and document length. Counting files now vary from 1–2× the 4,096-token window; evaluated OOLONG documents span 1-8×. Counting questions reveal neither row count nor file length. With Delegate and the outcome reward, success is 90% at 1–1.5× and 87% at 1.5–2×, against 9% and 7% before training (Table 7). For Delegate and Compact + Delegate with α = 0.5, three completed OOLONG-training seeds each give 64–65% success at 1–4× and 37–38% at 4–8×, against less than 1% before training (Table 8).
We also measure explicit context-related reasoning accompanied by a meta-tool call. An untrained Qwen3.8-27B judge labels whether each response discusses remaining capacity (majority of five samples); we check whether that response calls Compact or Delegate. With the outcome reward, episodes containing such a response rise from 28–41% to 98–100%, and success among those episodes rises from 11–34% to 71–89% (Figure 1).
Tasks with unknown input size. All counting-trained runs are evaluated on BrowseComp-Plus: 830 deep-research questions over 100,195 web documents, with no training on this benchmark. At an 8,192-token window, success rises from 0.8–4.2% before training to 4.6–22.9% afterward, across reward/combination means (Table 1).
Reward and prompt confounds. Within each tool combination, rewards share the same prompt, data, seeds, and training settings. The prompt states that tool calls can run concurrently, but does not ask for parallel work or less work. With Delegate, parallel fan-out rises from 22% to 98–99% under every reward (Table 9). We removed the cost-aware prompt arm and the claim that cost rewards cause parallel decomposition. The results show no consistent increase from pricing critical-path cost. The success–failure reward gap still differs, so we do not isolate cost shaping from reward scale.
Baselines and RL. We added trained automatic-compaction and single-meta-tool baselines on the same model and data. Automatic compaction triggers at 75% of the window; we train it alone and with the meta-tools. These are mechanism baselines, not full reproductions of RLM or MemAgent or a fixed map-reduce workflow. Every trained agent is compared with its untrained counterpart using the same tools. With Delegate and the outcome reward, counting success rises from 7.8% to 88.2%. On BrowseComp-Plus at 4,096 tokens, adding automatic compaction raises outcome-reward success from 0.9% to 6.5% with Delegate, and from 1.2% to 4.5% with Compact + Delegate (Table 1).
Run reporting. Table 3 lists all 42 counting runs and their seeds, rewards, tools, and scores. Each is scored at step 20, with no seed or checkpoint selection. Table 1 reports three-seed means and standard deviations; Figure 1 uses outcome-reward runs. Final OOLONG results are currently available for 22 runs; Tables 5–6 mark available seed counts and resumed jobs. The remaining OOLONG results will be included shortly.
OOLONG initialization. Our direct transfer evidence precedes OOLONG training: counting training raises OOLONG success from 0.2% to 21.2% with Delegate under the outcome reward (Table 1). Further OOLONG learning may benefit from these initial successes. We have not compared with a short-to-long OOLONG curriculum and do not claim superiority to it.
We would appreciate your reassessment of the paper in light of these revisions and new experiments.