acceptodds
Under review as a conference paper at ICLR 2027

TORCH: Jointly Evolving Task Execution and Online Supervision

Abstract

Language-model agents increasingly perform complex tasks through sustained interactions with tools and environments. During execution, agents may repeat ineffective actions, overlook contradictory evidence, or terminate before adequately verifying results. Existing methods primarily use post-hoc reflection to improve subsequent attempts or online supervision to correct ongoing execution, with some further retaining skills and memory for experience reuse. However, task execution and supervisory assessment involve distinct decisions, and how to accumulate their respective experience from the same interaction so that both roles can continually improve remains an open question. In this paper, we propose TORCH, a framework that corrects execution trajectories through online supervision and supports separate experience accumulation for the worker and supervisor. Its core idea is to maintain independent long-term memories for the two roles, enabling the worker to learn task-solving knowledge and the supervisor to learn diagnostic and guidance strategies, so that both roles can continually evolve while model parameters remain fixed. Further, we introduce TORCH-GA, a harness built on GenericAgent that fully implements the TORCH framework. We evaluate runtime supervision on external harnesses and memory evolution and transfer in TORCH-GA. Empirical results demonstrate that TORCH improves task performance, that joint evolution achieves higher mean later-round performance than either one-sided variant across three independent ten-round runs per condition, and that accumulated experience remains useful across models and task domains without further memory updates. Our code is available at https://anonymous.4open.science/r/TORCH-D8FF/.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM agents, coding agents, runtime supervision, trajectory correction, self-evolving agents, agent harnesses, agent memory, long-horizon agents

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 4 reviews, above 68% of the others.
Reviewer_kb63OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper proposes TORCH, a framework that pairs an LLM worker with a same-model runtime supervisor and maintains separate long-term memories for both the worker and the supervisor. During execution, the supervisor periodically inspects the worker's trajectory and can inject guidance. After each task is evaluated, both roles reflect on their own interaction records using the shared reward, and the proposed memory updates are reviewed and merged.

Critical strengths
  1. The idea is clear. The reward of a task has different meaning for the supervisor and the worker, the worker needs to learn how to complete tasks and verify the rewards, and the superviser needs to learn when to interfere and under what condition should it interfere.

  2. The held-out evaluation results are promising. After freezing the memory evolved on Terminal-Bench and transferring it to FeatureBench, the pass rate increased from 13.0% to 71.0%. This is the most convincing evidence in the paper.

  3. The paper is honest in giving failed cases and analyzing how the supervisor can steer the worker wrong.

Critical weaknesses
  1. The headline results in Table 1 are obtained after evolving memory on the evaluation tasks. TORCH-GA's worker and supervisor memories are evolved with reward feedback for 20 rounds on all Terminal-Bench 2.1 tasks and for 5 rounds on all SWE-bench Verified tasks, and are then evaluated on the same tasks. I think this is a serious problem, because this means that the scores do not measure generalization. A held-out split evaluation might be needed.

  2. The comparison with existing baselines is not controlled. The three baselines differ from TORCH in harness (SWE-agent, mini-swe-agent, the authors' own systems) and in supervisor model (Claude Sonnet 4.6, a trained 4B critic, GPT-5.3-Codex). Their supervisor-off scores on Terminal-Bench are far below TORCH's supervisor-off scores with the same Qwen3.8-27B worker, which is a sign of substantial differences in setup. In addition, only TORCH's supervisor memory has been tuned on the evaluation tasks.

  3. Results rely on single runs and do not control for compute. Tables 1 and 2 only report one execution per task per configuration. The paper's own evolution study shows substantial run-to-run variation, so results relying on single runs might be a problem. Supervision also adds model calls, and in several configurations it increases total token usage (Table 5). However, there is no equal-budget baseline. It is unclear how much of the gain comes from supervision rather than variance or extra computation.

  4. I think the novelty of this paper is relatively limited, because these ideas of evolving context as well as superviser exists, and the merge is mainly implemented by prompts. This paper is more an engineering contribution combining ideas together.

Questions for authors
  1. Can the authors report Table 1 results on tasks the memories were never evolved on?
  2. For the external harnesses in Table 1, what are the results when the supervisor uses its initial memory template rather than the evolved one?
  3. For the baselines, can SupervisorAgent, LivePlan, or Steer be attached to the same worker harness and model used for TORCH, with TORCH's supervisor using its initial memory?
  4. Can the authors report repeated runs for the main comparisons, together with an equal-budget baseline such as best-of-2 sampling?
4 comments
AuthorsOpenReview ↗

Response to Reviewer kb63 (1/2)

We thank the reviewer for the detailed review, and we are glad the reviewer found the frozen-memory transfer to FeatureBench (13.04% → 71.01%) the most convincing evidence. The new held-out evaluation extends exactly this kind of evidence to Terminal-Bench 2.1 and SWE-bench Verified.

Weakness 1 / Question 1: "The headline results in Table 1 are obtained after evolving memory on the evaluation tasks." "Can the authors report Table 1 results on tasks the memories were never evolved on?"

Conclusion: yes; the gains hold on tasks the memories never saw.

A fixed random seed splits each benchmark into an evolution split and a disjoint held-out split (45 / 44 tasks on Terminal-Bench 2.1, 250 / 250 on SWE-bench Verified). Memories are evolved only on the evolution split, frozen, and evaluated on the held-out split, three runs each, with the same configuration grid as Table 1.

  • Both evolved memories vs. the initial worker without supervision: 56.82% → 70.45% on Terminal-Bench (+13.6), 69.87% → 87.47% on SWE-bench Verified (+17.6).
  • Evolved supervisor memory on top of evolved worker memory: +7.6 and +12.1 points.
  • Frozen supervisor memory on external harnesses: Claude Code 54.55% → 67.42% (+12.9), mini-SWE-agent 76.53% → 85.07% (+8.5).

Question 2: "For the external harnesses in Table 1, what are the results when the supervisor uses its initial memory template rather than the evolved one?"

Conclusion: the initial supervisor helps only modestly; the large gains come from evolved supervisory experience.

Table 1 now reports this setting for every harness. With its initial memory, the supervisor improves performance by 1.5–6.4 points on Terminal-Bench and 0.9–1.3 points on SWE-bench Verified; with evolved memory, by 11.2–17.6 and 3.5–8.1 points. For example, Claude Code on Terminal-Bench: 61.80% without supervision, 64.42% with the initial supervisor, 79.40% with evolved supervisor memory. On held-out tasks the same holds: 61.36% with the initial supervisor, 67.42% with evolved supervisor memory.

Weakness 2 / Question 3: "The comparison with existing baselines is not controlled." "For the baselines, can SupervisorAgent, LivePlan, or Steer be attached to the same worker harness and model used for TORCH, with TORCH's supervisor using its initial memory?"

Conclusion: cross-system comparability and the effectiveness of evolved supervisor memory within a fixed worker setup are distinct questions. Our evidence for the latter is directly controlled.

(a) Cross-system comparisons. We have not transplanted all baseline supervisors into a common harness. Their supervision logic is tightly coupled with their native implementations: SupervisorAgent in a modified smolagents, Steer in the authors' modified mini-swe-agent 1.11.1, and LivePlan in SWE-agent 1.1.0. Transplanting them would require implementation changes that could alter their behavior and introduce confounds. We therefore follow the original papers and released code, preserving each method's native supervision logic and using its recommended or best-performing available configuration: Steer's best-performing released critic and GPT-5.3-Codex as LivePlan's advisor, since the original GPT-5.2-Codex was discontinued. All systems use the same worker model. We report each baseline together with its own supervisor-off/on comparison, while recognizing that cross-system scores also reflect harness differences.

(b) Controlled within-harness comparisons. Within each harness, we fix the worker, model, and tasks and compare supervision Off / Initial / Evolved, on both in-distribution and disjoint held-out tasks. The Initial column explicitly tests TORCH without evolved supervisory experience, and evolved supervisor memory improves over Initial in every harness. These comparisons directly establish the benefit of evolving supervisor memory for a fixed worker setup. Differences between baseline harnesses do not invalidate this within-harness evidence.

AuthorsOpenReview ↗

Response to Reviewer kb63 (2/2)

Weakness 3 / Question 4: "Results rely on single runs and do not control for compute." "Can the authors report repeated runs for the main comparisons, together with an equal-budget baseline such as best-of-2 sampling?"

Conclusion: all main results are now averaged over three runs, and at an equal budget the supervisor still wins clearly.

  • Repeated runs: all main results are mean ± SD over three runs, including the original run; the conclusions are unchanged.

  • Equal budget: under an identical per-task token cap, the independent supervisor reaches 61.36% and self-review in the worker's own context 52.27%, a gap of 9.09 points, at 31.9M vs. 31.6M tokens (44 held-out Terminal-Bench tasks).

  • Without a cap, total tokens with supervision on are 0.72–1.34× those with supervision off (single run), and fewer for Terminus-2 and TORCH-GA. Best-of-2 costs about 2× and needs a selector to pick one of the two attempts; using the hidden tests as that selector would leak test information, which our setting does not allow. The matched-budget self-review control is the equal-budget comparison we report.

Weakness 4: "I think the novelty of this paper is relatively limited ... This paper is more an engineering contribution combining ideas together."

Conclusion: the revision establishes findings that do not follow from simply combining supervision and memory.

(i) How memory is organized matters: putting both roles' experience into one memory degrades performance to 41.00%, versus 64.33% for separate, jointly evolved memories.

(ii) Supervisory experience improves a fixed worker on its own: Claude Code on held-out tasks goes from 61.36% to 67.42% with the worker unchanged, and annotated beneficial interventions rise from 32% to 49%.

(iii) An independent context matters: at the same token budget, the supervisor outperforms self-review by 9.09 points.

(iv) The experience transfers frozen across harnesses, held-out tasks, model families, and benchmarks (e.g., FeatureBench 13.04% → 71.01%).

The reviewer's main concerns, evaluation on the evolution tasks, single runs, and compute, are now answered by the held-out evaluation, three-run averages, and the matched-budget control, and our conclusions rest on fully controlled within-harness comparisons. We hope the reviewer will reconsider the evaluation, in particular the soundness score.

AuthorsOpenReview ↗

Follow-up on our response

Hi Reviewer kb63, as the discussion window closes in about two days, we wanted to check in on our response. We included held-out test evaluations and repeated-run results, and clarified the distinction between cross-system comparisons and supervisor memory gains under a fixed worker. Please let us know if any soundness concerns remain so we can discuss them before the deadline.

Thanks, my concerns are addressed, I decide to maintain my score.

Reviewer_YzkKOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

TORCH combines online supervision with separate long-term memories for a task-executing worker and a supervisor, allowing both roles to accumulate experience while keeping model parameters fixed. The paper evaluates runtime supervision across agent harnesses, compares joint memory evolution with one-sided variants, and studies frozen-memory transfer across models and task domains.

Critical strengths

The paper separates execution experience from supervisory experience and tests this distinction through joint-versus-one-sided evolution and memory-transfer experiments. This makes the proposed framework more compelling than an end-to-end performance comparison alone.

Critical weaknesses

The paper’s central claim depends on the supervisor learning to make better intervention decisions and provide more useful guidance. However, the evolution experiments report task success and task transitions, not whether supervisory decisions become more accurate over rounds. End-to-end gains cannot establish that the supervisor itself improves: they may also reflect worker-memory changes or execution variability. Relatedly, the paper does not systematically assess how incorrect or noisy supervisory guidance affects performance or subsequent memory updates. This matters because such errors could be reinforced during joint evolution.

Questions for authors
  1. Can you evaluate how the supervisor’s intervention decisions and guidance quality change across evolution rounds—for example, using annotated review points or a controlled comparison of supervisor-memory snapshots with the worker memory held fixed?
  2. How robust is joint evolution to incorrect or noisy supervisory guidance? If a controlled noise experiment is infeasible, could you at least quantify harmful interventions and explain whether they enter subsequent supervisor memory?
3 comments
AuthorsOpenReview ↗

Response to Reviewer YzkK

We thank the reviewer for focusing on the central claim of the paper. Our new evidence shows directly that the supervisor itself improves: with the worker held fixed, evolving only the supervisor's memory raises performance, and human annotation shows its guidance becoming more useful over rounds while harmful interventions disappear.

Concern: "End-to-end gains cannot establish that the supervisor itself improves: they may also reflect worker-memory changes or execution variability."

Question 1: "Can you evaluate how the supervisor's intervention decisions and guidance quality change across evolution rounds—for example, using annotated review points or a controlled comparison of supervisor-memory snapshots with the worker memory held fixed?"

Conclusion: the supervisor itself improves; neither worker-memory changes nor execution variability can explain the gain. We provide both kinds of evidence the reviewer suggests.

(a) Controlled comparisons with the worker held fixed. The cleanest case is Claude Code on held-out Terminal-Bench tasks: the worker is unchanged and never evolved, and the tasks were never used for evolution. Success is 54.55% without supervision, 61.36% with the initial supervisor, and 67.42% with evolved supervisor memory; only the supervisor's memory differs between the last two. The same pattern holds everywhere only the supervisor changes:

  • External harnesses: the initial supervisor improves performance by 1.5–6.4 points on Terminal-Bench and 0.9–1.3 points on SWE-bench Verified; evolved supervisor memory by 11.2–17.6 and 3.5–8.1 points.
  • TORCH-GA on Terminal-Bench with evolved worker memory held fixed: 74.53% without supervision, 77.53% with the initial supervisor, 85.39% with evolved supervisor memory.
  • Supervisor-only evolution on 40 Terminal-Bench tasks, where the worker memory is frozen and only the supervisor evolves: from about 45% in round 1 to 55.17% averaged over the last five rounds.

All results are averaged over three runs, which addresses execution variability.

(b) Human annotation of guidance quality. We randomly sampled 100 supervisor interventions actually injected into the worker from early rounds (1–3) and 100 from late rounds (8–10). Two human experts labeled each as beneficial, neutral, or harmful, independently and without knowing the round; a third expert resolved disagreements, and every harmful label was checked against the trajectory. Raw agreement between the two experts was 69% (Cohen's κ = 0.48).

  • Early rounds: 32 beneficial, 64 neutral, 4 harmful.
  • Late rounds: 49 beneficial, 51 neutral, 0 harmful.

Beneficial interventions increase from 32% to 49% (Fisher's exact test, p = 0.02).

Question 2: "How robust is joint evolution to incorrect or noisy supervisory guidance? If a controlled noise experiment is infeasible, could you at least quantify harmful interventions and explain whether they enter subsequent supervisor memory?"

Conclusion: harmful guidance is rare, disappears in late rounds, and in the cases we traced it entered memory only as a lesson about what to avoid.

  • Harmful interventions: 4% in early rounds, 0% in late rounds.
  • Of the four early harmful interventions, two can be traced to candidate memory updates, which yield four entries after deduplication; the other two left no traceable update. All four entries record the error as a correction to avoid, and the merge kept them in that form.

This follows from the merge design: the merge step separates observed facts from unverified suggestions and rewrites or rejects suggestions that the task evidence does not support.

Both of the reviewer's questions, whether the supervisor itself improves and whether harmful guidance is reinforced, are now answered with direct evidence. We hope this addresses the reviewer's central concern, and we would be grateful if the reviewer would reconsider the evaluation in light of it.

Thank you for your response, I will raise my score.

Thank you for taking the time to read our response and reconsider your assessment. Your questions helped us strengthen the paper.

Reviewer_HhN1OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes TORCH, an approach to harness optimization. The key idea is to separate the task execution from supervision by having two instances of the same models running with two separate contexts: worker and supervisor. The worker focuses on performing the task while the supervisor checks its progress and provides corrective guidance --- essentially acting like a reward or critique model in the execution loop. What's interesting is that both the worker and the supervisor use the same frozen LLM, thus "learning" takes place only through the evolving memories each model keeps. This is a simple idea that shows empirical gains over various competitive baselines and recent approaches.

Critical strengths

The proposed approach is simple and effective. The idea of separating out the supervisor from the worker makes a lot of sense, though perhaps not strikingly novel, in that the supervisor seems analogous to a critique model.

The empirical results look promising, and the authors provide good analysis, including when the supervisor misses errors.

Critical weaknesses

I wonder if the authors could perform a bit more thorough ablation study to parse out where the gain comes from.

-Separate memories vs a shared memory? What if we give both roles access to one consolidated memory, matching total memory capacity and adaptation budget. Without this comparison, the paper does not establish that separating the two memory slots is beneficial.

-Compute-matched alternatives? The separate supervisor uses extra compute and time, thus it would be good to investigate additional worker reasoning via self-reviews and self-verifications.

-Generalization across other benchmarks? It would be interesting to test whether the memories of the worker or the supervisor can generalize to other benchmarks.

Questions for authors

PILOT seems quite related to the proposed work. A clarification on the relative novelty would be helpful.

5 comments
AuthorsOpenReview ↗

Response to Reviewer HhN1 (1/2)

We thank the reviewer for the positive assessment. All three ablations the reviewer asked for are now done, and each supports the design. We restate each comment below.

Comment 1: "Separate memories vs a shared memory? What if we give both roles access to one consolidated memory, matching total memory capacity and adaptation budget. Without this comparison, the paper does not establish that separating the two memory slots is beneficial."

Conclusion: separate memories clearly outperform one consolidated memory (64.33% vs. 41.00%).

We added a shared-memory condition to the joint vs. one-sided study (40 Terminal-Bench tasks, ten rounds, three runs per condition). All conditions start from the same memories and use the same merge algorithm; in the shared condition, both roles read and update one memory, which starts as the deduplicated union of the two starting memories.

  • Round 1: shared memory 45.0%, the same as the other conditions (44.2–45.8%), so all conditions start level.
  • Last five rounds: shared memory 41.00 ± 5.27%; separate, jointly evolved memories 64.33 ± 4.01%; worker-only 60.33 ± 3.69%; supervisor-only 55.17 ± 3.33%.
  • Success retention between adjacent rounds: 62.5% for shared memory, versus 77.7–79.9% for the other conditions.
  • Memory size after round 10: 1.04M characters for the shared memory, 1.25M for the two separate memories combined. Size is the outcome of the same evolution and merge process, not a budget we set.

Why this settles the question: the shared memory receives the same information and the same merge algorithm, yet it degrades, because the two roles' lessons collide in one place. In a typical case, an earlier step and a later correction define the same output field in incompatible ways, and the supervisor's completion rules end up inside the worker's procedures. Separate role memories avoid this.

Comment 2: "Compute-matched alternatives? The separate supervisor uses extra compute and time, thus it would be good to investigate additional worker reasoning via self-reviews and self-verifications."

Conclusion: the same compute spent on self-review does not reproduce the gain; an independent supervisor is 9.09 points better.

On 44 held-out Terminal-Bench tasks with evolved worker memory, both conditions run under the same per-task token cap, fixed in advance from the per-task usage of an earlier supervised run. In the self-review condition, the supervisor is disabled; at the same review points, the worker receives a fixed prompt to re-check its progress against the supervisor's initial checklist, inside its own context.

  • Self-review: 52.27 ± 2.27% (24 / 22 / 23 tasks solved in three runs).
  • Independent supervisor with its initial memory: 61.36 ± 4.55% (25 / 27 / 29), better in every run.
  • Total tokens: 31.6M vs. 31.9M, essentially equal.

Reviewing inside its own context, the worker either repeats its earlier reasoning or discards working progress and starts over; the restarts exhausted the budget on 27, 29, and 20 of the 44 tasks, whereas the three supervised runs stayed within it on every task. The benefit therefore comes from an independent context, not from extra tokens; with evolved supervisor memory, the same split reaches 70.45% (without a cap). Without a matched budget, supervision also does not necessarily cost more: total tokens with supervision on are 0.72–1.34× those with supervision off, and fewer for Terminus-2 and TORCH-GA.

AuthorsOpenReview ↗

Response to Reviewer HhN1 (2/2)

Comment 3: "Generalization across other benchmarks? It would be interesting to test whether the memories of the worker or the supervisor can generalize to other benchmarks."

Conclusion: both memories generalize, separately and together, always frozen.

  • Worker memory alone: held-out Terminal-Bench tasks 56.82% → 62.88%; FeatureBench, a different coding benchmark, 13.04% → 43.48%.

  • Supervisor memory alone: on held-out tasks, Claude Code improves by 12.9 points and mini-SWE-agent by 8.5 points; with the worker unchanged, Claude Code goes from 61.36% (initial supervisor) to 67.42% (evolved supervisor memory).

  • Both memories: held-out gains of 13.6 points (Terminal-Bench) and 17.6 points (SWE-bench Verified); FeatureBench 13.04% → 71.01%; office tasks 71.60 → 82.58; web tasks 75.40 → 85.00; with other model families, +15.73 points for GLM-5.3 and +8.24 for DeepSeek-V4.1-Flash.

Question: "PILOT seems quite related to the proposed work. A clarification on the relative novelty would be helpful."

Conclusion: TORCH contributes experimentally supported findings about role-specific experience that PILOT does not test. PILOT combines same-model steering with live skill and memory updates and had no public implementation at submission. Our experiments establish:

(i) role-separated memory matters: a shared memory reaches 41.00%, separate memories 64.33%;

(ii) supervisory experience improves a fixed worker on its own: Claude Code on held-out tasks goes from 61.36% to 67.42% with the worker unchanged, and annotated beneficial interventions rise from 32% to 49%;

(iii) this experience transfers frozen: across four external harnesses, held-out tasks, other model families, and other benchmarks (e.g., FeatureBench 13.04% → 71.01%).

These findings rest on design choices PILOT does not make: separate memories for the two roles, post-evaluation reflection that also keeps evidence-supported lessons from failed runs, and a plug-in supervisor that leaves the worker's loop unchanged.

In summary, each ablation the reviewer requested is now run and supports the design: separate memories beat a shared one, an independent supervisor beats self-review at equal compute, and both memories generalize. We hope these results strengthen the reviewer's assessment.

AuthorsOpenReview ↗

Follow-up on our response

Hi Reviewer HhN1, thank you again for your constructive feedback. Our response brings together role-separation ablations, supervisor memory gains under a fixed worker, and frozen-memory transfer across harnesses to clarify the novelty and generalization of our approach. We would love to know if these results address your remaining questions.

Reviewer_6XuxOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

TORCH is a framework that improves tool-usage agents by splitting the worker agent into a solver and a memory supervisor. The memory supervisor reflects on the task trajectory after every task and updates the memory for the solver and the memory agent. This process creates a stronger memory for the model to work with and thereby improving performance as claimed in the paper. The paper evaluates supervision across several agent harnesses, compares joint/single memory evolution, and studies frozen-memory transfer across benchmarks.

Critical strengths
  • The paper addresses an important problem in long-horizon agents: successful intermediate actions do not necessarily imply correct task completion.

  • Framework works for Cross-benchmark transfer. The memory created in Benchmark A helps boost performance in B. Notably in Table 3, FeatureBench, Office and Web have improved significantly.

  • Ideally, a stronger model would not need two separate roles/instances. The existence of this paper shows there is a better frontier to achieve for future models.

Critical weaknesses
  • Table 1: For comparison a single benchmark is run multiple times because by the virtue of its design the memory M_t accumulates after the task has been attempted by the solver agent. This breaks the core paradigm of the benchmarks which are supposed to reflect capability of a system when deployed in the wild. How much of the improvement comes from reusable experience versus familiarity with those particular tasks? Hidden tests being inaccessible does not remove the feedback obtained from task rewards. Authors should separate memory evolution and evaluation using disjoint task sets.

  • Further on the previous point, the numbers are not iso-compute during evaluation as baseline would receive an order less compute then the method since it will run multiple passes to build the memory.

  • Memory updates are merged sequentially in a fixed execution order (sec 3.5). Does changing this order affect the resulting memory and subsequent performance? A sensitivity analysis would help establish whether the accumulated experience is stable across different task orders.

Questions for authors
  • Can the authors create an iso-compute eval comparison? If not, I am looking forward to their justification.

  • How much of the headline improvement remains when memory evolution and evaluation use disjoint tasks?

  • Does separate worker/supervisor memory outperform a shared memory under matched information and update budgets?

2 comments
AuthorsOpenReview ↗

Response to Reviewer 6Xux

We thank the reviewer for the precise questions. The two central concerns, evaluation on the evolution tasks and compute, are now tested directly, and so is the shared-memory comparison.

Question 2 / Weakness 1: "How much of the headline improvement remains when memory evolution and evaluation use disjoint tasks?"

Conclusion: most of the improvement remains on disjoint tasks.

A fixed random seed splits each benchmark into an evolution split and a disjoint held-out split (45 / 44 tasks on Terminal-Bench 2.1, 250 / 250 on SWE-bench Verified). Memories are evolved only on the evolution split, frozen, and evaluated on the held-out split, three runs each.

  • Both evolved memories vs. the initial worker without supervision: 56.82% → 70.45% on Terminal-Bench (+13.6), 69.87% → 87.47% on SWE-bench Verified (+17.6).
  • Evolved supervisor memory on top of evolved worker memory: +7.6 and +12.1 points.
  • Frozen supervisor memory on external harnesses: +12.9 points for Claude Code, +8.5 for mini-SWE-agent.

Held-out tasks never take part in evolution, so these gains can only come from reusable experience. On SWE-bench Verified the held-out gain (+17.6) is close to the in-distribution gain (+18.5), and cross-benchmark transfer shows the same on entirely different benchmarks (FeatureBench 13.04% → 71.01% with frozen memories).

Question 1 / Weakness 2: "Can the authors create an iso-compute eval comparison? If not, I am looking forward to their justification."

Conclusion: at equal compute the supervisor still wins clearly, and the evolution cost is paid once.

  • Matched compute: under an identical per-task token cap fixed in advance from an earlier supervised run, the independent supervisor reaches 61.36% and self-review in the worker's own context 52.27%, a gap of 9.09 points, at 31.9M vs. 31.6M total tokens (44 held-out Terminal-Bench tasks, three runs).
  • Evaluation cost without a cap: total tokens with supervision on are 0.72–1.34× those with supervision off, and fewer for Terminus-2 and TORCH-GA.
  • Evolution cost: evolution is a one-time cost. All held-out, cross-model, and cross-benchmark results use frozen memories with no updates on the evaluated tasks, so the cost is amortized over all later tasks, as with any learned component.

Weakness 3: "Memory updates are merged sequentially in a fixed execution order (sec 3.5). Does changing this order affect the resulting memory and subsequent performance?"

We did not run a separate permutation study. The merge is designed to limit order effects: each candidate is compared with both the round-start memory and the memory updated by earlier candidates, every modified file is reviewed against the task reward and the role's reflection, and conflicting advice is rewritten as a neutral verification principle or rejected. A lesson rejected in one round can be proposed again when the same failure recurs. Across three independent runs, which produce different candidates in every round, joint evolution reaches similar performance (64.33 ± 4.01% over the last five rounds).

Question 3: "Does separate worker/supervisor memory outperform a shared memory under matched information and update budgets?"

Conclusion: yes, by a wide margin (64.33% vs. 41.00%).

All conditions start from the same memories and use the same merge algorithm; in the shared condition, both roles read and update one memory, which starts as the deduplicated union of the two starting memories, so both roles see the same information.

  • Round 1: shared memory 45.0%, the same as the other conditions (44.2–45.8%).
  • Last five of ten rounds: 41.00 ± 5.27% (shared) vs. 64.33 ± 4.01% (separate).
  • Success retention: 62.5% vs. 77.7–79.9%.
  • Memory size after round 10: 1.04M characters (shared) vs. 1.25M (both separate memories combined); size is the outcome of evolution, not a budget we set.

The decline comes from conflicting entries, such as an earlier step and a later correction defining the same output field in incompatible ways.

The reviewer's three questions are now answered by the held-out evaluation, the matched-budget control, and the shared-memory condition. We hope the reviewer will reconsider the evaluation in light of this evidence.

AuthorsOpenReview ↗

Follow-up on our response

Hi Reviewer 6Xux, since the discussion period is wrapping up soon, we wanted to briefly follow up on our rebuttal. Specifically, we added held-out test results, a compute-matched self-review comparison, and shared-versus-separated memory ablations. Could you let us know if these new results address your concerns?

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response to All Reviewers

We thank all reviewers. The revision adds controlled experiments that directly test the main concerns; all changes in the PDF are marked in blue. In brief:

  • Gains hold on tasks never used for evolution. With frozen memories, held-out Terminal-Bench 2.1 improves from 56.82% to 70.45% and held-out SWE-bench Verified from 69.87% to 87.47%.
  • The supervisor itself improves. With the worker held fixed (Claude Code, held-out tasks), evolving only the supervisor's memory raises success from 61.36% to 67.42%.
  • The gain is not extra compute. Under the same token budget, an independent supervisor outperforms self-review by 9.09 points (61.36% vs. 52.27%).
  • Separate role memories matter. A single shared memory falls to 41.00%, versus 64.33% for separate, jointly evolved memories.
  • All main results are now mean ± SD over three runs.

Table numbers have changed: Table 2 is the new held-out evaluation, Table 3 is cross-model transfer, and Table 4 is cross-benchmark transfer (Table 3 in the submission).

(1) Held-out evaluation (6Xux, kb63). The improvement is not explained by familiarity with the evaluated tasks. A fixed random seed splits each benchmark into an evolution split and a disjoint held-out split (45 / 44 tasks on Terminal-Bench 2.1, 250 / 250 on SWE-bench Verified). Memories are evolved only on the evolution split and then frozen; held-out tasks never take part in evolution. Relative to the initial worker without supervision, the two evolved memories improve Terminal-Bench by 13.6 points (56.82 → 70.45) and SWE-bench Verified by 17.6 points (69.87 → 87.47). Evolved supervisor memory adds 7.6 and 12.1 points on top of evolved worker memory, and frozen supervisor memory attached to external harnesses improves Claude Code by 12.9 points and mini-SWE-agent by 8.5 points.

(2) Supervisor improvement with the worker fixed (YzkK, kb63). Evolved supervisory experience, not an extra review step, produces the gain. When only the supervisor's memory changes, the initial supervisor gives modest gains (1.5–6.4 points on Terminal-Bench, 0.9–1.3 on SWE-bench Verified), while evolved supervisor memory gives large ones (11.2–17.6 and 3.5–8.1). On held-out tasks with Claude Code: 54.55% without supervision, 61.36% with the initial supervisor, 67.42% with evolved supervisor memory. Human annotation of 200 injected interventions (two experts blind to the round, a third adjudicating) shows beneficial interventions rising from 32% to 49% (p = 0.02) and harmful ones falling from 4% to 0%.

(3) Matched-budget self-review (HhN1, 6Xux, kb63). The same compute spent on self-review does not reproduce the gain. On 44 held-out Terminal-Bench tasks with evolved worker memory, each task's total token budget is capped at a level fixed in advance from an earlier supervised run. With nearly identical total tokens (31.9M vs. 31.6M), the independent supervisor reaches 61.36% and self-review 52.27%, a gap of 9.09 points, with the supervisor ahead in every run. Self-review either repeats its earlier reasoning or discards working progress and restarts; it exhausted the budget on 27, 29, and 20 of the 44 tasks, while the three supervised runs stayed within it on every task.

(4) Shared vs. separate memory (HhN1, 6Xux). Separating the two roles' memories matters. All conditions start from the same memories and use the same merge algorithm; in the shared condition, both roles read and update one memory, which starts as the deduplicated union of the two starting memories. In round 1 it matches the other conditions (45.0% vs. 44.2–45.8%); over the last five of ten rounds it averages 41.00 ± 5.27%, versus 64.33 ± 4.01% for separate, jointly evolved memories, and its success retention is 62.5% versus 77.7–79.9%. The cause is conflicting entries in one memory, for example an earlier step and a later correction that define the same output field in incompatible ways.

(5) Three runs (kb63). All main results are now averaged over three runs; for configurations reported in the submission, the original run is one of the three. The conclusions are unchanged.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.