- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
The paper proposes TORCH, a framework that pairs an LLM worker with a same-model runtime supervisor and maintains separate long-term memories for both the worker and the supervisor. During execution, the supervisor periodically inspects the worker's trajectory and can inject guidance. After each task is evaluated, both roles reflect on their own interaction records using the shared reward, and the proposed memory updates are reviewed and merged.
Critical strengths
-
The idea is clear. The reward of a task has different meaning for the supervisor and the worker, the worker needs to learn how to complete tasks and verify the rewards, and the superviser needs to learn when to interfere and under what condition should it interfere.
-
The held-out evaluation results are promising. After freezing the memory evolved on Terminal-Bench and transferring it to FeatureBench, the pass rate increased from 13.0% to 71.0%. This is the most convincing evidence in the paper.
-
The paper is honest in giving failed cases and analyzing how the supervisor can steer the worker wrong.
Critical weaknesses
-
The headline results in Table 1 are obtained after evolving memory on the evaluation tasks. TORCH-GA's worker and supervisor memories are evolved with reward feedback for 20 rounds on all Terminal-Bench 2.1 tasks and for 5 rounds on all SWE-bench Verified tasks, and are then evaluated on the same tasks. I think this is a serious problem, because this means that the scores do not measure generalization. A held-out split evaluation might be needed.
-
The comparison with existing baselines is not controlled. The three baselines differ from TORCH in harness (SWE-agent, mini-swe-agent, the authors' own systems) and in supervisor model (Claude Sonnet 4.6, a trained 4B critic, GPT-5.3-Codex). Their supervisor-off scores on Terminal-Bench are far below TORCH's supervisor-off scores with the same Qwen3.8-27B worker, which is a sign of substantial differences in setup. In addition, only TORCH's supervisor memory has been tuned on the evaluation tasks.
-
Results rely on single runs and do not control for compute. Tables 1 and 2 only report one execution per task per configuration. The paper's own evolution study shows substantial run-to-run variation, so results relying on single runs might be a problem. Supervision also adds model calls, and in several configurations it increases total token usage (Table 5). However, there is no equal-budget baseline. It is unclear how much of the gain comes from supervision rather than variance or extra computation.
-
I think the novelty of this paper is relatively limited, because these ideas of evolving context as well as superviser exists, and the merge is mainly implemented by prompts. This paper is more an engineering contribution combining ideas together.
Questions for authors
- Can the authors report Table 1 results on tasks the memories were never evolved on?
- For the external harnesses in Table 1, what are the results when the supervisor uses its initial memory template rather than the evolved one?
- For the baselines, can SupervisorAgent, LivePlan, or Steer be attached to the same worker harness and model used for TORCH, with TORCH's supervisor using its initial memory?
- Can the authors report repeated runs for the main comparisons, together with an equal-budget baseline such as best-of-2 sampling?
4 comments
Response to Reviewer kb63 (2/2)
Weakness 3 / Question 4: "Results rely on single runs and do not control for compute." "Can the authors report repeated runs for the main comparisons, together with an equal-budget baseline such as best-of-2 sampling?"
Conclusion: all main results are now averaged over three runs, and at an equal budget the supervisor still wins clearly.
-
Repeated runs: all main results are mean ± SD over three runs, including the original run; the conclusions are unchanged.
-
Equal budget: under an identical per-task token cap, the independent supervisor reaches 61.36% and self-review in the worker's own context 52.27%, a gap of 9.09 points, at 31.9M vs. 31.6M tokens (44 held-out Terminal-Bench tasks).
-
Without a cap, total tokens with supervision on are 0.72–1.34× those with supervision off (single run), and fewer for Terminus-2 and TORCH-GA. Best-of-2 costs about 2× and needs a selector to pick one of the two attempts; using the hidden tests as that selector would leak test information, which our setting does not allow. The matched-budget self-review control is the equal-budget comparison we report.
Weakness 4: "I think the novelty of this paper is relatively limited ... This paper is more an engineering contribution combining ideas together."
Conclusion: the revision establishes findings that do not follow from simply combining supervision and memory.
(i) How memory is organized matters: putting both roles' experience into one memory degrades performance to 41.00%, versus 64.33% for separate, jointly evolved memories.
(ii) Supervisory experience improves a fixed worker on its own: Claude Code on held-out tasks goes from 61.36% to 67.42% with the worker unchanged, and annotated beneficial interventions rise from 32% to 49%.
(iii) An independent context matters: at the same token budget, the supervisor outperforms self-review by 9.09 points.
(iv) The experience transfers frozen across harnesses, held-out tasks, model families, and benchmarks (e.g., FeatureBench 13.04% → 71.01%).
The reviewer's main concerns, evaluation on the evolution tasks, single runs, and compute, are now answered by the held-out evaluation, three-run averages, and the matched-budget control, and our conclusions rest on fully controlled within-harness comparisons. We hope the reviewer will reconsider the evaluation, in particular the soundness score.
Follow-up on our response
Hi Reviewer kb63, as the discussion window closes in about two days, we wanted to check in on our response. We included held-out test evaluations and repeated-run results, and clarified the distinction between cross-system comparisons and supervisor memory gains under a fixed worker. Please let us know if any soundness concerns remain so we can discuss them before the deadline.
Thanks, my concerns are addressed, I decide to maintain my score.
Response to Reviewer kb63 (1/2)
We thank the reviewer for the detailed review, and we are glad the reviewer found the frozen-memory transfer to FeatureBench (13.04% → 71.01%) the most convincing evidence. The new held-out evaluation extends exactly this kind of evidence to Terminal-Bench 2.1 and SWE-bench Verified.
Conclusion: yes; the gains hold on tasks the memories never saw.
A fixed random seed splits each benchmark into an evolution split and a disjoint held-out split (45 / 44 tasks on Terminal-Bench 2.1, 250 / 250 on SWE-bench Verified). Memories are evolved only on the evolution split, frozen, and evaluated on the held-out split, three runs each, with the same configuration grid as Table 1.
Conclusion: the initial supervisor helps only modestly; the large gains come from evolved supervisory experience.
Table 1 now reports this setting for every harness. With its initial memory, the supervisor improves performance by 1.5–6.4 points on Terminal-Bench and 0.9–1.3 points on SWE-bench Verified; with evolved memory, by 11.2–17.6 and 3.5–8.1 points. For example, Claude Code on Terminal-Bench: 61.80% without supervision, 64.42% with the initial supervisor, 79.40% with evolved supervisor memory. On held-out tasks the same holds: 61.36% with the initial supervisor, 67.42% with evolved supervisor memory.
Conclusion: cross-system comparability and the effectiveness of evolved supervisor memory within a fixed worker setup are distinct questions. Our evidence for the latter is directly controlled.
(a) Cross-system comparisons. We have not transplanted all baseline supervisors into a common harness. Their supervision logic is tightly coupled with their native implementations: SupervisorAgent in a modified smolagents, Steer in the authors' modified mini-swe-agent 1.11.1, and LivePlan in SWE-agent 1.1.0. Transplanting them would require implementation changes that could alter their behavior and introduce confounds. We therefore follow the original papers and released code, preserving each method's native supervision logic and using its recommended or best-performing available configuration: Steer's best-performing released critic and GPT-5.3-Codex as LivePlan's advisor, since the original GPT-5.2-Codex was discontinued. All systems use the same worker model. We report each baseline together with its own supervisor-off/on comparison, while recognizing that cross-system scores also reflect harness differences.
(b) Controlled within-harness comparisons. Within each harness, we fix the worker, model, and tasks and compare supervision Off / Initial / Evolved, on both in-distribution and disjoint held-out tasks. The Initial column explicitly tests TORCH without evolved supervisory experience, and evolved supervisor memory improves over Initial in every harness. These comparisons directly establish the benefit of evolving supervisor memory for a fixed worker setup. Differences between baseline harnesses do not invalidate this within-harness evidence.