acceptodds
Withdrawn from ICLR 2027

AI Designs and Trains Tool-Using Agent Teams

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Developing a tool-using agent requires deciding what to test, how to organize work, and how to learn from execution. These decisions interact: a change in system design alters both the failures that remain and the experience available for training. We introduce ResidualForge, an AI-directed framework that connects executable task construction, failure-guided system design, and native policy training. The designer refines a single actor, tests changes to its skills and division of work, and records which cases each change repairs or regresses. The selected design is then fixed while its policies learn from trajectories that retain actual tool observations and role-to-role handoffs. Across ClawEval, QwenClawBench, and TeamBench, we evaluate complete systems, their development components, and transfer across execution harnesses. On QwenClawBench, on-policy distillation improves the selected Qwen3.5-9B system from 34.36 to 37.42. Removing team handoffs from training context causes the largest component-study drop on ClawEval and QwenClawBench, and pooled training rollouts improve performance in an unseen harness. System selection also retains a single actor on TeamBench when larger candidates perform worse. These results connect autonomous engineering decisions to the quality of the resulting system and the execution experience used to train it.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Harness, RSI

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_NivxOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces ResidualForge, an AI-directed framework that connects executable task construction, failure-guided system design and native policy training. The paper conducts experiments of training policy with three different methods, including SFT, OPD and GRPO. Experiments use Qwen3.5-9B on ClawEval, QwenClawBench, and TeamBench, with design/training comparisons, component ablations, and cross-harness transfer, proving that this method surpasses traditional single-actor control.

Critical strengths
  1. This framework is coherent which connects system development to the execution conditions under which policies learn.
  2. The experiments shows the effectiveness of each component and improvement on existing benchmarks, including QwenClawBench and ClawEval.
Critical weaknesses
  1. The paper only compares ResidualForge with single-actor control rather than popular agent framework such as openhands, claude code.
  2. The paper uses Qwen3.5-122B as the teacher to train a Qwen3.5-9B student. While the improvement from OPD is encouraging, it is important to distinguish gains from strong-teacher supervision from those specifically attributable to ResidualForge's automated system design and role-conditioned training.
Questions for authors
  1. Could the authors compare ResidualForge against an established agent framework such as OpenHands and Claude Code using the same Qwen3.5-9B backbone?
  2. How much of the performance improvement can be attributed to ResidualForge rather than strong-teacher distillation? In particular, how would a strong existing agent system (e.g., OpenHands), or a well-configured single-agent baseline, perform when trained in the same way?
Reviewer_nbPkOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes RESIDUALFORGE, a framework that connects task construction, agent team design, and policy training. It uses execution failures to revise the system, then trains the selected design on trajectories containing real tool observations and role handoffs. The reported results show improvements across three benchmarks, and ablations suggest that preserving handoff context is useful. Its main strength is treating system design and training as connected decisions. However, the ClawEval training comparison uses a longer time limit after OPD, while cost reporting and evaluation across model families remain limited.

Critical strengths

1.Comprehensive problem formulation. The paper brings development task construction, agent team design, and model training into a single workflow, addressing the practical fact that these three components influence one another.

2.Training data grounded in real execution. The trajectories retain actual tool outputs, workspace states, and handoffs between roles. Performance declines when handoff information is removed, providing experimental support for this design choice.

Critical weaknesses
  1. The ClawEval before-and-after training comparison is confounded by the time limit. The per-task time limit is doubled after OPD, so the improvement from 32.10% to 35.80% cannot be attributed entirely to training.

  2. Insufficient cost reporting. The method uses a strong teacher model, multiple rounds of system search, and online execution. Although the paper lists cost categories, it does not provide enough numerical detail to assess whether the gains justify the resources used.

  3. Limited model coverage. The main results are concentrated on a single student model family.

Questions for authors
  1. Could the authors reevaluate the models before and after OPD under exactly the same ClawEval time limit? This would help isolate the gain attributable to training.

  2. How many designer calls, teacher tokens, agent execution hours, and training compute are required for one complete development run? How does its cost-effectiveness compare with human-designed systems or simply allocating more inference compute?

Reviewer_KmmrOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes ResidualForge, a framework in which an LLM "designer" constructs development tasks, proposes system revisions, and specifies policy training for tool-using agents. The designer iterates on the system design (prompts, skills, roles, communication, routing) based on the tasks that still fail. Once a design is chosen, it is frozen, and its policies are trained with OPD, optionally followed by GRPO, on trajectories collected in the actual harness. The authors evaluate on ClawEval, QwenClawBench and TeamBench and report four main findings: design search and OPD both help; the ranking of two designs flips after training; keeping inter-role context during training matters; and training on several harnesses transfers to an unseen one.

Critical strengths
  • I like the question the paper asks. When you build an agent system, the structure you choose also shapes the data its policies will later learn from, so treating design and training together makes sense. Doing this in realistic tool-use environments is a plus.

  • Some of the controls are well thought out. Fixed and adaptive designs are compared under the same development conditions, continued OPD and GRPO start from the same checkpoint, and extra roles are compared against simply giving the parent agent more compute.

  • The OPD result on QwenClawBench (34.36 → 37.42) is the cleanest in the paper, since the time limit, task split and grader are the same before and after training. The ablations also point in a consistent direction on inter-role context and multi-harness training, although I'm not sure how much weight they can carry (see Weakness 1).

Critical weaknesses
  • Limited statistical evidence and unclear evaluation sets.

I couldn't find explicit evaluation-task counts for each condition. The paper also does not report independent repetitions of design search or training, or confidence intervals. Judging from the scores, ClawEval appears to be an 81-task set, so one task is worth about 1.23 points. If that is right, the ranking reversal in Table 2 is a one-task gap before training and a one-task gap after, at least on ClawEval (on QwenClawBench the gaps are 0.22 and 0.60 points). Several of the ablation differences are about the same size.

A related issue is that not all ClawEval rows seem to come from the same evaluation set. The OPD+GRPO scores (36.02/67.08) don't fit 81 tasks but match 58/161 and 108/161, and the SFT scores (27.99/56.85) fit neither 81 nor 161. The paper doesn't explain this. Without uncertainty estimates and clarification of the evaluation-task counts, I find it hard to assess the reliability and comparability of these results.

  • The ClawEval OPD gain is confounded.

According to Table 5 and Appendix B, the OPD model was evaluated with twice the task-time limit of the selected design. Since timeouts count as failures, part of the jump from 32.10 to 35.80 could come from the extra time rather than from training alone. The authors mention this in the Limitations section, but it's easy to miss when reading the main results. To be fair, the paper reports that on development tasks, with the doubled limit both before and after training, rubric scores still go up (61.53 → 64.92). I would still like to see the test-set number under the original time limit.

  • It isn't clear that multiple agents actually help.

The selected system ends up being a single agent on TeamBench and a two-role system on ClawEval. In Appendix D.2, simply giving the parent agent the same extra compute matches the conditional-role system on ClawEval (35.80) and comes very close on QwenClawBench (37.28 vs. 37.42). There is also no single-agent + OPD baseline on ClawEval or QwenClawBench.

TeamBench doesn't help settle this either. The benchmark is meant to test coordination when the Planner, Executor and Verifier roles are kept separate, but here it is run with the full-access interface. Table 1 also places the paper's strict-verdict scores next to published full-team scores that use a different scoring rule. So I can't tell how much of the improvement comes from the multi-agent structure itself.

  • Missing implementation details and baselines.

The paper never says which LLM is used as the designer, which matters a lot since the designer is the core of the method. Search budgets, stopping criteria, training hyperparameters and compute costs are also missing or only partly given. There is no comparison with existing automated design methods under a matched budget, and there are very few concrete examples of generated tasks, selected designs or revision records. As it stands, the method would be hard to reproduce, and it is hard to judge what each part contributes.

  • Relation to prior work.

The cross-harness transfer experiment closely resembles the setup in OpenForgeRL's Section 5.2, which also compares ZeroClaw-only training against pooled ZeroClaw, OpenClaw and Codex training, but OpenForgeRL isn't cited in that section. Although the models and training methods differ, the paper does not clearly explain what additional insight this experiment provides beyond the earlier study. Related work on automated prompt and topology search for multi-agent systems (MASS), joint optimization of model roles and weights (Heterogeneous Swarms), and joint optimization of prompts and model parameters in compound AI systems (Optimas) is also not discussed. A clearer comparison would help establish what is new in ResidualForge.

Questions for authors
  1. How many unique tasks are used in each evaluation condition, and how many independent design-search or training runs were conducted? Were the ClawEval OPD+GRPO (36.02/67.08) and SFT (27.99/56.85) scores computed on different task sets? Could you provide uncertainty estimates for the main comparisons, especially the ranking reversal?

  2. What does the OPD model score on ClawEval under the original time limit?

  3. Could you add a single-agent + OPD baseline on ClawEval and QwenClawBench, and a comparison with an existing automated design method under the same budget?

  4. Why was TeamBench run with the full-access interface, and how should these results be read against the full-team leaderboard?

  5. Which LLM is the designer? Could you report the search budget, stopping criteria, training hyperparameters and compute cost?

  6. Where do the external reference scores in Table 1 come from, in particular the TeamBench entries and Gemini 3.1 Pro's QwenClawBench score?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.