acceptodds
Withdrawn from ICLR 2027

Self-Evolving Agent Team for Open-Ended Creation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

One agent is too slow to build something large. A natural next step is a bigger team, yet we find that more agents, without structure, deliver less. They collide, repeat work, and leave parts of the job with no one responsible. In any task, new demands appear as work progresses, and the team has to reorganize itself to handle them. We present Self-Evolving Agent Team, a multi-agent system that reorganizes itself as it works. As the work progresses, a dedicated agent watches the team, finds what is falling behind, and reshapes structure and roles in response. Like human teamwork, the system rests on three backbones: a reporting tree for accountability, a messaging layer for coordination, and a shared codebase for integration. Across software builds, knowledge work, and standard agent benchmarks, ours outperforms fixed teams of the same size and improves as it grows

Primary area:
foundation or frontier models, including LLMs
Keywords:
Agents, Inference, Multi-Agent

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_ndwdOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes a dynamically evolving agent team built upon three components: an organizational structure that defines who does what and who reports to whom, a communication layer for coordination, and a shared workspace that integrates all work into a single artifact.

Critical strengths

The results presented in Table 2 appear strong.

Critical weaknesses
  • The claim in L90–92 is insufficiently supported. Assigning distinct roles to agents is not a new idea; well-known frameworks such as MetaGPT have incorporated role specialization for years. Several existing methods, including DyLAN and Self-Organized Agents, also support dynamically changing the team size.
  • Furthermore, the paper claims that adding more agents leads to worse outcomes. However, the evidence supporting this claim primarily comes from a car racing game benchmark, which substantially limits the generalizability of the conclusion.
  • In L114, the authors state: "By letting the work itself shape the team, we turn the common finding that more agents make things worse into a setting where more agents make things better." For the reasons mentioned above, it is questionable whether this can be characterized as a "common finding." The argument would be considerably more convincing if the authors could statistically demonstrate this phenomenon across a broader and more diverse set of tasks, including larger-scale tasks."

Novelty

  • I have concerns about the novelty of the proposed approach. Although the framework consists of three components, none of these components appears particularly novel, either individually or in combination.

Experimental Evaluation

  • First, the baselines do not include methods that dynamically adjust team size, such as DyLAN. This omission makes it difficult to assess the advantages of the proposed method over existing approaches to dynamic team organization.
  • Second, Table 2, which presents the most substantial performance improvements, does not appear to include a comparison against native Codex with the /goal command. Codex can also perform multi-agent task decomposition and parallel execution using its sub-agent functionality, potentially with higher reasoning effort settings such as ultra. However, the paper does not provide a comparison against such configurations.
Questions for authors
  • In the related work section, the authors make the following claim regarding existing systems, including DyLAN and Self-Organized Agents: "Across these systems, adding agents brings diminishing or negative returns, mostly through specification and coordination failures rather than reasoning." What specific evidence supports this statement?
Reviewer_S3aPOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes Self-Evolving Agent Team, a multi-agent framework that dynamically reorganizes agent roles and hierarchies during task execution. It combines hierarchical coordination, inter-agent communication, and shared workspaces, with an Architect agent adjusting team structures based on observed progress. Experiments on game development, coding benchmarks, and real-user tasks report improvements over several baselines.

Critical strengths
  1. The paper addresses a relevant problem in multi-agent collaboration, particularly coordination failures in complex artifact-generation tasks.
  2. The reported performance improvements are promising, especially on game development and ProgramBench.
Critical weaknesses
  1. Limited novelty and unclear research contribution. Dynamic agent recruitment, hierarchical coordination, and feedback-driven adaptation have already been extensively studied. Although the paper compares several multi-agent frameworks, it omits more directly relevant approaches such as MegaAgent, which supports dynamic hierarchical team organization, and EvoMAC, which adapts collaboration networks through feedback from software development. The proposed Architect primarily relies on LLM prompting to reorganize roles, and it remains unclear what novel mechanism or generalizable finding the paper establishes beyond existing approaches.
  2. Insufficient empirical validation. The experiments do not convincingly establish whether organizational restructuring itself, rather than the overall coordination framework, accounts for the improvements. The main comparisons lack uncertainty estimates, and Table 5 reports only one run per ablation condition on a single task. The evaluation also lacks systematic comparisons across prompting strategies, backbone models, and alternative coordination configurations, limiting the robustness and generalizability of the conclusions.
Questions for authors
  1. What specific new insight does the proposed approach establish beyond existing dynamic multi-agent organization methods?
  2. How does the proposed framework compare with MegaAgent and EvoMAC on comparable coding or artifact-generation tasks under matched experimental conditions?
  3. How sensitive are the results to different Architect prompting strategies and backbone models?
Reviewer_89t8OpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper uses the following tasks:

  • eight browser games
  • GDPval: knowledge-work tasks from ten occupations (one task each)
  • ten randomly selected problems each from ProgramBench and NL2RepoBench
  • three real-user requests

The open-ended means that in each of these tasks the model is given a vague/underspecified request. The agents are Codex CLI sessions running GPT-5.6 Sol. Some of the tasks are judged by a language model as well. “Self-evolving” refers to the team’s organization changing over time.

  • The pipeline starts with a single “director” agent receiving the request.
  • The director and the Architect decide on the first positions to create, and the director gives them tasks.
  • The positions work on their tasks. some are given the ability to hire positions below them. a position with two or more positions below it only assigns and reviews work instead of building. so the agents are choosing whether to do their tasks or spawn more agents below them to solve the task.
  • Each position completes its tasks and reports upward.
  • The director decides when to ship a version at the end of each round.
  • After each round, the Architect reviews the shipped version and organizes the team again. This is the evolution "evolution" part of the paper.
  • The run always lasts a fixed ten rounds.
Critical strengths

S1. The paper tackles a timely problem. Scaling LLM agent teams for open-ended software development. It shows simply adding agents degrades output.

S2. The method overall follows the spirit of the bitter lesson by letting the team’s structure emerge from a general outer loop (the Architect) rather than fixing it in advance.

Critical weaknesses

W1. Open ended simply means that the request is vague in this setup. The request names what to build but doesn’t define the features or any specifics. I think what open-ended means in this setup should be clarified in the paper.

W2. Self-Evolving means simply that a separate Architect agent changes the team’s organization between rounds. I think this should also be clarified in the paper.

W3. Although the paper claims the team “is never designed,” much of the system is hand-designed (the tree hierarchy, charters, a standard set of positions, etc.). The ablations show that performance depends heavily on these choices. For example, removing the standard positions drops the score from 74 to 14.

W4. The evaluation mostly rests on a single run per system and task, so the reported gaps have no variance estimates and may not be robust.

W5. It is unclear what gap this paper fills, since prior work already grows or restructures agent teams during execution, and the claimed differences are neither clearly articulated nor tested against most of these systems.

Questions for authors

Can you clarify how these results advance science?

what was previously unknown in the literature? what is the gap you identified? what is the theoretical or empirical evidence that you provide to fill this gap?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.