- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This paper proposes a dynamically evolving agent team built upon three components: an organizational structure that defines who does what and who reports to whom, a communication layer for coordination, and a shared workspace that integrates all work into a single artifact.
Critical strengths
The results presented in Table 2 appear strong.
Critical weaknesses
- The claim in L90–92 is insufficiently supported. Assigning distinct roles to agents is not a new idea; well-known frameworks such as MetaGPT have incorporated role specialization for years. Several existing methods, including DyLAN and Self-Organized Agents, also support dynamically changing the team size.
- Furthermore, the paper claims that adding more agents leads to worse outcomes. However, the evidence supporting this claim primarily comes from a car racing game benchmark, which substantially limits the generalizability of the conclusion.
- In L114, the authors state: "By letting the work itself shape the team, we turn the common finding that more agents make things worse into a setting where more agents make things better." For the reasons mentioned above, it is questionable whether this can be characterized as a "common finding." The argument would be considerably more convincing if the authors could statistically demonstrate this phenomenon across a broader and more diverse set of tasks, including larger-scale tasks."
Novelty
- I have concerns about the novelty of the proposed approach. Although the framework consists of three components, none of these components appears particularly novel, either individually or in combination.
Experimental Evaluation
- First, the baselines do not include methods that dynamically adjust team size, such as DyLAN. This omission makes it difficult to assess the advantages of the proposed method over existing approaches to dynamic team organization.
- Second, Table 2, which presents the most substantial performance improvements, does not appear to include a comparison against native Codex with the /goal command. Codex can also perform multi-agent task decomposition and parallel execution using its sub-agent functionality, potentially with higher reasoning effort settings such as ultra. However, the paper does not provide a comparison against such configurations.
Questions for authors
- In the related work section, the authors make the following claim regarding existing systems, including DyLAN and Self-Organized Agents: "Across these systems, adding agents brings diminishing or negative returns, mostly through specification and coordination failures rather than reasoning." What specific evidence supports this statement?