- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes OpenAgency, a multi-agent system that treats collaboration protocols as main objects for governance and evolution, aiming to strike a balance between flexibility and controllability. Authors demonstrate the effectiveness of the proposed approach through experiments across 14 benchmarks.
Critical strengths
-
The paper has a relatively complete structure and content.
-
The experiments covered a wide range of configurations.
Critical weaknesses
-
The paper does not sufficiently explain how the initial protocol was constructed and how the experimental baselines were configured. In particular, the construction of the initial protocol (e.g., initial agent number, role assignment, collaboration topology, and completion conditions) and the detailed configurations of the single-agent and multi-agent (init) setups are unclear.
-
The paper does not sufficiently demonstrate that the optimization effect comes from trace-driven evolution. Since the initial protocol design is not clearly specified and may contain task-specific limitations, the observed gains could partly come from refining an initially inadequate collaboration structure. More analysis is needed to evaluate the robustness of the evolution process under different initial protocols.
Questions for authors
-
How is the initial collaboration protocol constructed? Specifically, what are the initial agents, role assignments, collaboration topology, and completion conditions?
-
How sensitive is the evolution process to different initial protocols?
Dear Reviewers,
Could you please review the authors’ rebuttals and follow up with any further comments or questions?
Many thanks for your time and efforts.
Best regards, AC