- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
The paper proposes Hyperbolic Graph-of-Agents (HGoA), a training-free framework for coordinating multiple LLM-based agents without iterative natural-language message passing. It constructs a dual-channel agent graph from observable outputs (a factual channel based on response-key agreement and a semantic channel using distances in a hyperbolic-tangent space), and formulates collaboration as minimizing a convex consensus energy over representation vectors, followed by a lightweight meta readout for final answer synthesis. Experiments on six benchmarks show that HGoA achieves competitive or superior accuracy to strong baselines (notably GoA) while substantially reducing LLM calls, tokens, and latency.
Critical strengths
-
Training-free MAS topology construction is a novel research.
-
The use of hyperbolic space for MAS topology construction is interesting and reminds me of HGCN [1].
-
The public code enhances the reproducibility of the work.
-
The paper is clearly written, with intuitive figures.
[1] Hyperbolic Graph Convolutional Neural Networks, NeurIPS 2019.
Critical weaknesses
- Unfair Comparison
Firstly, as shown in Table 1, GoA and HGoA use only three agents, whereas other methods use six. However, more agents do not necessarily lead to better performance, as shown by AgentPrune [1] and AgentDropout [2].
Secondly, RL-based baselines typecally face greater optimization challenges as the topology design space grows [3]. Thus, the performance gap may partly reflect differences in design difficulty rather than methodological superiority.
Additionally, the paper states that its agent pool mainly consists of six LLMs with varying capabilities. HGoA's performance may therefore benefit from selecting a particularly strong subset of three agents.
My major concern is the lack of a controlled comparison of topology design methods. A fairer evaluation would use the same agent pool and initial topology, then compare how different methods prune redundant edges or add beneficial ones.
- Questionable feasibility
Firstly, the scalability of the proposed training-free approach remains unclear. As the number of agents increases, effective topology design typically require iterative feedback and refinement, as explored by RL-based methods [3] and LLM-driven search [4]. Whether HGoA remains effective as the design space grows is not demonstrated.
Secondly, the factual and semantic channels may not generalize well across heterogeneous or component MAS scenarios. The factual channel relies on answer agreement, while the semantic channel uses embedding similarity. However, real-world MASs often involve agents with different roles and expertise, whose responses may be complementary rather than similar. Consequently, relying on answer agreement and semantic similarity may overlook valuable complementary information and limit the method's applicability across diverse tasks.
References
[1] CUT THE CRAP: An Economical Communication Pipeline for LLM-Based Multi-Agent Systems. ICLR 2025.
[2] AgentDropout: Dynamic Agent Elimination for Token-Efficient and High-Performance LLM-Based Multi-Agent Collaboration. ACL 2025.
[3] GPTSwarm: Language Agents as Optimizable Graphs. ICML 2024.
[4] AFLOW: Automating Agentic Workflow Generation. ICLR 2025.
Questions for authors
Please refer to weaknesses.
4 comments
Weaknesses 4: My major concern...add beneficial ones.
Thank you for your insightful comment. Our responses are as follows:
1. Problem Clarification: HGoA is not a training-based pruning or edge-adding method. It constructs the agent graph in a training-free manner at test-time inference based on the observable initial responses of each agent under the current query. Therefore, HGoA differs from methods such as AgentPrune and G-Designer in that it does not require training-based pruning or adding operations on the agent graph under the same initial graph.
2. Controlled Comparison Experiments: To fairly compare the performance of topology design methods, we have added controlled comparison experiments in the Appendix A.5. We vary only the topology design method of the agent graph while keeping all other components identical. Full denotes the fully connected model that retains all edges of the agent graph. Sampling denotes the model that constructs a sparse topology using the edge sampling mechanism of GoA. AgentPrune-like denotes the model that constructs the topology using the one-shot low-rank pruning strategy of AgentPrune. The experimental results are as follows:
New Table A3: Experimental results of topology design comparison.
| Model | MMLU | MMLU-Pro | GPQA | MATH | Human Eval | MedMCQA |
|---|---|---|---|---|---|---|
| Full | 73.21 | 53.88 | 41.92 | 70.48 | 82.93 | 56.64 |
| Sampling | 74.67 | 54.67 | 40.40 | 68.78 | 85.37 | 57.30 |
| AgentPrune-like | 76.54 | 55.41 | 40.91 | 69.56 | 84.27 | 59.29 |
| HGoA | 78.25 | 57.43 | 43.43 | 72.20 | 86.59 | 62.06 |
As shown in Table A3, the performance of HGoA is overall superior to that of the other comparison baselines. Compared with Full and Sampling, HGoA effectively eliminates redundant interference. The experimental results demonstrate the effectiveness of dynamically selecting the topology based on the query. Compared with AgentPrune-like, HGoA, as a training-free method, can construct an effective topology without training. Therefore, the above experimental results demonstrate the effectiveness of the hyperbolic-aware dual-channel graph construction method.
Weaknesses 5: Firstly, the scalability ... not demonstrated.
Thank you for your insightful comment. Our responses are as follows:
1. Problem Clarification: HGoA does not design topological structures through iterative feedback or search. Instead, it constructs the agent graph in a training-free manner at test-time inference based on the observable initial responses of each agent under the current query. Consequently, HGoA avoids the issue in RL methods where iterative costs grow exponentially with the number of agents.
2. Scalability Experiments: We have reported the scalability experiments of HGoA in detail in Table 3 of Subsection 4.3 and Table A1 of Appendix A.3. To further validate its effectiveness as the design space expands, we have added 4 agents to the original 6 agents. The experimental results are as follows:
Revised Table 3 (Partial): Specific experimental results regarding model scalability on GPQA.
| Pool of agents | Acc | Calls | Token (k) | Time (s) |
|---|---|---|---|---|
| GPT-5.6-Terra | 83.23 | 1 | 0.44 | 25.34 |
| HGoA (2 Agents) | 83.33 | 4 | 1.49 | 55.81 |
| HGoA (3 Agents) | 87.47 | 5 | 1.78 | 69.23 |
| HGoA (6 Agents) | 88.48 | 8 | 2.46 | 109.95 |
| HGoA (10 Agents) | 89.80 | 12 | 4.88 | 198.72 |
Revised Table A1 (Partial): Experimental results regarding model scalability.
| Pool of agents | GPQA | MATH | MedMCQA |
|---|---|---|---|
| GPT-5.6-Terra | 83.23 | 88.80 | 91.25 |
| HGoA(2 Agents) | 83.33 | 98.56 | 92.52 |
| HGoA(3 Agents) | 87.47 | 99.64 | 94.74 |
| HGoA(6 Agents) | 88.48 | 99.76 | 95.70 |
| HGoA (10 Agents) | 89.80 | 99.84 | 96.59 |
As shown in Tables 3 and A1, HGoA exhibits an overall upward trend in performance as the number of agents gradually increases, indicating that HGoA has good scalability.
Weaknesses 6: Secondly, the ... diverse tasks.
Thank you for your thoughtful comment. Our responses are as follows:
1. Elaboration of Complementary Information: The factual channel serves as a conditional hard constraint rather than a global consistency requirement. When answers conflict, edge cutting is intended to block error propagation, but cutting edges does not mean discarding complementary information. Semantic channels are modeled as soft edge weights rather than similarity thresholds, with soft connections constructed using the median bandwidth of the current query distance distribution. In addition, the graph gradient flow in HGoA exchanges only consistency and reliability information, while the complete complementary content is preserved in the frozen original responses and integrated by a single meta-LLM. Therefore, the dual-channel design does not require different expert responses to be similar, nor does it ignore complementary information due to answer consistency or embedding similarity.
2. Applicability Discussion: The six agent roles and their knowledge in Table A3 are all different, constituting a typical heterogeneous MAS scenario. HGoA still outperforms baselines such as Full, Sampling, and AgentPrune-like on the six tasks, demonstrating that the hyperbolic-aware dual-channel graph construction method has good applicability across multiple tasks.
Response
(1) As I mentioned, the comparison is unfair because HGoA incorporates agent selection (AS), while some baselines could also benefit from such selection strategies. When agent selection is removed, the improvement is not significant as shown in Table A2. For example, both GPTSwarm and GOA outperform HGoA without AS.
(2) You should also compare HGoA with existing baselines to demonstrate its scalability, rather than reporting results for HGoA alone.
By the way, I would suggest that the authors use a single dialogue box in future rebuttals. Excessively fragmented responses only give the impression that the authors struggle to be concise, which also raises concerns about whether the original manuscript really suffers from the shortcomings.
In summary, I lean towards rejection.
Weaknesses 1: Firstly, as shown... and AgentDropout [2].
Thank you for your thoughtful comment. Our responses are as follows:
In Table 1, the agent pools of all multi-agent methods (including HGoA and GoA) consist of 6 agents. HGoA and GoA additionally incorporate an Agent Selection module, which selects 3 relevant agents from the 6 candidates based on the current query to participate in subsequent multi-agent collaboration. Therefore, the number of agents (3 Agents) in Table 1 refers to the number of agents that actually participate in multi-agent collaboration after agent selection, rather than indicating that the agent pool contains only 3 agents. Our intention is to demonstrate that, given the same agent pool, after agent selection, the performance of HGoA with only 3 agents can achieve or even surpass that of the baselines using 6 agents, rather than to argue that fewer agents are always better. To this end, we have made the corresponding revisions in Table 1.
Weaknesses 2: Secondly, RL-based... methodological superiority.
Thank you for your thoughtful comment. Our responses are as follows:
1. Experimental Analysis: We agree with your comment. We have added the relevant statement to the experimental analysis in Table 1.
2. Methodological Advantages: RL-based baselines are difficult to optimize due to the large search space, leading to longer training time or higher token consumption. However, HGoA constructs an agent graph from observable agent responses, enabling direct use for test-time inference without topology training. As shown in Table 2, the inference costs of HGoA (Calls, Token, and Time) are all lower than those of the compared baselines. HGoA converts the collaboration medium of multi-agent systems from natural language text into representation vectors, effectively improving the collaboration efficiency among agents. This advantage is one of the inherent strengths of HGoA.
3. Performance Comparison: Our primary baselines are not limited to RL-based baselines but also include non-RL-based baselines such as MoA, GoA, AgentDropout, and Grammar. Under identical experimental settings, HGoA achieves overall superior performance compared with these baselines. In addition, ablation results (subsection 4.4) confirm the performance contribution of each component of HGoA, demonstrating that its performance advantage is not attributable to the difficulty of RL optimization.
Weaknesses 3: Additionally, the paper ... of three agents.
Thank you for your insightful comment. Our responses are as follows:
We have added a new Subsection A.4 (Agent Selection Experiments) in the Appendix. We fix six agents and feed them separately into HGoA and the compared baselines. Specifically, we remove the agent selection module of HGoA to evaluate whether HGoA's performance depends on the performance of the agents themselves. The experimental results are as follows:
New Table A2: Comparison experiment results of agent selection. HGoA w/o AS denotes the model with the Agent Selection module removed from HGoA.
As listed in Table A2, the performance of HGoA w/o AS on the six datasets is overall superior to that of the other comparison baselines, indicating that the performance advantage of HGoA benefits not only from the agent selection module but also from the proposed hyperbolic-aware dual-channel graph construction method and the hyperbolic collaboration method based on consensus energy.