acceptodds
Under review as a conference paper at ICLR 2027

Hyperbolic Graph-of-Agents: A Training-Free Framework for LLM-Based Multi-Agent Collaboration in Hyperbolic Space

Abstract

With the continuous advancement of agents, the need to orchestrate multi-agent collaboration to enhance task performance has become increasingly urgent. Existing multi-agent collaboration methods typically use natural language text as the collaboration medium for multiple rounds of message passing, causing inference costs to grow rapidly with the number of collaboration rounds and agents, and introducing semantic drift through multiple rounds of comprehension and restatement. To this end, we propose Hyperbolic Graph-of-Agents (HGoA), a training-free framework for large language model-based multi-agent collaboration in hyperbolic space. Specifically, a novel hyperbolic-aware dual-channel graph construction method is proposed to adaptively construct an agent graph from observable outputs in a training-free manner. Next, a novel hyperbolic collaboration method based on consensus energy is proposed, which uses representation vectors instead of natural language text as the collaboration medium, thereby mitigating semantic drift while reducing inference costs. Furthermore, we leverage hyperbolic space to overcome the limitation of Euclidean space in failing to capture the hierarchical structures that exist among agent responses. Finally, experiments on six benchmarks demonstrate that, compared with mainstream methods, HGoA can improve the efficiency of multi-agent collaboration on various complex tasks.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
Multi-agent collaboration, Message passing, Hyperbolic space, Representation vectors

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_d5EfOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper proposes Hyperbolic Graph-of-Agents (HGoA), a training-free framework for coordinating multiple LLM-based agents without iterative natural-language message passing. It constructs a dual-channel agent graph from observable outputs (a factual channel based on response-key agreement and a semantic channel using distances in a hyperbolic-tangent space), and formulates collaboration as minimizing a convex consensus energy over representation vectors, followed by a lightweight meta readout for final answer synthesis. Experiments on six benchmarks show that HGoA achieves competitive or superior accuracy to strong baselines (notably GoA) while substantially reducing LLM calls, tokens, and latency.

Critical strengths
  1. Training-free MAS topology construction is a novel research.

  2. The use of hyperbolic space for MAS topology construction is interesting and reminds me of HGCN [1].

  3. The public code enhances the reproducibility of the work.

  4. The paper is clearly written, with intuitive figures.

[1] Hyperbolic Graph Convolutional Neural Networks, NeurIPS 2019.

Critical weaknesses
  1. Unfair Comparison

Firstly, as shown in Table 1, GoA and HGoA use only three agents, whereas other methods use six. However, more agents do not necessarily lead to better performance, as shown by AgentPrune [1] and AgentDropout [2].

Secondly, RL-based baselines typecally face greater optimization challenges as the topology design space grows [3]. Thus, the performance gap may partly reflect differences in design difficulty rather than methodological superiority.

Additionally, the paper states that its agent pool mainly consists of six LLMs with varying capabilities. HGoA's performance may therefore benefit from selecting a particularly strong subset of three agents.

My major concern is the lack of a controlled comparison of topology design methods. A fairer evaluation would use the same agent pool and initial topology, then compare how different methods prune redundant edges or add beneficial ones.

  1. Questionable feasibility

Firstly, the scalability of the proposed training-free approach remains unclear. As the number of agents increases, effective topology design typically require iterative feedback and refinement, as explored by RL-based methods [3] and LLM-driven search [4]. Whether HGoA remains effective as the design space grows is not demonstrated.

Secondly, the factual and semantic channels may not generalize well across heterogeneous or component MAS scenarios. The factual channel relies on answer agreement, while the semantic channel uses embedding similarity. However, real-world MASs often involve agents with different roles and expertise, whose responses may be complementary rather than similar. Consequently, relying on answer agreement and semantic similarity may overlook valuable complementary information and limit the method's applicability across diverse tasks.

References

[1] CUT THE CRAP: An Economical Communication Pipeline for LLM-Based Multi-Agent Systems. ICLR 2025.

[2] AgentDropout: Dynamic Agent Elimination for Token-Efficient and High-Performance LLM-Based Multi-Agent Collaboration. ACL 2025.

[3] GPTSwarm: Language Agents as Optimizable Graphs. ICML 2024.

[4] AFLOW: Automating Agentic Workflow Generation. ICLR 2025.

Questions for authors

Please refer to weaknesses.

4 comments

Weaknesses 1: Firstly, as shown... and AgentDropout [2].

Thank you for your thoughtful comment. Our responses are as follows:

In Table 1, the agent pools of all multi-agent methods (including HGoA and GoA) consist of 6 agents. HGoA and GoA additionally incorporate an Agent Selection module, which selects 3 relevant agents from the 6 candidates based on the current query to participate in subsequent multi-agent collaboration. Therefore, the number of agents (3 Agents) in Table 1 refers to the number of agents that actually participate in multi-agent collaboration after agent selection, rather than indicating that the agent pool contains only 3 agents. Our intention is to demonstrate that, given the same agent pool, after agent selection, the performance of HGoA with only 3 agents can achieve or even surpass that of the baselines using 6 agents, rather than to argue that fewer agents are always better. To this end, we have made the corresponding revisions in Table 1.


Weaknesses 2: Secondly, RL-based... methodological superiority.

Thank you for your thoughtful comment. Our responses are as follows:

1. Experimental Analysis: We agree with your comment. We have added the relevant statement to the experimental analysis in Table 1.

2. Methodological Advantages: RL-based baselines are difficult to optimize due to the large search space, leading to longer training time or higher token consumption. However, HGoA constructs an agent graph from observable agent responses, enabling direct use for test-time inference without topology training. As shown in Table 2, the inference costs of HGoA (Calls, Token, and Time) are all lower than those of the compared baselines. HGoA converts the collaboration medium of multi-agent systems from natural language text into representation vectors, effectively improving the collaboration efficiency among agents. This advantage is one of the inherent strengths of HGoA.

3. Performance Comparison: Our primary baselines are not limited to RL-based baselines but also include non-RL-based baselines such as MoA, GoA, AgentDropout, and Grammar. Under identical experimental settings, HGoA achieves overall superior performance compared with these baselines. In addition, ablation results (subsection 4.4) confirm the performance contribution of each component of HGoA, demonstrating that its performance advantage is not attributable to the difficulty of RL optimization.


Weaknesses 3: Additionally, the paper ... of three agents.

Thank you for your insightful comment. Our responses are as follows:

We have added a new Subsection A.4 (Agent Selection Experiments) in the Appendix. We fix six agents and feed them separately into HGoA and the compared baselines. Specifically, we remove the agent selection module of HGoA to evaluate whether HGoA's performance depends on the performance of the agents themselves. The experimental results are as follows:

New Table A2: Comparison experiment results of agent selection. HGoA w/o AS denotes the model with the Agent Selection module removed from HGoA.

MMLUMMLU-ProGPQAMATHHuman EvalMedMCQA
Debate72.5347.0529.2970.0840.2453.05
Refine77.4054.7138.9271.6080.4954.94
GPTSwarm77.9355.4840.9168.3283.5459.89
AgentDropout75.4454.3838.3865.5279.8858.40
MoA75.7153.3332.8365.8476.2254.94
Self-MoA78.1454.1933.8469.1679.2755.56
MAGRPO74.3954.5240.4065.3675.6152.43
Grammar73.4453.9037.8868.0883.5460.12
GoA79.1854.7839.9869.7684.6760.04
HGoA78.2557.4343.4372.2086.5962.06
HGoA w/o AS77.6756.5742.5470.8085.9861.92

As listed in Table A2, the performance of HGoA w/o AS on the six datasets is overall superior to that of the other comparison baselines, indicating that the performance advantage of HGoA benefits not only from the agent selection module but also from the proposed hyperbolic-aware dual-channel graph construction method and the hyperbolic collaboration method based on consensus energy.

Weaknesses 4: My major concern...add beneficial ones.

Thank you for your insightful comment. Our responses are as follows:

1. Problem Clarification: HGoA is not a training-based pruning or edge-adding method. It constructs the agent graph in a training-free manner at test-time inference based on the observable initial responses of each agent under the current query. Therefore, HGoA differs from methods such as AgentPrune and G-Designer in that it does not require training-based pruning or adding operations on the agent graph under the same initial graph.

2. Controlled Comparison Experiments: To fairly compare the performance of topology design methods, we have added controlled comparison experiments in the Appendix A.5. We vary only the topology design method of the agent graph while keeping all other components identical. Full denotes the fully connected model that retains all edges of the agent graph. Sampling denotes the model that constructs a sparse topology using the edge sampling mechanism of GoA. AgentPrune-like denotes the model that constructs the topology using the one-shot low-rank pruning strategy of AgentPrune. The experimental results are as follows:

New Table A3: Experimental results of topology design comparison.

ModelMMLUMMLU-ProGPQAMATHHuman EvalMedMCQA
Full73.2153.8841.9270.4882.9356.64
Sampling74.6754.6740.4068.7885.3757.30
AgentPrune-like76.5455.4140.9169.5684.2759.29
HGoA78.2557.4343.4372.2086.5962.06

As shown in Table A3, the performance of HGoA is overall superior to that of the other comparison baselines. Compared with Full and Sampling, HGoA effectively eliminates redundant interference. The experimental results demonstrate the effectiveness of dynamically selecting the topology based on the query. Compared with AgentPrune-like, HGoA, as a training-free method, can construct an effective topology without training. Therefore, the above experimental results demonstrate the effectiveness of the hyperbolic-aware dual-channel graph construction method.


Weaknesses 5: Firstly, the scalability ... not demonstrated.

Thank you for your insightful comment. Our responses are as follows:

1. Problem Clarification: HGoA does not design topological structures through iterative feedback or search. Instead, it constructs the agent graph in a training-free manner at test-time inference based on the observable initial responses of each agent under the current query. Consequently, HGoA avoids the issue in RL methods where iterative costs grow exponentially with the number of agents.

2. Scalability Experiments: We have reported the scalability experiments of HGoA in detail in Table 3 of Subsection 4.3 and Table A1 of Appendix A.3. To further validate its effectiveness as the design space expands, we have added 4 agents to the original 6 agents. The experimental results are as follows:

Revised Table 3 (Partial): Specific experimental results regarding model scalability on GPQA.

Pool of agentsAccCallsToken (k)Time (s)
GPT-5.6-Terra83.2310.4425.34
HGoA (2 Agents)83.3341.4955.81
HGoA (3 Agents)87.4751.7869.23
HGoA (6 Agents)88.4882.46109.95
HGoA (10 Agents)89.80124.88198.72

Revised Table A1 (Partial): Experimental results regarding model scalability.

Pool of agentsGPQAMATHMedMCQA
GPT-5.6-Terra83.2388.8091.25
HGoA(2 Agents)83.3398.5692.52
HGoA(3 Agents)87.4799.6494.74
HGoA(6 Agents)88.4899.7695.70
HGoA (10 Agents)89.8099.8496.59

As shown in Tables 3 and A1, HGoA exhibits an overall upward trend in performance as the number of agents gradually increases, indicating that HGoA has good scalability.

Weaknesses 6: Secondly, the ... diverse tasks.

Thank you for your thoughtful comment. Our responses are as follows:

1. Elaboration of Complementary Information: The factual channel serves as a conditional hard constraint rather than a global consistency requirement. When answers conflict, edge cutting is intended to block error propagation, but cutting edges does not mean discarding complementary information. Semantic channels are modeled as soft edge weights rather than similarity thresholds, with soft connections constructed using the median bandwidth of the current query distance distribution. In addition, the graph gradient flow in HGoA exchanges only consistency and reliability information, while the complete complementary content is preserved in the frozen original responses and integrated by a single meta-LLM. Therefore, the dual-channel design does not require different expert responses to be similar, nor does it ignore complementary information due to answer consistency or embedding similarity.

2. Applicability Discussion: The six agent roles and their knowledge in Table A3 are all different, constituting a typical heterogeneous MAS scenario. HGoA still outperforms baselines such as Full, Sampling, and AgentPrune-like on the six tasks, demonstrating that the hyperbolic-aware dual-channel graph construction method has good applicability across multiple tasks.

Reviewer_d5EfOpenReview ↗

Response

(1) As I mentioned, the comparison is unfair because HGoA incorporates agent selection (AS), while some baselines could also benefit from such selection strategies. When agent selection is removed, the improvement is not significant as shown in Table A2. For example, both GPTSwarm and GOA outperform HGoA without AS.

(2) You should also compare HGoA with existing baselines to demonstrate its scalability, rather than reporting results for HGoA alone.

By the way, I would suggest that the authors use a single dialogue box in future rebuttals. Excessively fragmented responses only give the impression that the authors struggle to be concise, which also raises concerns about whether the original manuscript really suffers from the shortcomings.

In summary, I lean towards rejection.

Reviewer_Zym4OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes Hyperbolic Graph-of-Agents (HGoA), a training-free framework for reducing communication costs in LLM-based multi-agent systems. It selects agents using model cards and constructs a graph from answer agreement and semantic similarity. Support-based radial initialization and consensus energy minimization in tangent space yield aggregation weights, which guide answer selection or a single meta-LLM aggregation step. Experiments on six benchmarks, including ablation, robustness, and scaling studies, show that HGoA achieves the best performance on five benchmarks among the evaluated methods while reducing calls, token usage, and latency relative to GoA. The work explores representation-based aggregation as an efficient alternative to repeated textual interaction.

Critical strengths
  1. Practical efficiency gains. HGoA reduces repeated generation through lightweight processing of initial response representations. Table 2 reports reductions of 54.55% in LLM calls, 59.55% in average token usage, and 39.01% in average latency relative to GoA, demonstrating its practical potential.

  2. Broad empirical support. Evaluation covers knowledge reasoning, mathematics, code generation, and medical question answering, supplemented by ablation, agent-replacement robustness, and scaling experiments. Leading results on five of six benchmarks provide initial evidence that the benefits extend across tasks.

Critical weaknesses
  1. The distinct role of hyperbolic graph construction is unclear. In Eqs. (1)–(3), ignoring numerical stabilization terms, the uniform scaling induced by the logarithmic map cancels through the kernel bandwidth, making this stage equivalent to Euclidean RBF graph construction. The necessity of hyperbolic geometry therefore remains unsubstantiated.
  2. Controlled comparisons isolating the geometric contribution are missing. The ablations in Table A1 change multiple factors simultaneously. Comparisons against Euclidean graph smoothing, simple voting, and a single meta-LLM aggregation using identical initial responses are needed to identify the source of the performance gains.
  3. Answer disagreement is not fully handled. Under Eqs. (3)–(4), answers A, A, B assign zero weight to B, even if it is correct. When all three answers differ, all weights become zero, yet the paper does not specify a tie-breaking rule.
  4. The objective and solution formulas are inconsistent. Eq. (5) uses an unsquared fidelity norm, whereas the gradient in Eq. (7) and the closed-form solution in Eq. (9) correspond to a squared Frobenius norm objective. The mathematical formulation and its implementation should be clarified and reconciled.
Questions for authors
  1. How do the authors explain the scaling cancellation in Eqs. (1)–(3) and the resulting equivalence to Euclidean RBF graph construction? Is the subsequent node-dependent radial initialization the primary source of the geometric benefits?

  2. With identical initial responses and inference budgets, does HGoA consistently outperform Euclidean graph smoothing, simple voting, and a single meta-LLM aggregation?

  3. When all three parseable answers differ and all support scores are zero, how does the implementation determine the final answer?

  4. Should the fidelity term in Eq. (5) use the squared Frobenius norm? Which objective and solver are implemented, and are they consistent with Eqs. (7) and (9)?

2 comments

Weaknesses 1 and Questions 1: The distinct role of ... remains unsubstantiated. How do the ... geometric benefits?

Thank you for your thoughtful comment. Our responses are as follows:

1. Computational Equivalence: The hyperbolic graph construction is equivalent to Euclidean RBF graph construction, which is merely a local metric equivalence in the hyperbolic tangent space, intended to avoid unstable computation on the manifold.

2. The Distinctive Role of Hyperbolic Graph Construction: Multi-agent responses exhibit a hierarchical structure ranging from shared consensus to individual divergence. Hyperbolic graph construction effectively models this hierarchical structure. It maps nodes to the Poincaré ball (Eq. (2)) and establishes the topological structure in conjunction with the hard constraints of the factual channel (Eq. (3)). This capability of constructing graphs on a manifold substrate with hierarchical structure is beyond what Euclidean RBF graphs can provide.

3. The Necessity of Hyperbolic Geometry: Message passing mechanisms in Euclidean space are prone to inducing node feature homogenization (over-smoothing). We perform consensus energy optimization in the hyperbolic tangent space (Subsection 3.5), leveraging the capacity of hyperbolic geometry to preserve hierarchical distances and incorporating the fidelity term constraint in the objective function, thereby effectively mitigating the over-smoothing phenomenon.


Weaknesses 2 and Questions 2: *Controlled comparisons... the performance gains. With identical ... meta-LLM aggregation? *

Thank you for your insightful comment. Our responses are as follows:

We have modifed the method of agent collaboration and response generation while keeping the other components of HGoA unchanged. We have then added the following models to Table 4 (Original Table A1): (1) Euclidean graph smoothing (EGS): EGS replaces hyperbolic collaboration to isolate the contribution of hyperbolic geometry. (2) Simple voting (SV): SV performs majority voting over identical initial responses. (3) Single meta-LLM aggregation (SMA): SMA aggregates identical initial responses with a single meta-LLM, without using geometric metrics or geometric prompts. The experimental results are as follows:

Revised Table 4: Experimental results of the ablation study.

ModelMMLUMMLU-ProGPQAMATHHuman EvalMedMCQAAvg_Token
HGoA w/ EGS76.6655.3840.6170.1284.1558.115.78k
HGoA w/ SV76.1254.3938.8969.5683.5456.715.17k
HGoA w/ SMA77.4255.7641.0170.8483.7858.835.32k
HGoA w/o GC74.6754.6740.4068.7885.3757.308.48k
HGoA w/o RV77.7556.2942.4271.7284.7660.6313.45k
HGoA78.2557.4343.4372.2086.5962.065.44k

As listed in Table 4, the performance of HGoA w/ EGS is inferior to that of HGoA, indicating that hyperbolic geometry serves as an additional source of gain for modeling the hierarchical structure of responses. The performance of both HGoA w/ SV and HGoA w/ SMA is inferior to that of HGoA, demonstrating that simple voting and aggregation operations are unable to exploit the critical information (geometric metrics) among agent responses to further improve the final response quality.


Weaknesses 3 and Questions 3: Answer disagreement is ... a tie-breaking rule. When all three ... the final answer?

Thank you for your insightful comment. Our responses are as follows:

When the answer is parseable and simultaneously produces disagreement, Fij=0F_{ij}=0. This only cuts off the response divergence between two nodes and does not amount to removing the answer from the candidate set. If there exists a node whose hierarchical support degree sis_{i} equals 0 due to Fij=0F_{ij}=0, then Path A (Meta-free) does not directly output the response of that node but instead switches to Path B (Meta-select). Path B incorporates a new rule, namely that nodes whose sis_{i} is set to zero and whose answer keys are parseable are permitted to enter Path B. For the case (A, A, B), B is still fed into the meta-LLM as the original response. If all si≤εs_{i}\le\varepsilon, the response of each node directly enters Path B. This establishes an explicit tie-breaking rule even when the three answers are mutually distinct. Furthermore, when an answer is unparseable, all answers directly enter Path B.

Weaknesses 4 and Questions 4: *The objective and ... and reconciled. Should the... Eqs. (7) and (9)? *

Thank you for your helpful comment. Our response is as follows:

We omitted FF (the Frobenius norm) in the original Eq. (5), which had caused a coefficient mismatch between Eqs. (5) and (7). The revised Eq. (5) is given as follows:

Y(Z)=12∥Z−Z(0)∥F2+λ2tr⁡(ZTLZ),(5)Y(Z)=\frac{1}{2}\left\|Z-Z^{(0)}\right\|_{F}^{2}+\frac{\lambda}{2}\operatorname{tr}\left(Z^{\mathrm{T}}LZ\right), \qquad (5)

Since LL is positive semidefinite, the Hessian becomes positive definite after adding the identity matrix term. Therefore, the objective function is strictly convex and its minimum is unique. The closed-form solution is (I+λL)−1Z(0)(I+\lambda L)^{-1}Z^{(0)}, which is consistent with Eq. (9).

Reviewer_VitkOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper focuses on how to combine answers from several pretrained language models while limiting the expense of repeated discussion. It also aims to prevent later interactions from corrupting useful initial answers. The proposed HGoA addresses these goals by keeping the original answers fixed and performing most subsequent processing on numerical representations. The implementation uses six specialist/general-purpose models of roughly 7–8 billion parameters, usually selecting three, with a frozen all-MiniLM-L6-v2 encoder and Qwen2.5-7B-Instruct as the coordinator.

Critical strengths
  1. Tables 1–2 show that the reduced communication budget does not generally come at the expense of benchmark performance. Against the three-agent GoA configuration, HGoA improves five scores while reducing calls.

  2. The method requires generated answers, model descriptions, and a separate encoder, rather than compatible internal model states or task-specific fine-tuning, which makes it potentially useful where different models are accessed through separate interfaces.

  3. Table 1 includes knowledge questions, mathematics, programming, and medical questions. This shows broader applicability than evaluation on a single multiple-choice dataset.

  4. Figure 3 examines imperfect model selection, and Table 3 exposes diminishing returns as the number of agents increases, which are useful additions to the main accuracy table.

Critical weaknesses
  1. Tables 1–2 demonstrate end-to-end gains but omit simple voting, confidence-weighted aggregation, and one-call synthesis using identical initial responses. This major evidence gap leaves it unclear whether improvements arise from the proposed geometry or simply from avoiding repeated discussion.

  2. Equation (5) uses an unsquared norm and coefficients that do not match Equation (7). Consequently, the stated gradient and strict-convexity claim do not follow as written.

  3. Section 4.1 omits sampled test-set sizes, repeated-run counts, and uncertainty estimates, making small gains difficult to assess. Table 2 also reports five calls throughout despite the coordinator-free branch.

  4. Figure 3 tests replacement with less relevant agents, not information preservation or error propagation. Keeping initial responses unchanged does not establish that weighting or final synthesis preserves their useful content. The authors should provide direct evidence or narrow this claim to avoiding repeated textual revision.

Questions for authors
  1. What do the existing ablations show about geometry and aggregation?

  2. How frequently is each output branch used? Give the approximate per-dataset fraction using the coordinator-free branch, and explain why Table 2 reports five calls throughout. Are those actual averages, fixed budgets, or upper bounds?

  3. What is the evaluation unit and repetition protocol? State the test-set sizes, number of independent decoding runs, and whether table entries are single runs or averages. Were baseline methods evaluated on the same sampled questions and initial candidate responses where applicable?

  4. What existing evidence supports the drift claim? Is there a measured change in answer correctness or information preservation across processing stages, or is the claim limited to retaining the initial texts unchanged?

3 comments

Weaknesses 1: Tables 1–2 demonstrate ... avoiding repeated discussion.

Thank you for your insightful comment. Our responses are as follows:

1. Performance Gain Analysis: The end-to-end gains shown in Tables 1 and 2 mainly stem from two aspects: (1) The hyperbolic-aware dual-channel graph construction method effectively models the topological structure of multi-agent collaboration in a training-free manner, thereby improving the collaboration efficiency of multi-agent systems. (2) HGoA transforms the traditional collaboration medium of multi-agent systems from natural language text into representation vectors, thereby avoiding the repeated transmission of long texts between agents.

2. Comparative Experiment Analysis: To further analyze the sources of performance gains in Tables 1 and 2, we have added new comparison baselines to the Table 1: (1) Simple Voting (SV) is a model that performs majority voting over identical initial responses. (2) Confidence-Weighted Aggregation (CWA) is a model that performs weighted aggregation of initial responses according to the model confidence. (3) One-Call Synthesis (OCS) is a model that concatenates identical initial responses and invokes the meta-LLM only once. The experimental results are as follows:

Revised Table 1 (Partial): Results of the effectiveness experiments

MMLUMMLU-ProGPQAMATHHuman EvalMedMCQA
SV67.12±0.1246.56±0.2931.72±0.4269.64±0.2365.12±1.0952.24±0.15
CWA70.83±0.3949.32±0.3332.63±0.7765.04±0.2976.34±1.1753.21±0.08
OCS68.30±0.3047.08±0.3128.38±0.6667.56±0.1778.66±1.8354.72±0.12
HGoA78.25±0.3457.43±0.6343.43±1.1372.20±0.1486.59±0.7562.06±0.28

As shown in Table 1, although SV and CWA avoid repeated discussion, their performance lags behind that of HGoA, indicating that simple aggregation cannot effectively handle semantic conflicts and hierarchical structures among agents. Meanwhile, OCS concatenates long texts in a single call, leading to information redundancy, whereas HGoA converts the collaboration medium into representation vectors, effectively suppressing semantic drift while avoiding multi-round transmission. Therefore, the performance gain of HGoA does not merely stem from avoiding repeated discussion, but is fundamentally attributed to its graph construction in hyperbolic space and the hyperbolic collaboration method based on consensus energy.


Weaknesses 2: Equation (5) uses an ... follow as written.

Thank you for your thoughtful comment. Our responses are as follows:

We omitted FF (the Frobenius norm) in the original Eq. (5), which had caused a coefficient mismatch between Eqs. (5) and (7). The revised Eq. (5) is given as follows:

Y(Z)=12∥Z−Z(0)∥F2+λ2tr⁡(ZTLZ),(5)Y(Z)=\frac{1}{2}\left\|Z-Z^{(0)}\right\|_{F}^{2}+\frac{\lambda}{2}\operatorname{tr}\left(Z^{\mathrm{T}}LZ\right), \qquad (5)

Since LL is positive semidefinite, the Hessian becomes positive definite after adding the identity matrix term. Therefore, the objective function is strictly convex and its minimum is unique. The closed-form solution is (I+λL)−1Z(0)(I+\lambda L)^{-1}Z^{(0)}, which is consistent with Eq. (9).


Weaknesses 3 and Questions 2: Section 4.1 omits sampled ... coordinator-free branch. How frequently ...or upper bounds?

Thank you for your insightful comment. Our responses are as follows:

1. Test Set Sizes: MMLU comprises 57 subjects, with each subject containing 50 questions. MMLU-Pro comprises 14 categories, with each category containing 150 questions. GPQA contains 198 questions. MATH contains 500 questions. HumanEval contains 164 questions. MedMCQA contains 4,183 questions.

2. Repeated Runs and Uncertainty: The experimental results in the original manuscript were averaged over 5 runs without reporting standard deviations, making small gains difficult to assess. To address this, we have revised the experimental results in Table 1 to report the mean and standard deviation.

3. Calls: Table 2 reports the protocol upper bounds. In Table 2, since the Calls (the number of LLM calls) of both MoA and GoA are at their maximum, the number of LLM calls of HGoA is also set to its maximum, thereby ensuring a fair comparison among HGoA, MoA, and GoA.

4. Proportion of Samples Assigned to the Coordinator-free Branch: for Path A (coordinator-free branch), the approximate proportions are about 89% on MMLU, about 73% on MedMCQA, about 70% on GPQA, about 26% MMLU-Pro, and about 0% on MATH, and HumanEval. The remaining samples follow Path B.

Weaknesses 4: Figure 3 tests ... repeated textual revision.

Thank you for your thoughtful comment. Our responses are as follows:

In Fig. 3, our experimental objective is to simulate scenarios of erroneous agent selection using noisy agents (i.e., agents with lower relevance) in order to evaluate the robustness of HGoA. In the original analysis of the experimental results, we explained that one reason for HGoA's strong robustness is that it freezes the initial responses to prevent noise from being repeatedly propagated during multi-agent collaboration. However, this robustness analysis may be ambiguous, causing readers to misinterpret the experimental results as demonstrating that HGoA's weighting or final synthesis preserves useful content under the condition that the initial responses remain unchanged. To this end, we have revised this part and narrowed the experimental claims, as detailed below:

The main reason is that the fidelity term in HGoA suppresses over-smoothing while avoiding repeated text revisions, thereby reducing the likelihood of noisy text emerging during the revision process.


Questions 1: What do the existing ablations show about geometry and aggregation?

Thank you for your insightful comment. Our responses are as follows:

We modify the method of agent collaboration and response generation while keeping the other components of HGoA unchanged. We then add the following models to Table 4 (Original Table A1): (1) Euclidean graph smoothing (EGS): EGS replaces hyperbolic collaboration to isolate the contribution of hyperbolic geometry. (2) Simple voting (SV): SV performs majority voting over identical initial responses. (3) Single meta-LLM aggregation (SMA): SMA aggregates identical initial responses with a single meta-LLM, without using geometric metrics or geometric prompts. The experimental results are as follows:

Revised Table 4: Experimental results of the ablation study.

ModelMMLUMMLU-ProGPQAMATHHuman EvalMedMCQAAvg_Token
HGoA w/ EGS76.6655.3840.6170.1284.1558.115.78k
HGoA w/ SV76.1254.3938.8969.5683.5456.715.17k
HGoA w/ SMA77.4255.7641.0170.8483.7858.835.32k
HGoA w/o GC74.6754.6740.4068.7885.3757.308.48k
HGoA w/o RV77.7556.2942.4271.7284.7660.6313.45k
HGoA78.2557.4343.4372.2086.5962.065.44k

As listed in Table 4, the performance of HGoA w/ EGS is inferior to that of HGoA, indicating that hyperbolic geometry serves as an additional source of gain for modeling the hierarchical structure of responses. The performance of both HGoA w/ SV and HGoA w/ SMA is inferior to that of HGoA, demonstrating that simple voting and aggregation operations are unable to exploit the critical information (geometric metrics) among agent responses to further improve the final response quality.


Questions 3: *What is the ....where applicable? *

Thank you for your thoughtful comment. Our responses are as follows:

1. Evaluation Units: The basic unit is a single question. Accuracy is reported for MMLU, MMLU-Pro, GPQA, MATH, and MedMCQA, and pass@1 is reported for HumanEval.

2. Test Set Sizes: MMLU comprises 57 subjects, with each subject containing 50 questions. MMLU-Pro comprises 14 categories, with each category containing 150 questions. GPQA contains 198 questions. MATH contains 500 questions. HumanEval contains 164 questions. MedMCQA contains 4,183 questions.

3. Repeated Runs and Uncertainty: Each question is sampled only once and kept fixed across all experiments. The five repetitions vary only the decoding randomness, and the same set of random seeds is used for all baselines and HGoA. The experimental results in the original manuscript were averaged over 5 runs without reporting standard deviations, making small gains difficult to assess. To address this, we have revised the experimental results in Table 1 to report the mean and standard deviation.

4. Evaluation Conditions: All baselines and HGoA use the same fixed set of questions and the same six-model agent pool.

Questions 4: *What existing evidence ... texts unchanged? *

Thank you for your thoughtful comment. Our responses are as follows:

1. Drift Claim: We restrict the drift claim to the collaborative process avoiding repeated revisions to the initial text. Subsection 3.5 performs collaboration on representation vectors while keeping the initial response frozen. The fidelity term in Eq. (5) constrains the representation vectors from deviating from the hierarchical initial values, thereby suppressing over-smoothing. In Subsection 3.6, Path A directly returns the frozen answer when the answer key is unique, while Path B invokes the meta-LLM only once, with the original response and geometric prompt as input.

2. Future Work: We will explore direct and indirect measurements across processing stages, including answer accuracy at each stage and the extent to which key information from initial responses is retained after collaboration and final synthesis, so that drift claims can be quantified in terms of correctness.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.