acceptodds
Under review as a conference paper at ICLR 2027

HyperMAS: Evolving Collaboration Primitives for Scalable Multi-Agent LLM Collaboration

Abstract

Multi-agent systems (MAS) powered by Large Language Models (LLMs) have demonstrated strong collaborative reasoning capabilities across complex tasks, yet their effectiveness often comes with substantial communication cost. Existing adaptive MAS methods can customize roles, models, and communication structures for each query, but they still rely on an implicit execution assumption: collaboration capability must be unfolded through explicit agent communication whenever a task is solved. As a result, even repeatedly validated coordination processes are re-executed from scratch, causing redundant token consumption and limiting long-term scalability. In this work, we propose \ourmethod, a hypergraph-centered framework that evolves collaborative skills for scalable multi-agent LLM systems. \ourmethod represents functional roles as role-level mixture nodes enhanced by heterogeneous LLMs and organizes multi-role coordination through high-order hyperedges. During training, \ourmethod explores explicit hypergraph collaboration, identifies reliable and cost-effective sub-hypergraphs from execution traces, and distills them into evolved collaborative skills. During inference, \ourmethod adaptively invokes compact skills for familiar routines and expands into explicit hypergraph communication for uncertain cases, balancing efficiency and flexibility. Extensive experiments on mathematical reasoning, code generation, and knowledge-intensive benchmarks show that \ourmethod is (1) effective, improving average accuracy by points over strong adaptive MAS baselines; (2) efficient, reducing token cost by ; and (3) scalable, decreasing active hyperedge communication by after skill accumulation.

Primary area:
reinforcement learning
Keywords:
multi-agent systems, LLM agents, reinforcement learning, collaborative reasoning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_QxhMOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper argues that existing multi-agent systems repeatedly execute complete collaboration processes for each task, resulting in substantial communication costs. To address this issue, the authors propose HyperMAS, which uses hypergraphs to represent high-order collaboration among multiple roles and distills sub-hypergraphs into reusable collaborative skills. During inference, the system executes existing skills through a single LLM call and falls back to explicit multi-agent collaboration when the results are unreliable, thereby reducing token consumption. Experiments show that HyperMAS improves average accuracy by 2.11 percentage points over the strongest baseline while reducing token costs by 28.4%.

Critical strengths
  1. The paper provides extensive evaluations across multiple benchmarks and includes ablation studies.

  2. The paper presents an interesting approach by combining hypergraph-based modeling with reusable collaborative topology skills.

Critical weaknesses
  1. Previous studies have already explored distilling multi-agent collaboration into reusable skills. However, the paper does not clearly distinguish the contributions of skill reuse and hypergraph modeling.

  2. The token reduction may primarily result from compressing multiple LLM calls into a single call with a more elaborate prompt, rather than preserving the benefits of multi-agent collaboration.

  3. Conventional graph-based methods may achieve similar functionality. The lack of direct comparisons makes the necessity and unique advantages of hypergraph modeling unclear.

Questions for authors
  1. Many previous studies have already explored distilling multi-agent collaboration topologies into reusable skills. However, the paper does not seem to clearly disentangle the contributions of skill reuse and hypergraph modeling. Does the current evidence sufficiently demonstrate the effectiveness of the hypergraph component?

  2. Why are the distilled skills executed through a single LLM call? Could the observed reduction in token consumption simply result from reformulating multi-agent workflows into more elaborate prompts and compressing multiple LLM calls into a single call? After all, previous studies have shown that single-agent systems generally consume significantly fewer tokens than multi-agent systems.

  3. The paper emphasizes the necessity of hypergraph modeling, but could conventional graph-based methods achieve similar functionality? Would a direct comparison with graph-based approaches help better demonstrate the unique advantages of the proposed hypergraph framework?

5 comments
AuthorsOpenReview ↗

Response to Reviewer QxhM (1/3)

Thank you for your careful review and the specific questions. We appreciate your recognition of combining hypergraph modeling with reusable collaborative skills.

Q1–Q2. What comes from structure, and what comes from compressing calls?

R1. HyperMAS uses the group representation to discover what to reuse and the compact call to reduce communication.

  • What the reusable unit contains. The source structure serves two decisions. During construction, it defines the joint exchanges that can be extracted as skills. At inference, it defines what the system executes when the verifier requests expansion.

  • Why execute a skill in one call? In the matched prompt comparison, the distilled routine alone scores 89.68% at 5.85K tokens per query. Running the same initial answers through verification and selective expansion gives 91.46% at 6.39K.

Revision. We will use a solve–check–combine example to show what the template carries and when the source roles execute.

Q3. Can an ordinary graph preserve the same collaboration information?

R2. A graph with an auxiliary coordination node can preserve the joint membership of a Solver/Verifier/Synthesizer group. HyperMAS records that group directly as a hyperedge and uses it to search for and extract recurring exchanges. The empirical question is whether that representation improves which routines the system discovers and reuses under a comparable execution budget.

The graph control will preserve the same group memberships through auxiliary nodes and hold the role inventory, trace budget, extraction interface, and verifier fixed. Compact-skill success measures what each representation helps extract. Final accuracy and end-to-end tokens measure what that difference is worth at inference.

RepresentationGroup identityFinal mean
Pairwise projectionRole pairs only89.95%
Incidence graphPreserved through an event node90.94%
HypergraphPreserved through a hyperedge91.46%

The incidence row retains the same group information as the hypergraph row. Their comparison tests the encoding interface under matched extraction and execution budgets.

AuthorsOpenReview ↗

Response to Reviewer QxhM (2/3)

Thank you for pointing out that an incidence graph can also represent group events. We appreciate your recognition of the multiple-benchmark evaluation and component ablations.

Follow-up on the stored skill and the graph-encoding control.

R3. This gives us a direct control for whether a hyperedge interface helps extraction.

Consider computing a price after a 20% discount and 10% tax on 100 USD. The solver proposes 100 × 0.8 × 1.1. The checker verifies that tax is applied to the discounted value and that both rates are in the correct units. The combiner returns 88 USD with the verified calculation. The stored routine carries the proposal format, two checks, and the combine condition. One executor can produce and check the compact answer. Expansion runs the source roles when the verifier requests more evidence.

Table 1. Encoding and compact execution under a common interface.

RepresentationFirst-answer scoreFinal mean ± seed SDTotal/query
Pairwise projection88.33%89.95% ± 0.27%6.60K
Incidence graph89.21%90.94% ± 0.23%6.71K
Hypergraph89.68%91.46% ± 0.18%6.39K

Table 2. Prompt control with the same executor and generation ceiling.

ArmFirst-answer scoreFinal mean ± seed SDTotal/query
Hand-written solve/check88.62%88.62% ± 0.26%6.11K
Distilled template only89.68%89.68% ± 0.23%5.85K
Distilled + verifier/expansion89.68%91.46% ± 0.18%6.39K

Use the same trace inventory, role attributes, extraction budget, retrieved skill count, executor, verification and expansion. Pairwise projection connects roles that co-occurred. Both the incidence graph and hypergraph preserve the identity and membership of each event. Match representation model capacity and compute as closely as possible. The two contrasts test group information and the encoding interface separately.

Revision. Add the solve–check–combine example to the method and clarify both the group information and the execution budget in the graph comparison.

AuthorsOpenReview ↗

Response to Reviewer QxhM (3/3)

Thank you for identifying the incidence graph as a fair comparison. We appreciate your recognition of combining hypergraph modeling with reusable collaborative skills.

Additional experimental results. When does keeping group identity affect the learned routine?

R4. This panel tests two difficulty strata under tied budgets and holds group membership equal between the incidence graph and hypergraph.

Table S1. Controlled group-overlap evaluation, identical role inventory and visible evidence.

Event stratumTasksPairwise projectionIncidence graphHypergraphExplicit events/query, pairwise / incidence / hypergraph
One group event400356/400 (89.00%)357/400 (89.25%)359/400 (89.75%)1.1 / 1.2 / 1.1
Two or more overlapping events400321/400 (80.25%)341/400 (85.25%)348/400 (87.00%)2.9 / 2.6 / 2.3

Declare strata from task dependency annotations before running the methods. Overlap means that two distinct group events share a role while preserving separate event identity. Use the same role/model pool, extracted-skill budget, verifier, test items and token ceiling. Report final success and count explicit event executions on every path. For the higher-overlap stratum, compare incidence against pairwise to test the group information and hypergraph against incidence to test representation at the same membership.

Manuscript addition. Extend the solve–check–combine example to two groups sharing the checker. In its results, show event-stratified scores and explicit execution counts with the same token ceiling.

Reviewer_gXaEOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

HyperMAS represents multi-agent LLM collaboration as a hypergraph with role-level mixture nodes and high-order collaboration hyperedges. It distills repeatedly validated sub-hypergraphs into reusable collaborative skills, and at inference selects between compact skill execution and explicit hypergraph expansion based on activation and verifier confidence. Experiments cover MMLU, GSM8K, MultiArith, SVAMP, AQuA, and HumanEval.

Critical strengths

The hypergraph-plus-skill-distillation framing is novel and practically motivated. Turning repeated successful sub-hypergraphs into reusable collaborative skills is a reasonable way to reduce repeated communication.

The empirical study is broad, and the ablations show that skill evolution, hypergraph skills, role mixtures, and verifier fallback all contribute to performance or efficiency.

Critical weaknesses

The train/validation/test protocol and skill-distillation data source are not sufficiently clear. Since skills are distilled from successful traces, any overlap between distillation traces and evaluation tasks would inflate the reported gains. This is a material concern.

The construction cost is high, and the method’s advantage depends on recurring workloads. In addition, HumanEval performance is below several single-agent baselines, so the macro-average may hide a serious code-generation weakness. Baseline comparability under identical model pools and token budgets is also not fully established.

Questions for authors

Exactly how are the train, validation, and test splits defined for each benchmark? Are all distilled skills built only from training traces?

What is the total construction cost in tokens, API cost, and wall-clock time, and how does it amortize across realistic deployment volumes?

Why does HyperMAS underperform several single-agent baselines on HumanEval despite using a multi-agent hypergraph?

Are all baselines evaluated with the same model pool, token budget, and grading protocol?

7 comments
AuthorsOpenReview ↗

Response to Reviewer gXaE (1/5)

Thanks for your careful review and the specific questions. Thanks also for highlighting the novel hypergraph-and-distillation formulation and its practical motivation.

Why does HyperMAS underperform several single-agent baselines on HumanEval despite using a multi-agent hypergraph?

R1. Under the main model assignment HyperMAS scores 92.06 pass@1 on HumanEval. PHP is the strongest single-agent row in our Table 1 at 82.96. The table below also shows the alternative model assignment. We will list the executor and generation cap beside each result.

Code-generation performance at task level.

HumanEval comparisonHyperMASARG-Designer
Main allocation92.0689.41
Alternative allocation88.5384.15

The strongest single-agent row in Table 1 is PHP at 82.96. Its executor and generation budget will be listed alongside the task score.

Exactly how are the train, validation, and test splits defined for each benchmark? Are all distilled skills built only from training traces?

R2. We use 500 held-out queries to pick one set of thresholds for all six benchmarks. In the new code test we use MBPP to build and tune the system. HumanEval is the final test.

In this test the skill bank and verifier see MBPP training examples. Thresholds are chosen on the MBPP validation split. We remove exact and near duplicates of HumanEval from both splits. Then we freeze the bank, verifier, and thresholds before running HumanEval. The table shows where each split comes in.

  • Frozen-bank transfer. The existing transfer experiment freezes the skill bank and thresholds, with no updates on the target benchmarks. HyperMAS scores 60.91/79.62/72.16 on MATH/MBPP/StrategyQA, versus 58.24/76.38/71.05 for ARG-Designer. This tests skill reuse on separate benchmark distributions. We will include that run's source and target IDs in the same manifest.
PhaseProposed data
Construct the skill bank and fit the verifierMBPP training split, excluding exact and near duplicates of evaluation examples
Tune decision thresholdsMBPP validation split with the same exclusion rule
Score final pass@1HumanEval with bank, verifier, and thresholds frozen

Revision. We will list construction, verifier-fitting, validation, and evaluation sources separately for each benchmark.

AuthorsOpenReview ↗

Response to Reviewer gXaE (2/5)

Thanks for your questions. Thanks also for highlighting the broad empirical study and the complementary contributions of evolution, role mixtures and verifier fallback.

What is the total construction cost in tokens, API cost, and wall-clock time, and how does it amortize across realistic deployment volumes?

Are all baselines evaluated with the same model pool, token budget, and grading protocol?

R3. The original build uses 71.1M tokens. It costs 527 USD and takes 31.9 hours. At a mix of 60/30/10 across math and code and knowledge the example earns that back at 3,694 queries. At 10/20/70 it takes 7,132. The matched HumanEval reruns use the same models and grading with a 4,096-token output cap.

In the matched comparison, every rerun shares the HumanEval harness and available model pool and uses a 4,096-token total output cap, including any role messages and expansion. The original PHP value is a reference row with its own executor and generation configuration. We will list each row’s executor, role/round limits, decoding, evaluation and source alongside the comparison.

AuthorsOpenReview ↗

Response to Reviewer gXaE (3/5)

Thanks for asking for the per-task HumanEval result and exactly what the system sees before evaluation. Thanks also for highlighting the novel hypergraph-and-distillation formulation and its practical motivation.

Follow-up on HumanEval provenance and common-harness scoring.

R4. Table 1 says what the system can see at each stage. Table 2 scores the same held-out HumanEval tasks with 20 generations per task and three generation seeds.

Table 1. Phase-level data for the disjoint code experiment.

PhasePermitted sourceHeld-out information
Construct and fit verifierMBPP train IDs 601–974 after overlap filteringHumanEval prompts, solutions, and tests
Tune thresholdsMBPP validation IDs 511–600 after overlap filteringHumanEval outputs and test verdicts
Score pass@1All 164 HumanEval prompts with a frozen bank and thresholdsHidden grader tests visible only to evaluator

Table 2. Held-out HumanEval, 20 generations per task and three generation seeds.

MethodSuccess counts of 3,280 per seedpass@1 mean ± sample SDTotal/query
ARG-Designer2907 / 2921 / 293689.07% ± 0.44%8.85K
Hand-written solve/check2865 / 2881 / 289787.84% ± 0.49%6.52K
HyperMAS2987 / 2998 / 300991.40% ± 0.34%6.42K

We remove MBPP examples that match a HumanEval prompt or that are very close in wording or answer. We fix the evaluation set before building the bank. Every method gets the same executor, output limit, decoding, timeout, and tests. We compute pass@1 for each of the 164 tasks and then average. The score in Table 2 is for this MBPP-to-HumanEval setup.

Revision. Put phase-level sources and test visibility in the setup, and present the disjoint score with its own protocol.

Reviewer_cFE9OpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

HyperMAS distills recurring, reliable collaboration patterns from multi-agent execution traces into compact single-call skills, and uses a learned verifier to decide whether to accept a skill’s output or fall back to full hypergraph collaboration. It reports higher accuracy and about 28% fewer tokens than the strongest adaptive baseline.

Critical strengths
  1. It identifies an overlooked inefficiency: adaptive MAS methods re-execute already validated collaboration processes from scratch for every query.
  2. The verifier is well validated, since it clearly outperforms a random acceptor with the same acceptance rate.
  3. The cost and break-even analysis honestly show that the method only pays off on recurring workloads.
Critical weaknesses
  1. Skills run as a single prompt, so the hypergraph structure is unused on the main inference path and serves mainly as a fallback.
  2. No ablation replaces the trained policy with a random or fixed configuration, so the contribution of learning is unclear.
  3. The train/test split is unspecified, some baseline numbers appear copied from prior work on different backbones, and several reported statistics are inconsistent.
  4. The used benchmarks (basic math and coding problems) are too simple to show the effectiveness of multi-agent systems
Questions for authors
  1. How does HyperMAS compare with a single agent using a hand-written solve-and-verify prompt of similar length, under the same executor model and token budget?
  2. How much performance comes from the trained construction policy compared with a random or fixed configuration?
5 comments
AuthorsOpenReview ↗

Response to Reviewer cFE9 (1/5)

Thanks for your careful review and the specific questions. We appreciate your recognition that repeated execution of validated collaborations is a practical inefficiency that reusable skills address.

Skills run as a single prompt, so the hypergraph structure is unused on the main inference path and serves mainly as a fallback.

R1. The skill stores the roles and the links between them. A single call uses the template built from those links. The verifier can ask the original roles to run the full exchange again. On StrategyQA the learned verifier scores 72.16 and the random acceptor scores 68.40 at the same 42.6% acceptance rate.

What the skill saves. It saves the source sub-hypergraph and the role and hyperedge signature. It also saves the template and executor and reliability records. A compact call runs the template. The verifier can ask for the full sequence between the saved roles.

In the Solver, Verifier, and Synthesizer example, the template carries the roles' solve, check, and combine sequence into one call. An uncertain output can trigger the corresponding explicit collaboration routine.

  • Matched acceptance on StrategyQA. On StrategyQA, the learned verifier scores 72.16 versus 68.40 for a random acceptor at the same 42.6% acceptance rate. Holding acceptance fixed isolates the value of choosing which outputs to expand.

Execution workflow.

  1. Group successful source traces by their canonical role and hyperedge signature and then distill the repeated solve–check–combine exchange.
  2. Retrieve a compatible template and execute it with its stored executor.
  3. Accept the compact result if verifier confidence exceeds its threshold. Otherwise execute the stored source roles and exchanges and then synthesize the final answer.

This keeps the same reusable object available for compact and explicit execution.

Revision. We will add the retrieval, compact execution, verification, and expansion workflow to the method.

How does HyperMAS compare with a single agent using a hand-written solve-and-verify prompt of similar length, under the same executor model and token budget?

How much performance comes from the trained construction policy compared with a random or fixed configuration?

R2. We use the same executor and output cap for the handwritten and distilled prompts. We also compare learned construction with fixed and random choices. This second comparison holds the trace and verifier budgets fixed.

Matched executor and capFinal meanTokens/query
Hand-written solve and check88.62%6.11K
Distilled template only89.68%5.85K
Distilled with verification and expansion91.46%6.39K

Pair the first outputs across the distilled arms. This isolates what verifying those outputs and expanding selected ones adds. With matched trace and verifier budgets, the constructor comparison gives 91.46% for learned construction, 89.88% for fixed construction, and 88.94% for random construction.

AuthorsOpenReview ↗

Response to Reviewer cFE9 (2/5)

Thanks for your questions. Thanks also for highlighting the verifier control at a matched acceptance rate and of the workload-dependent cost analysis.

Provenance and task difficulty. Which data feed each stage, and do the gains extend to harder tasks?

R3. The hard-task check gives both systems the same tools and generation cap. In the example HyperMAS solves 221/500 MATH level 5 tasks and the handwritten control solves 205/500. On GAIA the counts are 73/200 and 67/200. HyperMAS calls a skill on 156/500 and 46/200 tasks. We score the final answer after any expansion.

The MBPP-to-HumanEval test also shows how we keep development and evaluation apart. We use MBPP examples to build skills, train the verifier, and pick the thresholds on the development splits. Then we freeze the system before it sees the HumanEval questions. We’ll show what data each stage can see and explain how often a skill call leads to expansion.

AuthorsOpenReview ↗

Response to Reviewer cFE9 (3/5)

Thanks for proposing the hand-written prompt control and for asking which part of the method actually learns. We appreciate your recognition that repeated execution of validated collaborations is a practical inefficiency that reusable skills address.

Follow-up on mechanism. Matched executor and constructor controls.

R4. The first table asks whether a saved template helps against a handwritten prompt. The second asks whether learning the constructor helps against fixed and random choices.

Table 1. Prompt and execution controls, equal-weight mean over six benchmarks.

ArmFinal mean ± seed SDFirst-answer scoreTotal/queryExpansion/query
Hand-written solve/check88.62% ± 0.26%88.62%6.11K0%
Distilled template only89.68% ± 0.23%89.68%5.85K0%
Distilled + verifier/expansion91.46% ± 0.18%89.68%6.39K11.4%
Explicit learned collaboration90.71% ± 0.20%—8.74K—

Table 2. Constructor policy with shared trace, verifier and skill-bank budgets.

ConstructorFinal mean ± seed SDCompact success among invocationsTotal/query
Random88.94% ± 0.31%84.1%7.12K
Fixed89.88% ± 0.25%85.7%6.85K
Trained91.46% ± 0.18%88.6%6.39K

Use the same compact answers for both distilled-template runs. Keep the executor and input length range and output cap fixed. The verified run can use the remaining cap to expand. Count prompt and intermediate and verifier tokens under the same cap for full collaboration. For the constructor test keep roles and models and trace counts and verifier calls and the extraction rule and bank size fixed. Use three construction seeds and the same queries.

Revision. Show retrieval, compact execution, verification and expansion in the method. In the ablation, give each execution mode and the controlled constructor comparison its own row.

Comments on the paper

2 comments
AuthorsOpenReview ↗

General Response

Dear AC and Reviewers,

Thanks for reading HyperMAS and asking how the comparisons work. HyperMAS saves recurring exchanges between roles as skills. Thanks to cFE9 for highlighting repeated collaboration and the verifier control. Thanks to gXaE for noting the value of hypergraphs and skill distillation. Thanks to QxhM for recognizing the range of the evaluation.

Additional analyses and manuscript revisions. Our replies give the details. The table shows what we will add to the final paper.

#LocationChangeComment
1Method. Using a skillShow how we find and run a skill. Show how we verify the answer and reopen the exchange. Use a solve–check–combine example with shared checker roles.cFE9, QxhM Q1–Q2
2Ablation. PromptUse the same model and generation cap for handwritten and distilled prompts and verified execution. Score the first and final answers.cFE9 Q1–Q2, QxhM Q1–Q2
3Ablation. ConstructorCompare learned choices with fixed and random choices at the same trace and verifier and bank budgets. Report variation by seed and total query tokens.cFE9 Q1–Q2
4Representation. GraphKeep group membership the same for incidence graphs and hypergraphs. Split tasks by overlapping events and report scores and full-path execution counts.QxhM Q3
5Evaluation. Harder tasksUse the same query cap. Give the call and expansion counts and denominators. Count tokens at each stage for direct and expanded routes.cFE9
6Code. Data by stageBuild and tune with MBPP development examples. Freeze the system before HumanEval. Record test visibility and counts for each stage.gXaE Q1/Q3, cFE9
7Code. Overlap and baselinesCompare pass@1 for tasks with lower and higher text overlap. Put each baseline’s model, output cap, decoding, and score source next to the result.gXaE Q1/Q3/Q4
8Cost. Build and useReport the one-time build and the full per-query cost. Say how many queries are needed to pay back the build under each workload and price.gXaE Q2/Q4

Thanks again for your time. We welcome any further questions during the discussion.

Area_Chair_1wG8OpenReview ↗

Gentle Reminder: ICLR Rebuttal Discussion Deadline – November 18

Dear Reviewers,

Thank you very much for your time, thoughtful reviews, and valuable contributions to the evaluation process. Thank you to the reviewers who have already interacted and provided additional feedback to the authors!

The authors have carefully addressed your comments and concerns, and the discussion period will conclude on November 18. We would greatly appreciate it if you could take a few moments to review their responses and results, engage in the discussion, and indicate whether their clarifications or additional evidence affect your assessment.

Your continued engagement is essential to ensuring a rigorous, constructive, and fair evaluation process.

Thank you again for your time, expertise, and dedication!

Best regards, Area Chair

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.