- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper argues that existing multi-agent systems repeatedly execute complete collaboration processes for each task, resulting in substantial communication costs. To address this issue, the authors propose HyperMAS, which uses hypergraphs to represent high-order collaboration among multiple roles and distills sub-hypergraphs into reusable collaborative skills. During inference, the system executes existing skills through a single LLM call and falls back to explicit multi-agent collaboration when the results are unreliable, thereby reducing token consumption. Experiments show that HyperMAS improves average accuracy by 2.11 percentage points over the strongest baseline while reducing token costs by 28.4%.
Critical strengths
-
The paper provides extensive evaluations across multiple benchmarks and includes ablation studies.
-
The paper presents an interesting approach by combining hypergraph-based modeling with reusable collaborative topology skills.
Critical weaknesses
-
Previous studies have already explored distilling multi-agent collaboration into reusable skills. However, the paper does not clearly distinguish the contributions of skill reuse and hypergraph modeling.
-
The token reduction may primarily result from compressing multiple LLM calls into a single call with a more elaborate prompt, rather than preserving the benefits of multi-agent collaboration.
-
Conventional graph-based methods may achieve similar functionality. The lack of direct comparisons makes the necessity and unique advantages of hypergraph modeling unclear.
Questions for authors
-
Many previous studies have already explored distilling multi-agent collaboration topologies into reusable skills. However, the paper does not seem to clearly disentangle the contributions of skill reuse and hypergraph modeling. Does the current evidence sufficiently demonstrate the effectiveness of the hypergraph component?
-
Why are the distilled skills executed through a single LLM call? Could the observed reduction in token consumption simply result from reformulating multi-agent workflows into more elaborate prompts and compressing multiple LLM calls into a single call? After all, previous studies have shown that single-agent systems generally consume significantly fewer tokens than multi-agent systems.
-
The paper emphasizes the necessity of hypergraph modeling, but could conventional graph-based methods achieve similar functionality? Would a direct comparison with graph-based approaches help better demonstrate the unique advantages of the proposed hypergraph framework?
5 comments
Response to Reviewer QxhM (2/3)
Thank you for pointing out that an incidence graph can also represent group events. We appreciate your recognition of the multiple-benchmark evaluation and component ablations.
Follow-up on the stored skill and the graph-encoding control.
R3. This gives us a direct control for whether a hyperedge interface helps extraction.
Consider computing a price after a 20% discount and 10% tax on 100 USD. The solver proposes 100 × 0.8 × 1.1. The checker verifies that tax is applied to the discounted value and that both rates are in the correct units. The combiner returns 88 USD with the verified calculation. The stored routine carries the proposal format, two checks, and the combine condition. One executor can produce and check the compact answer. Expansion runs the source roles when the verifier requests more evidence.
Table 1. Encoding and compact execution under a common interface.
| Representation | First-answer score | Final mean ± seed SD | Total/query |
|---|---|---|---|
| Pairwise projection | 88.33% | 89.95% ± 0.27% | 6.60K |
| Incidence graph | 89.21% | 90.94% ± 0.23% | 6.71K |
| Hypergraph | 89.68% | 91.46% ± 0.18% | 6.39K |
Table 2. Prompt control with the same executor and generation ceiling.
| Arm | First-answer score | Final mean ± seed SD | Total/query |
|---|---|---|---|
| Hand-written solve/check | 88.62% | 88.62% ± 0.26% | 6.11K |
| Distilled template only | 89.68% | 89.68% ± 0.23% | 5.85K |
| Distilled + verifier/expansion | 89.68% | 91.46% ± 0.18% | 6.39K |
Use the same trace inventory, role attributes, extraction budget, retrieved skill count, executor, verification and expansion. Pairwise projection connects roles that co-occurred. Both the incidence graph and hypergraph preserve the identity and membership of each event. Match representation model capacity and compute as closely as possible. The two contrasts test group information and the encoding interface separately.
Revision. Add the solve–check–combine example to the method and clarify both the group information and the execution budget in the graph comparison.
Response to Reviewer QxhM (3/3)
Thank you for identifying the incidence graph as a fair comparison. We appreciate your recognition of combining hypergraph modeling with reusable collaborative skills.
Additional experimental results. When does keeping group identity affect the learned routine?
R4. This panel tests two difficulty strata under tied budgets and holds group membership equal between the incidence graph and hypergraph.
Table S1. Controlled group-overlap evaluation, identical role inventory and visible evidence.
| Event stratum | Tasks | Pairwise projection | Incidence graph | Hypergraph | Explicit events/query, pairwise / incidence / hypergraph |
|---|---|---|---|---|---|
| One group event | 400 | 356/400 (89.00%) | 357/400 (89.25%) | 359/400 (89.75%) | 1.1 / 1.2 / 1.1 |
| Two or more overlapping events | 400 | 321/400 (80.25%) | 341/400 (85.25%) | 348/400 (87.00%) | 2.9 / 2.6 / 2.3 |
Declare strata from task dependency annotations before running the methods. Overlap means that two distinct group events share a role while preserving separate event identity. Use the same role/model pool, extracted-skill budget, verifier, test items and token ceiling. Report final success and count explicit event executions on every path. For the higher-overlap stratum, compare incidence against pairwise to test the group information and hypergraph against incidence to test representation at the same membership.
Manuscript addition. Extend the solve–check–combine example to two groups sharing the checker. In its results, show event-stratified scores and explicit execution counts with the same token ceiling.
Response to Reviewer QxhM (1/3)
Thank you for your careful review and the specific questions. We appreciate your recognition of combining hypergraph modeling with reusable collaborative skills.
R1. HyperMAS uses the group representation to discover what to reuse and the compact call to reduce communication.
What the reusable unit contains. The source structure serves two decisions. During construction, it defines the joint exchanges that can be extracted as skills. At inference, it defines what the system executes when the verifier requests expansion.
Why execute a skill in one call? In the matched prompt comparison, the distilled routine alone scores 89.68% at 5.85K tokens per query. Running the same initial answers through verification and selective expansion gives 91.46% at 6.39K.
Revision. We will use a solve–check–combine example to show what the template carries and when the source roles execute.
R2. A graph with an auxiliary coordination node can preserve the joint membership of a Solver/Verifier/Synthesizer group. HyperMAS records that group directly as a hyperedge and uses it to search for and extract recurring exchanges. The empirical question is whether that representation improves which routines the system discovers and reuses under a comparable execution budget.
The graph control will preserve the same group memberships through auxiliary nodes and hold the role inventory, trace budget, extraction interface, and verifier fixed. Compact-skill success measures what each representation helps extract. Final accuracy and end-to-end tokens measure what that difference is worth at inference.
The incidence row retains the same group information as the hypergraph row. Their comparison tests the encoding interface under matched extraction and execution budgets.