acceptodds
Withdrawn from ICLR 2027

MAGIC: Mixed-Granularity Agent Graphs via Incremental Construction with Dense-Reward Reinforcement Learning

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Collaboration topology determines the performance and execution cost of multi agent systems based on large language models. Existing representative topology generation methods typically adopt a single collaboration granularity, using either individual agents throughout the topology for fine grained control or predefined groups to reuse established collaboration patterns. However, even within the same task, different functional roles may require substantially different levels of collaboration. A fixed collaboration granularity therefore fails to capture the diverse collaboration demands across functional roles. Our key insight is to flexibly select the collaboration granularity for each functional role, thereby combining the fine grained control of individual agents with the reusable collaboration patterns of predefined groups. Introducing mixed granularity, however, exponentially enlarges the search space for topology generation. Existing topology generation methods typically rely only on final task outcomes as reinforcement learning rewards. Such sparse rewards provide insufficient intermediate feedback in the enlarged mixed granularity search space, which can lead to unstable convergence or slow topology generation. To address this challenge, we propose MAGIC, a dense reward reinforcement learning framework for mixed granularity graph generation. MAGIC constructs a mixed granularity MAS topology incrementally. For each functional role, it determines whether the role should be instantiated as an individual agent or a reusable group, and then connects the instantiated unit to the existing MAS topology. We directly optimize the construction policy using returns from trajectories sampled by the current policy. We further introduce a reward shaping mechanism based on structural complexity, structural redundancy, and utility. This mechanism provides intermediate feedback through sampled utility signals and structural signals while preserving the cumulative task reward. Experimental results show that MAGIC achieves the highest mean score among the evaluated methods across eight benchmarks and achieves strong inference efficiency in the efficiency evaluation.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Multi-Agent Systems, Mixed-Granularity Organization, Graph Generation, Reinforcement Learning, Large Language Models

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_1SkgOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes MAGIC, which progressively generates multi-agent collaboration structures based on the input question, selecting each node's role, whether to use a single agent or a predefined group, and how they are connected. The authors use reinforcement learning to train the graph construction policy and provide intermediate rewards based on performance on other training questions, structural complexity, and role repetition. The paper evaluates the method on eight benchmarks covering knowledge reasoning, mathematics, coding, and tabular tasks, and includes component comparisons and comparisons of training costs and inference token consumption.

Critical strengths
  1. The proposed method achieves good results on multiple tasks. The main experiments cover eight benchmarks and include three complete training and evaluation runs, with mean scores higher than those of the compared methods. The efficiency experiments on four benchmarks also show that performance improvements do not generally lead to higher inference token consumption, which adds to the method's practical value.

  2. The paper compares the effects of using single agents or agent groups at different stages and finds that not all stages require the same scale of collaboration. On the tasks tested, selectively using groups provides a better performance–cost trade-off than using single agents throughout or groups throughout. This provides useful experimental guidance on how to allocate collaboration resources.

Critical weaknesses
  1. The method's added contributions are not clear enough

    Previous work such as MaAS already includes single-agent operators such as CoT and multi-agent modules such as Debate and Ensemble in the same operator library, selecting and combining them based on the question. This paper provides both single-agent and group implementations for the same role, but does not discuss how this design differs specifically from MaAS or what additional benefits these differences bring.

    For reward design, the authors use four other training questions to evaluate the structure being constructed, but how much this feedback helps construct the graph for the current question has not been evaluated separately. The existing comparisons remove this feedback together with the terms for structural complexity and role repetition, so they do not show how much benefit comes from the four-question evaluation itself. More detailed ablations are needed to determine how much additional value the new reward provides over simpler alternatives.

  2. The stability of the key experimental results needs to be verified

    The main experiments were run three times, but Q2 and Q3 were each run only once. In Q2, the new reward improves MMLU-Pro accuracy from 75% to 76%, which is only one more correct answer out of 100 test questions; the improvement on TAT-QA is only 0.5 F1 points. Q3 compares performance at the same training cost, but also lacks repeated runs. It is not yet clear whether these gains are stable.

Questions for authors
  1. The paper already reports some training costs and inference token consumption. What are the total training token consumption, training wall-clock time, and test wall-clock time for this method and the main comparison methods? What are the corresponding model-call concurrency settings? Do the performance gains justify the cost changes introduced by the new components?

  2. Q2 and Q3 have each been run only once. If training is repeated independently, do these gains remain stable? How much do the results vary?

  3. Keeping the final task reward, how do the results compare with the full method if only structural information is added, or if only feedback from four other training questions is added? How does the training cost change in each case? How much independent benefit comes from running these four additional questions?

  4. Previous work such as MaAS already includes single-agent operators and multi-agent modules in the same operator library. How does this paper's mixed-granularity approach differ in substance from those earlier approaches?

Reviewer_AZXJOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes MAGIC, a framework for LLM-based multi-agent systems. Its starting point is that existing topology generation methods use either individual agents throughout or predefined groups throughout, whereas different functional roles within the same task may require very different levels of collaboration; the authors therefore argue for flexibly choosing a mixed granularity, individual agent or reusable group, for each role. Mixed granularity, however, exponentially enlarges the search space, and sparse rewards based only on final task outcomes are insufficient for training in it. MAGIC thus builds the topology incrementally: it decides how to instantiate each role and connects it to the existing topology, directly optimizes the construction policy with returns from trajectories sampled by the current policy, and adds reward shaping based on structural complexity, structural redundancy, and utility to provide dense intermediate feedback. Experiments show that MAGIC achieves the highest mean score across eight benchmarks while maintaining strong inference efficiency.

Critical strengths

Introduce a framework for LLM-based multi-agent systems. Experiments show that MAGIC achieves the highest mean score across eight benchmarks while maintaining strong inference efficiency.

Critical weaknesses

The abstract says sparse rewards cause “unstable convergence or slow topology generation”; there is not a single training curve in the paper to support this. “Highest on all eight benchmarks” is literally true, but GSM8K +0.48 and TAT-QA +0.79 are statistical noise given ±1.2 standard deviations. The main text concedes “gains are smaller”; the abstract and conclusion keep only “highest on all eight”. “Mixed granularity yields a better performance–cost trade-off”: in Q2, Full saves 6–8% tokens versus All-Group but uses 0.76% more tokens than All-Atomic on TAT-QA for a 0.5-point gain. That is “comparable”, not “better”. “17 baselines” sounds impressive, but half of them underperform direct inference on MMLU-Pro, which casts doubt on the quality of the comparison set. Q4’s “Pareto frontier” is drawn against only LLM-Debate, G-Designer, ARG-Designer, GoAgent and four fixed topologies; the stronger DyLAN and MasRouter from the main table are absent.

Questions for authors

See weekness

Reviewer_uKQ9OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors brings a key insight which is to flexibly select the collaboration granularity for each functional role, thereby combining the fine-grained control of individual agents with the reusable collaboration patterns of predefined groups. They also claim that existing topology generation methods typically rely only on final task outcomes as reinforcement learning rewards. Such sparse rewards provide insufficient intermediate feedback in the enlarged mixed granularity search space, which can lead to unstable convergence or slow topology generation. To address this challenge, they propose MAGIC, a dense reward reinforcement learning framework for mixed granularity graph generation.

Critical strengths
  • The experiments are sufficient with extensive comparison with baselines.
  • The illustrative figures help the readers to understand the paper.
Critical weaknesses
  • The writing of this paper needs improvement, especially in the first two sections: (1) It doesn't motivate why mixed granularity is necessary, (2) what type of agent tasks does this paper focus on (there are a ton of different types of agent tasks, and I don't think this paper abstract a common characteristic from ALL types of agent tasks), and (3) more importantly, how this affects the end-to-end performance of the agent tasks compared with not using multi-agents or using homo granularity?
Questions for authors

I have no questions in addition to the weakness I summarized above.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.