acceptodds
Withdrawn from ICLR 2027

POET: Prior-Aware Preference Optimization for Multi-Agent Communication Topology Generation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

LLM-based multi-agent systems rely on communication topologies to organize agent participation and information exchange. Successful and failed executions establish graph preferences, yet using these preferences to learn task-adaptive topologies requires interpreting structural differences in the context of task difficulty. We introduce POET (Prior-aware Topology Preference Optimization), a framework for learning task-adaptive communication topologies through prior-guided graph preferences. POET interprets graph complexity relative to task difficulty and uses role diversity to distinguish team compositions beyond size and connectivity. These priors guide the selection of successful and failed graph pairs within each query and their influence in a weighted preference objective, while execution outcomes determine the preference direction. This turns recorded executions into task-aware supervision for the topology generator without additional LLM calls during preference learning. Experiments across six benchmarks show that POET achieves the highest average accuracy among the evaluated methods, while the learned generators transfer to a different LLM backbone without retraining.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
LLM-based Multi-Agent Systems, Graph Learning, Preference Optimization

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_wzhqOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes POET, a prior-aware preference optimization framework for generating task-adaptive communication topologies in LLM-based multi-agent systems. It constructs success–failure graph preferences from recorded executions and uses task difficulty, graph complexity, and role diversity to guide pair sampling and loss weighting. The approach trains an autoregressive topology generator without additional LLM calls during preference optimization. Experiments on six benchmarks report an average accuracy of 91.22%.

Critical strengths
  1. POET learns from both successful and failed topologies through a clear preference-learning formulation, reusing existing execution records without additional LLM calls during preference optimization.

  2. The paper compares several topology-design methods and includes plain-DPO ablations, query-conditioning tests, fixed-topology controls, and cross-model transfer evaluations, which help assess different aspects of the approach.

Critical weaknesses
  1. POET relies on relatively small preference datasets, with only five MMLU training queries containing both successful and failed executions. The paper also does not report variability across multiple runs, making it difficult to assess the stability of the reported improvements.

  2. On GSM8K, the fixed Tree-4 topology achieves 93.98% accuracy compared with POET's 94.06%, while consuming considerably fewer tokens. This raises concerns about whether the additional complexity of adaptive topology generation is justified by its performance gains.

  3. The component ablations are mainly conducted on MMLU and GSM8K, while the other benchmarks provide limited evidence about the individual contributions of task difficulty, graph complexity, and role diversity.

Questions for authors
  1. Could the authors report results across multiple random seeds to demonstrate the stability of POET's improvements, particularly given the limited number of training queries with mixed execution outcomes?

  2. Given that Tree-4 achieves nearly identical accuracy with fewer tokens on GSM8K, does POET retain an advantage when compared under similar inference budgets?

  3. Since POET generates larger teams after preference optimization, how much of the improvement comes from task-specific role and communication structures rather than simply using more agents?

Reviewer_hAHNOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies the broad problem of communication topologies in multi-agent systems. Building on existing generative methods, it focuses on incorporating task execution feedback from different topology structures into a generative model. POET introduces a preference-learning strategy based on task difficulty, graph complexity, and role diversity to fine-tune an existing multi-agent topology generator, ARG-Designer. The preference-learning stage reuses recorded executions without additional LLM calls.

The main idea is to use task difficulty, graph complexity, and role diversity to guide preference-data selection and modify the DPO objective for training. I view its core contribution as a sampling method. The paper evaluates the approach on six benchmarks. Although it does not strictly outperform other methods on every benchmark, it achieves the best average performance among the methods compared.

Critical strengths

The paper provides a concrete, practical method for topology generation using successful and failed records from multi-agent execution histories. By incorporating task difficulty, graph complexity, and role diversity into preference-pair construction and loss weighting, POET makes structured use of existing execution feedback and extends the previous positive-only training paradigm. The experimental results show that the proposed data construction method and weighted loss can improve the existing generator, with average performance exceeding all evaluated baselines. The ablation experiments further support the effectiveness of the combined approach.

An additional practical feature is that the preference-learning stage directly reuses recorded execution histories without requiring additional LLM calls.

Critical weaknesses
  1. The mechanisms and necessity of the individual priors have not been sufficiently established.

The current experiments demonstrate the effectiveness of the combined strategy in the settings examined, but do not sufficiently explain why each prior is effective. Specifically, the existing results are insufficient to establish whether the particular alignment between task difficulty and graph complexity is necessary. They also do not sufficiently distinguish the independent contributions of role-diversity-based sampling and loss weighting. Moreover, Table 6 shows that the full sampler increases the complexity gap between successful and failed topologies in both difficulty groups. How this result supports the original difficulty-dependent interpretation of the mechanism requires further clarification. These issues affect my assessment of the method's design rationale and the actual contribution of each prior.

  1. The practical value of learned topologies relative to fixed topologies remains unclear.

Figure 4 shows that POET achieves 94.06% accuracy on GSM8K, while the fixed Tree-4 topology reaches 93.98%, a difference of only one correctly answered question, yet POET uses more tokens. This raises substantial concerns about the method's practical value. Although an analysis on a single benchmark is insufficient to draw a general conclusion, the fixed-topology comparisons currently report results only on this benchmark. I therefore remain uncertain whether task-adaptive topology generation can deliver consistent benefits in accuracy and cost.

Questions for authors
  1. Necessity and independent contributions of the priors.

Could the authors provide controlled comparisons to test whether this particular alignment between task difficulty and graph complexity is necessary? For example, while keeping the remaining sampling and optimization settings unchanged, the authors could shuffle difficulty values across training queries, use a constant difficulty value, or reverse the direction of the complexity preference. Separately evaluating role-diversity-based sampling and role-diversity-based loss weighting would also help clarify their independent contributions. Furthermore, given that the full sampler increases the complexity gap in both difficulty groups, could the authors clarify how the results in Table 6 support the original difficulty-dependent interpretation of the mechanism?

  1. Practical value of learned topologies relative to fixed alternatives.

With the execution model, role pool, prompts, and evaluation protocol strictly aligned, could the authors extend the fixed-topology comparisons to one or two additional representative benchmarks, such as MMLU and HumanEval? Reporting both accuracy and token consumption would help determine whether task-adaptive topology generation offers consistent advantages beyond GSM8K.

Reviewer_XBkROpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a preference optimization framework for learning task-adaptive multi-agent communication topologies from recorded execution outcomes. It designs success-failure graph pairs to feed the training of an autoregressive topology generator. Experiments across six benchmarks show improved accuracy over existing topology design methods.

Critical strengths
  1. This paper presents a simple and computationally practical approach for topology generation for multi-agent systems.

  2. It provides preliminary evidence of the method's advantages by working on several reasoning and math benchmarks.

Critical weaknesses
  1. The contribution is incremental and the underlying techniques are not fundamentally new. Recent multi-agent topology optimization methods already exploit execution outcomes or task-dependent information to guide graph generation, and this paper fails to highlight its intrinsic novelty beyond related work. While task difficulty does not necessarily imply that more complex collaboration structures are beneficial, the rationale for defining the difficulty-conditioned complexity and role-diversity sampling and weighting needs special justification.

  2. More modern benchmarks are preferred for evaluation, and results from multiple runs that demonstrate statistically significant improvement are necessary.

Questions for authors

See weaknesses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.