- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
In this paper, the authors address the problem of automated multi-agent system (MAS) construction. Specifically, they identify two key bottlenecks in prior work (e.g., MaAS): (1) training queries carry unequal informational value, yet existing methods sample them uniformly; and (2) rewards are assigned only at the whole-workflow level, providing overly coarse credit assignment for individual graph-expansion actions. To overcome these limitations, the authors propose ASPO, which integrates an Adaptive Query Selection Mechanism (AQSM) that prioritizes "competence-boundary" queries, together with a substructure-aware reward for finer-grained policy optimization. Experiments across diverse benchmarks and scenarios demonstrate the effectiveness of ASPO.
Critical strengths
- Well-motivated problem with empirical support: The introduction clearly illustrates the research problem and provides a detailed comparison with related work. Furthermore, Figure 1 and Table 2 demonstrate that both adaptive query selection and the fine-grained reward improve the performance of MAS on downstream tasks.
- Comprehensive ablation studies: Section 4.3 presents transferability experiments of the automated construction method across different tasks and LLMs, which further validate the effectiveness of ASPO. Moreover, Section 4.4 provides a cost analysis showing that ASPO achieves strong performance at a lower cost.
Critical weaknesses
- The necessity of automated MAS construction on the provided benchmarks: My main concern is the necessity of automated MAS construction for the benchmarks used in this paper. As shown in Table 1, most MAS-Search methods achieve only comparable performance to the CoT baseline, and even underperform it on MMLU-Pro, MBPP+, and HumanEval+. Although the proposed ASPO achieves the best performance, it requires an additional training phase with extra training samples, which are not available to the CoT baseline. This raises the question: is automated MAS construction truly necessary for these tasks? I suggest that the authors include stronger yet simpler baselines for comparison, such as few-shot CoT (with in-context examples) or CoT with SFT training.
- The motivation curve is measured on the training set: In the Figure 1, the 59.0% → 74.3% AQSM gain is evaluated on the full training set, where concentrating budget on medium/hard training queries trivially raises training accuracy. Please provide the same grouped curves on the held-out test set, or state explicitly that the figure is only a mechanism illustration with generalization evidenced by Table 1.
- Heuristic design of the adaptive query selection: In AQSM, the authors estimate each query's success probability using a Beta posterior. I suggest that the authors include additional comparative baselines, such as query selection based on entropy or similarity, to provide stronger empirical support for the AQSM design.
Questions for authors
See my questions in weakness.
Response to Reviewer mte7
W1: The value of automated MAS construction and comparison with stronger CoT baselines
We thank the reviewer for highlighting this concern. The failures of existing MAS-search methods directly motivate our work. Below we first analyze these failures, then compare against few-shot CoT.
Existing MAS-search methods can fail to learn effective collaboration. As detailed in our response to Reviewer k8Ff’s Q2, (i) Most MaAS and RAAS test workflows on MMLU-Pro and HumanEval+ effectively rely on a single Generate operation, consistent with repeated training on easy queries reinforcing basic generation. (ii) Puppeteer receives only about 54 policy updates on HumanEval+ and exhibits premature termination, modification without an initial solution, and repeated reflection without execution feedback. ASPO improves policy learning through adaptive query selection and substructure-level feedback. With the same backbone and operator pool, it exceeds both the strongest agent-selection baseline and CoT on all benchmarks.
ASPO also outperforms few-shot CoT with access to the same training data. We also add a retrieval-augmented few-shot CoT baseline. For each test query, it retrieves the three most similar examples from the training split and incorporates their questions and reference solutions into the prompt.
Providing retrieved demonstrations improves CoT’s average score from 71.63 to 72.40. ASPO nevertheless outperforms this stronger baseline on all six benchmarks, by 5.79 points on average. These results support the benefit of learning MAS architectures beyond supplying training examples through in-context prompting.
Revision. We add the few-shot CoT results to Table 1 and its implementation details to Appendix D.3. We also add an analysis of why existing MAS-search methods fail or underperform to Appendix D.5.
W2: Grouped learning curves on the held-out test set
Thank you for this suggestion. We now add the same grouped learning curves for 490 held-out MATH test queries in Figure 1(b). The table below reports test accuracy (%); entries for epochs 4–6 show without / with AQSM.
At epoch 6, AQSM improves medium- and hard-query accuracy by 11.9 and 5.8 percentage points, respectively, while easy-query accuracy remains 99.0% in both conditions. Overall test accuracy reaches 59.4% versus 54.3% without AQSM, showing that the gains extend to held-out queries.
Revision. We add test-set query outcomes and grouped accuracy curves to Figure 1(b), and update the Introduction to discuss both training and test results.
W3: Rationale for AQSM and comparison with entropy-based query selection
Thank you for suggesting additional query-selection baselines. AQSM combines a theoretically motivated selection score with uncertainty-aware success-rate estimation. We clarify these design choices below and add an outcome-entropy baseline to strengthen the empirical evidence.
Appendix C.3 derives, under the stated assumptions, an upper bound proportional to pq(1−pq) on the improvement in query success probability under a gradient step. This motivates prioritizing queries that are solvable but not yet reliably solved. Since these probabilities must be estimated from limited rollout outcomes, we use a Beta posterior to represent estimation uncertainty and sample from it to preserve exploration.
Following the reviewer's suggestion, we add the w/ outcome entropy variant, which samples queries in proportion to the binary entropy of their empirical success rates. It achieves an average score of 80.59 versus ASPO’s 83.36, a 2.77-point gap. The existing w/o posterior variant similarly loses 2.67 points. These results suggest that changing the weighting function alone does not address unreliable success-rate estimates under sparse observations, supporting posterior-based estimation and exploration.
Revision. We have added the w/ outcome entropy comparison to Table 3 in Section 4.5.