- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The paper introduces MetaAgent-X, an end-to-end reinforcement learning framework that jointly optimizes automatic MAS design and execution, since it claims that recent works either perform training-free test-time search or optimize the meta-level designer with frozen executors. The method contains two components: Executor Designer Hierarchical Rollout and Stagewise Co-evolution. The hierarchical rollout procedure samples M designs for every query. Each sampled design is executed N times, with decomposed GRPO advantages computed for each run. Stagewise co-evolution alternates every K steps. One role’s trajectories update the policy at each interval, so the two roles do not send gradients simultaneously. The main setup uses a single shared backbone, and prompts distinguish the roles. Training begins with a supervised cold start using DeepSeek-V3.2 trajectories. For Qwen3-8B and Qwen3-4B, the RL model achieves a higher six-benchmark average than both the single-agent baseline and automatic-MAS baselines. The ablation studies compare hierarchical rollout against flatter allocation, stagewise updates against coupled updates, etc.
Critical strengths
The paper notes the co-evolution of designers and executors in multi-agent systems, which may have a wide range of applications in the construction of MAS. It proposes a method that has the same backbone for both writing and executing workflows. For better credit assignment and stable training, it separates the advantages into two sets: "whether the design is good" and "whether this execution is good", and uses stagewise alternating updates since simultaneous updates will collapse into repeated tokens. The experimental ablation is relatively comprehensive, which basically demonstrates the function of all designed components.
Critical weaknesses
- The writing is somewhat hard to follow, with some grammar errors. e.g., in line 16 "while keeping downstream execution agents frozen, which creating a frozen-executor ceiling and leaving the end-to-end training of self-designing and self-executing agentic models unexplored"; in line 48 "recent automatic MAS begin to embrace these paradigms, their transition remains incomplete."; in line 75 "The dynamics by which designer and executor could co-evolve under joint training, and where each role’s improvement remains unclear in practice and in understanding the mechanism." is not a complete sentence; in line 194 "we build a training framework contains predefined coordination structures"; The objective is notationally inconsistent, so its dependencies are unclear. Equation (1) defines the return as , with and
.
Equations (4)–(7) and Appendix B instead write , dropping and reversing the argument order. Equation (8) writes and conditions the executor only on , as in , so the query disappears from both the policy and the return. The objective is also denoted while the surrounding text uses , , and . The paper does not state explicitly how relates to the parameters in the shared- and split-policy settings.
-
Inconsistent experimental data analysis: "Search-based Auto MAS baselines degrade when instantiated with these backbones: ADAS falls −6.81 points to 20.35% on 8B, and AFlow reaches only 22.52% on 4B." The fact that AFlow outperforms the single-agent baseline (21.38) on the 4B model cannot be regarded as degradation.
-
Logical contradiction: The last paragraph of Section 3.3 reverses its own causal claim. The stagewise schedule is introduced to isolate each phase to one reward distribution and to reduce gradient interference between the two roles. The next sentence then describes the co-evolutionary loop and concludes, “As a result, the effective reward distribution becomes non-stationary, and the two role-specific objectives can produce noisy or conflicting updates.” As written, “as a result” attributes non-stationarity and conflicting updates to the proposed loop, which contradicts the stated motivation. If the claim is that the environment remains non-stationary even under alternation, that contrast should be explicit. If it instead describes simultaneous updates, it should not follow the positive description of the stagewise loop.
-
The hierarchical-rollout ablation does not match the budget it claims to control. Table 3 and Section 4.3 state that rollout variants are matched at M × N = 16 trajectories per query, but the flat variant is M=8, N=1, i.e., 8 trajectories, against M=4, N=4.
-
The paper does not clarify how the Separate Policy is updated. If strictly following the ablation setup, the Separate Policy is also updated in a stagewise manner. However, the Separate Policy has no issue of mutual gradient interference at all. Stagewise updates are likely a side effect, and the results of normal updates for each Separate Policy should be compared.
-
Regarding the fairness of the experiments, the method proposed in the paper adopts SFT, while other RL-based methods do not. The paper also fails to compare the performance of its own method when only RL is applied, or the performance of other RL-based methods using SFT-preprocessed models. This introduces a certain unfairness. Since SFT clearly brings performance gains, and as it learns from the outputs of a stronger model, it most likely amplifies the performance improvement brought by the subsequent RL stage. These aspects are not analyzed in the paper.
-
The paper only identifies three types of workflow topologies. For "automatically designing multi-agent systems", this action space is too small and lacks sufficient persuasiveness. Tables 6 and 7 categorize all workflows into single, reflection, and ensemble. The labels in the case studies also belong to these three categories: math_single_agent, ensemble_judge, and reflection; code_solver_unittest in the code examples is still marked as reflection. The template library in Appendix A also mentions solver-tester and "and etc.", yet the subsequent text provides no distributions for agent counts, edges, or control flow graphs. The authors refer to Table 7 as the coarse skeleton and state that 54 role names appeared during evaluation, 77.5% of prompts were unique per problem, and scripts were also generated per problem. These points illustrate that role names and prompts within the same skeleton vary, but they do not prove that the topology changes. All 54 names may be attached to solver–critic or ensemble–judge. On APPS, single accounts for 55.2%, meaning that more than half of these outputs are not even multi-agent. The routing modified by reinforcement learning is also limited: ensemble drops by roughly 30 percentage points, mostly shifting to reflection. This is more similar to selecting among three predefined templates and using an executor to run the chosen template more effectively, rather than openly designing multi-agent organizations.
Questions for authors
-
For the hierarchical-rollout ablation, which configuration was actually run? Table 3 lists M=8, N=1 against M=4, N=4, while the caption says both use M × N = 16 trajectories per query. If the flat run used 8 trajectories, the reported gain cannot be attributed to how the budget is allocated.
-
Was the separate-policy run also updated stagewise? Separate parameters do not write both roles' gradients into one backbone, so stagewise masking can itself discard half of each role's updates.
-
Did any training-based baseline receive the same DeepSeek-V3.2 trajectories? If not, how much of the gain over those baselines remains after giving them the same cold start?