- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes a multi-round LLM-based multi-agent framework with meta, subagent, and controller roles, and introduces MAMR-GRPO, which combines trajectory-level feedback with role-specific credits based on round values and counterfactual output deletion.
Critical strengths
None
Critical weaknesses
The architectural contribution is insufficiently justified: The paper presents the meta, subagent, and controller structure as a contribution, but similar planning, execution, and coordination structures already exist. The related work does not sufficiently clarify this distinction. More importantly, the paper provides limited intuition or theoretical justification for why this role separation is beneficial, and the experiments do not isolate its effect. Comparing the complete frameworks does not establish that the proposed three-role design explains the improvement.
The proposed credit design lacks sufficient supporting evidence: The paper introduces several credit definitions, but does not adequately justify these choices or demonstrate their benefits. Joint deletion in Eq. 9 is one example: its motivation is described, but the claimed benefit is not sufficiently validated. The concern is not simply that one particular comparison is missing. Overall, the paper proposes several design choices without enough analysis to support them. Given this complexity, the improvements in Table 1 are not sufficiently compelling to establish the significance of the work. I am not convinced that the contribution and supporting evidence are sufficient for publication at ICLR.
Questions for authors
What evidence supports the benefits of the proposed role separation and credit design, beyond the performance of the complete system?
2 comments
Response to Reviewer h9cX (2/2)
3. Evidence beyond the complete system (Question)
(a) Alternative worker-credit constructions. Appendix E.3 replaces the worker-credit construction within the shared execution setting:
- C3 SubMessage Lite, a C3-style leave-one-out credit over alternative worker messages, evaluated through the downstream Controller-to-answer path: 39.7 average EM.
- SHARP-MC, a Monte Carlo approximation to Shapley-style marginal contributions over worker permutations: 40.9 average EM.
MAMR-GRPO attains 46.7, exceeding these variants by 7.0 and 5.8 points, and both variants score below the same-framework GRPO baseline (43.6). The tested alternative worker-credit constructions do not reproduce the gain of MAMR-GRPO. This compares credit constructions within one execution setting, which is precisely evidence beyond a comparison of complete systems.
(b) Breadth of the improvement. Relative to same-framework GRPO, Table 1 reports gains on 2Wiki (+6.6), Bamboogle (+5.6), HotpotQA (+3.4), MuSiQue (+4.7), PopQA (+1.2), and TriviaQA (+0.7); NQ changes by -0.6. Bamboogle, PopQA, and TriviaQA are not part of the four-dataset RL training mixture.
(c) Collaboration within a fixed trained model. The inference-budget studies vary the execution limits while keeping the trained checkpoint fixed. Increasing the worker cap from two to four at a round cap of three raises average EM from 45.43 to 46.71; increasing the round cap from one to two at a worker cap of four raises it from 44.07 to 46.21. These comparisons show that parallel and multi-round execution are used productively by the trained system.
We thank the reviewer once again for the careful review and for encouraging further component-level comparisons. The reviewer's comments prompted us to state the contribution of the paper more precisely, and we are grateful for this. We sincerely apologize that, owing to limited GPU resources, we were unable to run additional training ablations during the rebuttal period. Such ablations would add a finer-grained view, while the evidence above, namely the fixed-graph comparison and the alternative credit constructions in Appendix E.3, already supports the role-structured credit design beyond the performance of the complete system.
Response to Reviewer h9cX (1/2)
We sincerely thank the reviewer for the time and effort devoted to reviewing our paper, for examining the role separation and the credit design, and for asking what evidence supports them beyond the performance of the complete system. This is a fair and important question, and we appreciate the opportunity to set out the evidence clearly.
The paper contains three kinds of such evidence: a training comparison with the execution graph fixed, alternative worker-credit constructions within the same execution setting, and collaboration-budget comparisons within a fixed trained model. We clarify below what each establishes.
1. Role separation: the main gain is measured with the execution graph fixed (Weakness 1)
What we claim as the contribution. Our methodological contribution is the definition of execution-structured credit: which quantity is evaluated, which reference it is compared against, and which generated segment receives the resulting signal. The revised contribution list foregrounds these definitions and identifies the framework as their execution setting.
Same-graph training comparison. Table 1 compares MAMR-Framework + GRPO (43.6 average EM) with MAMR-GRPO (46.7) on the same execution graph and Qwen3-4B backbone; both are trained on the same data for the same 780 optimizer steps. The 3.1-point improvement therefore compares training methods while the role structure is held fixed; it is not obtained by switching between frameworks. The concern that a comparison of complete frameworks cannot explain the improvement does not apply to this result.
Execution setting before RL. Separately, the inference-only comparison uses the same backbone before any RL: MAMR-Framework attains 37.2 average EM, compared with 28.8 for SingleAgent and 32.2 for MATPO-Framework. This measures the execution setting independently of RL training.
2. Credit design: three credits defined through one value quantity (Weakness 2)
The common value. The common quantity V is the expected task reward of answers sampled from the current policy after a virtual forced stop at a collaboration boundary. The three credits use this quantity with references matched to the flow of information:
Joint deletion (Eq. 9). Joint deletion addresses a concrete redundancy case. If two of three workers provide interchangeable evidence, removing either alone can leave V unchanged, whereas removing both lowers it; sampling joint deletions exposes this set contribution. Appendix D.4 specifies the budgeted procedure, which includes individual deletions and samples additional proper subsets.
Terminal advantage and coefficients. The terminal group advantage is retained. The terminal mixing coefficient and the three role-credit coefficients are all set to one. Value estimation uses task-answer rewards and requires neither a learned critic nor an LLM judge.