- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper formulates multi-agent collaboration as a cost-sensitive sequential decision problem. At each reasoning state, the controller chooses among exit, lightweight self-refinement, and high-cost team review. It uses capability-aware confidence features and offline counterfactual branching trees to train a fitted Q-network, while online deployment executes only the selected action. Experiments on MATH, MMLU-Pro, and SuperGPQA with three LLM scales show large token reductions and competitive accuracy.
Critical strengths
-
The cost-sensitive sequential formulation is well motivated and technically coherent. Counterfactual branching provides state-action supervision that single-path trajectories cannot provide, which is a meaningful contribution to credit assignment for adaptive collaboration.
-
The online results and ablations support the practical value of the approach: RLMAS substantially reduces token use relative to MAD and other multi-agent baselines while often preserving or improving accuracy.
Critical weaknesses
-
The training horizon is only H=2, and the action space is limited to E/L/H. The theoretical guarantees are for fixed counterfactual trees, not the underlying stochastic collaboration process, so the practical optimality claim remains narrow.
-
The offline counterfactual tree construction cost is not sufficiently integrated into the cost-benefit analysis. It is also unclear whether all baselines use identical token accounting, judge calls, and failure retries, which matters for the claimed efficiency gains.
Questions for authors
-
What is the total offline cost of constructing counterfactual trees, and at what deployment volume does it amortize?
-
How does the method behave when the decision horizon is extended beyond H=2?
-
Are token counts for all methods computed under the same protocol, including initial responses, judge/evaluator calls, and retries?
-
What is the practical implication of Theorem 3 given that it applies to fixed trees rather than the stochastic deployment environment?