- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper addresses the reliance on human labels in language model self-improvement and proposes MACA, a framework that uses consensus from multi-round multi-agent debate as a label-free training signal. It post-trains models with MV-SFT, MV-GRPO, MV-DPO, and MV-KTO, and introduces online self-debate GRPO for single-agent self-improvement, improving multi-agent collaboration and single-agent reasoning on math, science, and commonsense tasks.
Critical strengths
1.The paper systematically turns debate consensus into a training signal and compares SFT, GRPO, DPO, and KTO in one framework, offering useful evidence on whether multi-round debate consensus beats single-round majority voting. 2.Online self-debate is compared with TTRL, ScPO, and others under matched rollout budgets and training settings, with added debate-protocol ablations, making the conclusions reasonably convincing.
Critical weaknesses
1.Online self-debate is currently validated mainly on Qwen3-4B and GPQA, where the test set is relatively limited and the gain is close to its standard deviation; additional results on more models and benchmarks would strengthen the conclusion. 2.The paper reports that the debate majority is incorrect on about 37.9% of training examples; clarifying how performance changes on originally correct examples would help better delineate the method's applicability.
Questions for authors
1.Are there preliminary results for online self-debate on MATH or GSM8K, and if so, are they consistent with the GPQA findings? 2.For examples with an incorrect debate majority, could the authors briefly report how accuracy changes on examples that were originally correct?