- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes MatchCredit, a state-matched credit assignment framework for adaptive multi-agent reasoning. The core idea is to compare different collaboration operators from the same intermediate state and construct pairwise preferences based on downstream correctness and inference cost. A lightweight router is then trained to select appropriate operators. Experiments on six reasoning benchmarks demonstrate a favorable accuracy–cost trade-off compared with existing multi-agent reasoning methods.
Critical strengths
- The motivation is clear and well presented. The proposed state-matched credit assignment provides an intuitive way to address the limitations of trajectory-level supervision.
- The method is relatively simple and well motivated. The combination of shared-prefix comparisons and pairwise preference learning is easy to understand.
Critical weaknesses
- The generalization ability of MatchCredit remains somewhat limited. Although the appendix provides preliminary results on cross-dataset transfer and a unified router, these evaluations are restricted to reasoning benchmarks. Its applicability to more diverse and open-ended tasks remains unclear.
- The collaboration setting is relatively limited. The method only considers three operators (Accept, Debate, and Vote), while real-world multi-agent systems often involve more diverse collaboration strategies.
- The benchmarks are somewhat outdated, with several (e.g., MMLU, GSM8K, and MATH-500) being relatively saturated.
Questions for authors
- How reliable is the optimistic credit assignment used for Debate? The method selects the best sampled downstream continuation rather than estimating the expected outcome. Could this lead to overestimating the value of Debate, particularly when successful continuations are rare?
- The appendix shows that the unified router achieves comparable accuracy but requires more inference tokens than benchmark-specific routers. What causes this efficiency gap, and could it be reduced through improved training strategies?