acceptodds
Under review as a conference paper at ICLR 2027

MatchCredit: State-Matched Credit Assignment for Adaptive Multi-Agent Reasoning

Abstract

Multi-agent reasoning (MAR) can improve reasoning reliability through additional collaboration, but adaptive routing requires determining which collaboration operator is worthwhile at each intermediate state. Existing query-level labels or trajectory-level returns provide only coarse supervision for this decision, as the effect of an operator is confounded by the reached interaction state and subsequent routing choices. We propose MatchCredit, a state-matched credit assignment framework for efficient adaptive MAR. MatchCredit expands alternative operators from the same reached state and evaluates their downstream continuations under a shared prefix, thereby reducing prefix-state confounding. Terminal correctness and additional suffix cost are then converted into state-local pairwise operator preferences to train a lightweight router. Collaboration trees are used only for offline supervision construction, while test-time reasoning follows a single adaptive trajectory. Experiments on six reasoning benchmarks show that MatchCredit substantially reduces inference cost while preserving or modestly improving reasoning accuracy, yielding a favorable accuracy–cost trade-off over representative MAR methods. Averaged across the six benchmarks, MatchCredit achieves 86.66% accuracy with 2,193 tokens per query.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Large Language Model Agents, Multi-Agent Systems, Credit Assignment

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_SRzDOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes MatchCredit, a state-matched credit assignment framework for adaptive multi-agent reasoning. The core idea is to compare different collaboration operators from the same intermediate state and construct pairwise preferences based on downstream correctness and inference cost. A lightweight router is then trained to select appropriate operators. Experiments on six reasoning benchmarks demonstrate a favorable accuracy–cost trade-off compared with existing multi-agent reasoning methods.

Critical strengths
  1. The motivation is clear and well presented. The proposed state-matched credit assignment provides an intuitive way to address the limitations of trajectory-level supervision.
  2. The method is relatively simple and well motivated. The combination of shared-prefix comparisons and pairwise preference learning is easy to understand.
Critical weaknesses
  1. The generalization ability of MatchCredit remains somewhat limited. Although the appendix provides preliminary results on cross-dataset transfer and a unified router, these evaluations are restricted to reasoning benchmarks. Its applicability to more diverse and open-ended tasks remains unclear.
  2. The collaboration setting is relatively limited. The method only considers three operators (Accept, Debate, and Vote), while real-world multi-agent systems often involve more diverse collaboration strategies.
  3. The benchmarks are somewhat outdated, with several (e.g., MMLU, GSM8K, and MATH-500) being relatively saturated.
Questions for authors
  1. How reliable is the optimistic credit assignment used for Debate? The method selects the best sampled downstream continuation rather than estimating the expected outcome. Could this lead to overestimating the value of Debate, particularly when successful continuations are rare?
  2. The appendix shows that the unified router achieves comparable accuracy but requires more inference tokens than benchmark-specific routers. What causes this efficiency gap, and could it be reduced through improved training strategies?
Reviewer_hrsxOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

MatchCredit builds shared-prefix collaboration trees and assigns state-matched credits to sibling operators. It compares Accept, Debate, and Vote from the same reached state, constructs correctness-prioritized suffix credits, and trains a pairwise Bradley-Terry router. Experiments on six reasoning benchmarks show competitive average accuracy with lower token use than several multi-agent and routing baselines.

Critical strengths

The state-matched sibling comparison is a sensible credit-assignment design. It reduces the confounding that arises when complete trajectories differ at multiple decisions.

The empirical results are reasonably strong on the tested reasoning benchmarks, and the ablations support the importance of correctness-prioritized credit and state-conditioned routing.

Critical weaknesses

The operator set is very narrow: only Accept, Debate, and Vote. The paper claims to treat heterogeneous collaboration mechanisms as callable operators, but the actual adaptive space is limited, which weakens the generality of the contribution.

The router is benchmark-specific and trained with only 80 training questions per benchmark, with a single main model pair. Statistical significance is not reported, and several gains may be within noise. There are also notation errors, e.g., Eq. 22 uses wθ while the text defines uθ, which reduces confidence in careful execution.

Questions for authors

Why restrict the operator set to Accept/Debate/Vote, and how would the method scale to richer collaboration operators?

How does a single cross-domain router compare to benchmark-specific routers in accuracy, token cost, and training overhead?

Are the reported differences statistically significant across seeds?

How much does performance improve with more than 80 training questions per benchmark?

Please correct the notation inconsistency in Eq. 22 and clarify the router scoring function.

Reviewer_VdfVOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a framework matchcredit to enable adaptive multi agent reasoning (debate, invoke additional agents etc). It constructs shared prefix collab trees to assign local credit to alternate operators based on downstream correctness, cost, and trains a router using Bradley Terry preferences. Experiments across six benchmarks demonstrate a favorable accuracy cost tradeoff, achieving 86.66% avg accuracy with 2,193 tokens per query.

Critical strengths
  1. The paper is timely in terms of the problem direction: multi agent credit assignment ,assigning credit to intermediate collaboration decisions rather than relying solely on trajectory level outcomes.
  2. Strong accuracy efficiency trade-off - MatchCredit achieves 86.66% average accuracy with 2,193 tokens per query.
  3. Lightweight and practical inference. The proposed router is a small MLP, and the expensive tree expansion occurs offline.
  4. The paper seems well written and easy to follow.
Critical weaknesses
  1. Limited novelty - The idea of using counterfactual actions has been explored in the literature and the papers idea is just how to applied and execution details (router vs agent training, message vs operator substitute). Beyond that the Shared-prefix trees, backward credit propagation and adaptive collaboration are not novel. The authors also donot make a strong case explaining why these peices use is intuitive beyond variation/mere combination for improved performance. These methods are not included as baselines. The results are 1 table two semi tables and one graph and a lot of filler text in mainpaper.

  2. Questionable choices and eval setup - points eg the paper reports that 97.27% of GSM8K questions and 93.21% of MMLU questions terminate at initial accept - implying a simple router might achieve similar performance. Outdated models like Gpt 4.1 deepseek V3 and llama 4, non agentic evaluation setup + easy already saturated benchmarks like MATH500, GSM8K, GPQA etc

  3. The authors also donot present any form of controlled baselines - do not isolate the benefit of their credit-assignment procedure from simpler policies trained on the same search data

  4. Agentic motivation, non agentic setup - Despite framing the work as adaptive multi-agent reasoning, the actual environment consists of: two LLMs answering the same static question and a router deciding debate, accept vote etc. This is not what multi agent reasoning means in ICLR 2027 cycle.

Questions for authors

Every weakness induces a question and a todo for rebuttal, beyond which I have the following

  1. Does the optimistic credit assignment overestimate the value of Debate? Could you compare against expected value?

  2. Could the authors evaluate a simple agreement-based router, such as Accept on consensus and Debate/Vote otherwise? How much does the learned router improve over this heuristic?

  3. Does it scale beyond 2 heterogenous debate (communicating) agents? like simply or needs much modification?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.