- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes RACA, a role-aware credit assignment method for multi-agent LLM reasoning, to address inaccurate reward assignment across collaborating agents. It combines role-specific local rewards, context-aware advantage grouping, and separate credit assignment for different token spans. Using Qwen3-8B, RACA achieves 64.5% accuracy on MATH Level 5 and 16.0% on AIME, outperforming the compared baselines.
Critical strengths
- Reasonable motivation and method design.
Traditional Outcome GRPO assigns the same final reward to different agents, which may not reflect their individual contributions. RACA assigns credit based on agent roles and reasoning contexts, and separates solution generation from assistance-seeking decisions. The design is reasonable and has some novelty.
- Relatively comprehensive experimental comparisons.
The paper compares RACA with relevant methods, including AT-GRPO and GiGPO, and provides ablation studies for its main components. RACA achieves a clear accuracy improvement on MATH Level 5 while using fewer model calls than other adaptive multi-agent methods.
Critical weaknesses
- Limited performance gains and generalization evidence.
RACA improves performance on MATH, but its accuracy is only 2.1 percentage points higher than Single-Agent GRPO + Self-Consistency. On AIME, the improvement over the strongest baseline is only 1.3 percentage points on a test set of 150 problems, making it difficult to establish whether the gain is stable and statistically significant. In addition, the experiments only use Qwen3-8B on mathematical reasoning tasks, without evaluation across different model sizes or other tasks.
- Reliance on manually designed rewards and insufficient evidence of effective collaboration.
RACA relies on manually designed reward rules and weights, which may not accurately reflect each agent's actual contribution. For example, when the Proposer gives an incorrect answer, requesting assistance receives a higher reward (−0.02) than not requesting assistance (−0.10), even if the request fails to correct the answer. This may encourage unnecessary interactions rather than decisions based on their actual value. Similarly, the Critic's reward partly depends on whether the answer is later corrected, although a successful correction does not necessarily mean that the Critic made a causal contribution.
Although the ablation studies support the effectiveness of the credit assignment components, RACA still requests assistance more frequently for correct answers (63%) than for incorrect answers (58%). This suggests that the learned interaction policy remains imperfect. Therefore, while the method improves accuracy, the evidence is insufficient to establish that it achieves accurate credit assignment and effective selective collaboration.
Questions for authors
The improvement on AIME over the strongest baseline is relatively small. Could the authors provide additional statistical analysis to clarify whether this improvement is stable and significant?
Could the authors compare RACA with stronger single-model reasoning baselines under a similar inference compute budget, such as Qwen3-8B Thinking, to better demonstrate the practical benefit of multi-agent collaboration?
3 comments
W02: Reward design and evidence of effective collaboration
The interaction advantage is computed separately for initially correct and incorrect answers, because its comparison groups are conditioned on (Section 3.3). The reviewer’s comparison between −0.02 and −0.10 therefore concerns only the incorrect-answer groups. It does not determine the reward ordering for correct answers.
| Initial answer | Interaction outcome | Interaction reward |
|---|---|---|
| Incorrect | Request, successfully repaired | +0.28 |
| Incorrect | Request, repair unsuccessful | −0.02 |
| Incorrect | No request | −0.10 |
| Correct | Request | −0.07 |
| Correct | No request | 0 |
Within incorrect-answer groups, an unsuccessful request receives −0.02, compared with −0.10 for making no request; successful repair receives +0.28. This intentionally favors attempting assistance when the current answer is wrong, while rewarding successful assistance more strongly. Within correct-answer groups, the preference is reversed: requesting receives −0.07, whereas not requesting receives 0. These groups are normalized separately, so the reward design does not impose a uniform preference for interaction across both correctness conditions.
For the critic, Section 3.2 explicitly defines the repair signal as reflecting “observed local outcomes, not a counterfactual causal effect.” Eq. (2) combines correct error detection with the subsequent short-horizon repair outcome: false alarms are penalized, and the repair bonus is available only after a correctly flagged error. This provides locally grounded feedback from the original rollout without requiring counterfactual executions. A subsequent correction is therefore used as a training signal, not as proof that the critic independently caused the correction.
The empirical contribution of the credit-assignment mechanism is examined while holding the reward rules fixed. In Table 4, removing context conditioning reduces MATH/AIME accuracy by 4.7/2.7 points, and replacing separate functional channels with a single turn-level advantage reduces accuracy by 3.9/2.0 points. These comparisons show that the gains depend on how feedback is compared and applied, beyond introducing manually specified rewards.
Regarding interaction selectivity, Table 2 shows that RACA achieves the highest interaction rate on incorrect answers (58% versus 23–39%) and the smallest gap between correctness conditions (5 points versus 17–47 points) among the compared methods.
To further examine the remaining gap, we separate proposer-initiated requests across all reasoning rounds by their target, excluding forced interactions and mechanically scheduled calls:
| Request target | Request rate given an incorrect answer | Request rate given a correct answer | Fix rate | Break rate |
|---|---|---|---|---|
| Critic | 50.6% | 49.3% | 25.7% | 3.6% |
| Verifier | 7.4% | 13.7% | 12.0% | 1.2% |
The request rates are conditioned on answer correctness before interaction. Fix and break rates measure correctness changes by the end of the requested interaction chain within the same reasoning round.
The higher overall interaction rate after correct answers is associated primarily with verifier requests, which are 6.3 percentage points more frequent after correct answers. Critic requests are instead 1.3 points more frequent after incorrect answers. Thus, the aggregate 63% versus 58% combines different target-selection behaviors. The outcomes also differ: critic-initiated chains exhibit a higher fix rate, while verifier-initiated chains exhibit a lower break rate.
Figure 3(b) provides complementary evidence from training: requests after correct answers decrease substantially, while requests after incorrect answers initially decline and then partially recover. The overall interaction rate also declines. The narrowing gap therefore accompanies reduced interaction on already-correct answers, rather than a uniform increase in assistance-seeking.
These results support improved interaction behavior relative to the baselines. We retain the limitation explicitly acknowledged in Section 4.3: RACA’s assistance policy remains imperfect.
Q01: AIME statistical analysis
We have added three-seed evaluation and question-level paired significance tests on the expanded 450-problem AIME benchmark. The complete results are:
| Training seed | RACA | GiGPO | Difference | RACA-only / GiGPO-only correct | McNemar | Holm-adjusted |
|---|---|---|---|---|---|---|
| 42 | 87/450 (19.3%) | 71/450 (15.8%) | +3.56 pp | 19 / 3 | 0.0009 | 0.0017 |
| 123 | 97/450 (21.6%) | 79/450 (17.6%) | +4.00 pp | 22 / 4 | 0.0005 | 0.0016 |
| 2026 | 91/450 (20.2%) | 81/450 (18.0%) | +2.22 pp | 14 / 4 | 0.0309 | 0.0309 |
| Mean ± SD | 20.4 ± 1.1% | 17.1 ± 1.2% | +3.26 ± 0.93 pp | — | — | — |
We use two-sided exact McNemar tests, comparing the two methods’ predictions on the same questions within each seed. Holm correction is applied to the unrounded p-values across the three seed-specific comparisons. Means and sample standard deviations are computed across training seeds; repeated evaluations of the same questions are not treated as additional independent problems.
RACA improves over GiGPO for all three seeds, with gains ranging from 2.22 to 4.00 percentage points, and every comparison remains significant after correction. These results provide evidence of both training-seed consistency and statistical significance on the expanded benchmark.
These tests concern the expanded evaluation; we do not infer significance for the original 150-problem comparison from its aggregate means and standard deviations. Further evaluation details are provided in our response to Weakness 1.
Q02: Stronger single-model reasoning baseline
We have added the requested Qwen3-8B Thinking comparisons, evaluating both the SFT policy and its Self-Consistency variant with . The MATH Level-5 results are:
| Method | Accuracy | LLM calls | Input tokens | Output tokens |
|---|---|---|---|---|
| Qwen3-8B Thinking (SFT) | 47.1% | 1.0 | 284.8 | 715.5 |
| Qwen3-8B Thinking + Self-Consistency () | 58.4% | 8.0 | 2,278.4 | 4,683.2 |
| RACA | 64.5% | 7.4 | 3,617.4 | 2,014.3 |
Calls and token counts are averaged per problem.
Compared with Thinking + Self-Consistency, RACA improves accuracy by 6.1 percentage points, while using 7.5% fewer model calls and approximately 57.0% fewer generated tokens. RACA processes more input tokens because subsequent agents receive accumulated reasoning and feedback through the blackboard; the table reports this cost explicitly.
These additional evaluations address the requested Thinking-mode comparison and support RACA’s practical accuracy–cost benefits under the evaluated settings, alongside the controlled credit-assignment comparisons within the shared multi-agent framework.
W01: Performance gains and generalization evidence
Thank you for your careful review and constructive feedback.
We have strengthened the AIME evaluation with an expanded 450-problem benchmark, three independent training seeds, and paired significance tests. We also clarify the accuracy–cost comparison with the strongest single-agent baseline.
On MATH, the 2.1-point improvement over Single-Agent GRPO + Self-Consistency is accompanied by fewer model calls and generated tokens. Table 3 reports:
Thus, RACA’s improvement is achieved with fewer calls and approximately 52% fewer output tokens, although it processes more input tokens. Within the same four-role framework, RACA additionally improves over Outcome GRPO by 6.4 percentage points, providing a controlled comparison of credit-assignment mechanisms.
For AIME, we agree that the original 1.3-point difference on 150 problems requires further statistical evidence. The expanded evaluation gives:
The improvement is positive across all three seeds and statistically significant for each seed under two-sided exact McNemar tests with Holm correction across the three comparisons. Repeated evaluations of the same questions are not counted as additional independent problems.
Regarding generalization, AIME evaluates zero-shot transfer beyond the MATH training dataset: AIME evaluation problems are excluded from SFT and RL, and no solved AIME demonstrations are supplied at inference time. This provides cross-dataset evidence within mathematical reasoning. RACA’s credit-assignment formulation does not depend on Qwen-specific architectural features, but our empirical validation currently covers Qwen3-8B and mathematical tasks; broader cross-model and cross-task evaluation remains future work.