- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes SHARP for credit assignment in tool-using multi-agent LLMs. The method combines the accuracy reward, the Monte Carlo Shapley marginal credit reward, and the tool format reward, and optimizes each agent based on agent-specific group-relative advantage. To compute marginal credit, the authors sample agent coalitions, remove trajectory segments generated by agents not in the coalition, and perform counterfactual rerollouts for the affected downstream agents; the difference in the correctness of the final answer before and after including a particular agent represents its marginal contribution. The paper was trained on MuSiQue and evaluated on MuSiQue, GAIA-text, WebWalkerQA, FRAMES, and DocMath-Eval. Compared to baselines such as MATPO, SHARP reports performance improvements.
Critical strengths
- The method integrates counterfactual coalition evaluation into the training process of tool-using multi-agent LLMs, addresses a clearly defined problem, and requires no additional LLM judge. Table 2 indicates that this reward plays a practical role in the current system.
- The evaluation covers multiple tasks and reports the relationship between sampling budget and training overhead. In Table 1, the average score for the Qwen3-8B model improved from 30.74 under MATPO to 35.63 under SHARP. These results support the method’s practical potential.
Critical weaknesses
- The core attribution claim lacks independent verification. Table 2 shows that the marginal credit reward improves accuracy, but it does not test whether per-agent credit is accurate. Furthermore, Figure 4 defines useful and harmful agents based on Shapley credit itself, and thus is insufficient to independently support the paper’s conclusion regarding improved coordination.
- Insufficient necessity and novelty. Overall, SHARP is a direct combination of existing reward components and group-relative optimization, with the primary increment being the introduction of Monte Carlo Shapley credit. This increment is relatively limited, and the experiments only validated the benefits of adding marginal rewards without comparing the method to simple credit approaches with comparable costs. Therefore, the unique benefits of the Shapley design remain unclear.
Questions for authors
- Given the same backbone, workflow, and counterfactual evaluation budget, what are the accuracy and training overhead of Shapley credit compared to leave-one-out credit and other simple local credits?
- Do the signs and rankings of per-agent credit align with the changes in success rates caused by online agent removal with multiple random seeds on the held-out task? Can the “useful/harmful” ratio shown in Figure 4 be reproduced using independent metrics?
- After removing a planner or an intermediate agent, does counterfactual rerollout preserve the original call path? How large is the variance in credit when the same coalition is generated repeatedly?
4 comments
Q3. What happens to downstream calls after an intervention?
The dependency structure in Sec. 4.3 and Appendix C identifies which downstream executions used information from the removed segment. These dependencies arise primarily because workers execute subtasks assigned by the planner and rely on the instructions and context it provides. In full downstream rerollout, removing a planner segment or an intermediate call invalidates those executions. We discard the affected outputs and execute the eligible downstream steps again under the modified context. This allows the new execution to adapt its search, reasoning, and subsequent calls.
Comparison with fixed segment replacement. The fixed response baseline substitutes a preset segment and reuses the original intermediate steps. If a retained step relied on the original segment, its output can still carry information that the intervention was intended to remove. Regenerating the affected steps from the edited context addresses this dependency.
| Method | MuSiQue (%) | GAIA-text (%) | WebWalkerQA (%) | FRAMES (%) |
|---|---|---|---|---|
| Fixed segment replacement | 47.29 | 34.42 | 8.85 | 35.89 |
| SHARP | 54.58 | 37.63 | 12.85 | 37.48 |
SHARP achieved higher accuracy on all four benchmarks.
We will incorporate your valuable feedback into the revised PDF and upload the updated version shortly.
Thank the authors for their additional experiments, which addressed some of my questions. However, I still believe the innovation is limited and insufficient to meet the acceptance criteria of ICLR. Thus, I will maintain the initial score.
Thank you for reviewing our paper and sharing your concerns about its novelty. We would like to explain the motivation behind SHARP and the contribution we aim to make.LLM agents are developing at an extraordinary pace, bringing new forms of collaboration and new learning challenges. Credit assignment remains an important, incompletely resolved problem. Alongside practical solutions, we believe the field needs clear formal models and interpretable approaches grounded in theory. Connecting these emerging problems with the knowledge accumulated in cooperative game theory and multiagent learning is a direction we consider deeply worthwhile.
SHARP grew out of our experience in collective optimization and multiagent cooperation. We sought to bring that experience into LLM agent learning through a simple and effective approach with an explicit mathematical foundation that others can examine and build upon. We hope to be bridge builders between these research communities. For researchers in traditional optimization, LLM agents offer new questions and opportunities to extend established ideas. For researchers in LLM agents, those ideas offer tools for organizing problems and explaining mechanisms. We hope to help knowledge travel in both directions, giving researchers with different backgrounds a shared language and new ways to contribute.
This connection matters to us because a rapidly advancing field benefits from continuity with earlier research, while its emerging challenges can also inspire established disciplines. Helping these communities learn from one another is a contribution we sincerely wish to make. We respect the importance of originality and hope SHARP can also be considered for the problem formulation and concrete modeling work involved in building this connection, alongside the familiarity of its foundations. We would sincerely appreciate your reconsideration of its contribution from this perspective.
Q1. How does SHARP compare with simpler credit rules ?
Table 2 measures the effect of adding marginal credit. To test the choice of credit rule, we compare SHARP with full team leave one out (LOO), a fixed response difference reward, and uniform coalition weighting. Let vi(S) denote the expected terminal accuracy for a coalition S of calls from trajectory i. LOO measures vi(Mi)−vi(Mi∖{m}) for the target call m, where Mi is the full set of calls. The fixed response baseline replaces the target call's output with a preset response and reuses the original downstream steps. Uniform weighting averages the marginal effect of adding m with equal weight per subset. SHARP gives each coalition size equal total weight, as in Eq. (12).
The runs used the same backbone and workflow, with eight NVIDIA A100 GPUs for 180 updates each. We kept end to end training costs as close as practicable and estimate GPU hours for each method from the mean step time.
Full team LOO measures a call’s marginal contribution only when all other calls are present, so it cannot capture how that contribution varies across different sets of collaborators. The fixed response baseline reuses downstream steps, which limits how it captures changes in later searches or reasoning. Uniform weighting evaluates multiple collaborator sets and puts more total weight on sizes with more subsets. SHARP reruns affected downstream calls and gives each coalition size equal total weight, so a call can receive credit for being useful with fewer collaborators even when it is redundant in the full team.
Q2. How stable are the credit estimates, and how do they compare with online removal?
Stability across random seeds. We evaluated 315 subagent calls on MuSiQue. We fixed the reference trajectories and sampled coalitions and recomputed Shapley credit using three random seeds. The three estimates agreed in sign for 282 calls (89.5%). Across the three seed pairs, Spearman rank correlations ranged from 0.709 to 0.757, with a mean of 0.736.
Variance when repeating a fixed coalition. For each of 589 target call and collaborator set pairs (m,S), we evaluated S and S∪{m} under each of the three seeds and computed the marginal credit Yb(S∪{m})−Yb(S), where Yb is the terminal accuracy score. We computed the sample variance of the three differences for each pair. The three differences were identical for 545 of 589 pairs (92.5%). After averaging across the sampled coalitions with Shapley weights, the mean variance across seeds was 0.01693 over the 315 calls.
Independent evaluation with online LOO. Intermediate subagent calls have no ground truth labels indicating whether they help or harm task success. Asking an LLM judge to assess intermediate outputs would introduce subjective judgments and could make the labels sensitive to the judge and its prompt. We therefore re-evaluate subagent credit for all three systems using online LOO. We classify each call as helpful, harmful, or having no observed effect based on how removing it changes final task success. This gives us a way to test whether the coordination patterns in Fig. 4 hold under a criterion independent of Shapley credit.
Under online LOO, SHARP had the highest proportion of helpful calls and the lowest proportion of harmful calls. This shows that SHARP’s advantage also holds under an evaluation based on final task outcomes, independent of Shapley credit.