- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes C3, a method for credit assignment in cooperative LLM agent teams. When the execution trace are restored, C3 evaluates alternative messages by rerunning the workflow to one decision point and observing terminal rewards. It estimates how much each message contributes to the final outcome and examines the reliability of these estimates. Experiments on mathematical reasoning demonstrate improvements compared with MAPPO and MAGRPO.
Critical strengths
- The paper is clearly written and easy to follow. Rerunning the downstream workflow with alternative messages provides a direct way to estimate each message’s contribution to the final outcome.
- The method is effective on the evaluated math tasks. The experiments demonstrate improvements in both credit estimation and downstream training performance.
Critical weaknesses
- My main concern is that all tasks focus on mathematical tasks, and the collaborative environments are relatively simple. Math is naturally not an agent task. The authors should employ the methods on practical agent tasks such as coding and search.
- Another concern is the infrastructure overhead. When applying the method to actual coding or search environments, restoring the environment and replaying multiple alternative executions may place additional demands on the infrastructure. The authors should discuss this.
Questions for authors
The authors are suggested in addressing our main concern by demonstrating C3’s effectiveness on practical agent tasks.
5 comments
Follow-up: practical agent tasks
Thank you again for your review. Does Table R6 answer your main question on practical agent tasks?
On a coding team (MBPP) and a search team (HotpotQA), C3's split-half reliability is 0.81 and 0.94, against 0.64 on math. Appendix B.10 reports the measured cost. If anything remains open, we will answer it before the discussion closes on 18 November.
Thanks again for your valuable feedback!
A gentle reminder that today is the last day for discussion. We are happy to take any follow-up questions. In case you are satisfied with our rebuttal, we would really appreciate acknowledgement and raising the score.
Response: W1 & W2, practical agent tasks and the cost of restoring their environments
Thank you for the suggestions on practical agent tasks and infrastructure. Under W1 we run C3 on a coding team and a search team, and under W2 we measure what restoring their environments costs.
W1. Practical agent tasks.
Response. We ran C3 on both tasks you name. On a coding team (MBPP), the program of every continuation is executed against its unit tests in a sandbox. On a search team (HotpotQA), every continuation queries a BM25 retrieval index. C3's credit is more reliable on both than on math. Split-half reliability is 0.81 on code and 0.94 on search, against 0.64 on math, and rank correlation with an independent 16-continuation reference is 0.89 and 0.94, against 0.69. Nothing in C3 is specific to math. Its unbiasedness rests on what the trace records and what the reset restores (Definition 1). The revised Section 6 states, tool by tool, what a team must record and restore.
W2. Infrastructure overhead.
Response. In training, C3 runs each question to the terminal reward 8 times, as MAPPO and MAGRPO do (Section 5). Its runs branch from a restored decision point instead of each starting from the task, so the model generates fewer messages. In math training at a 2048-token limit, C3 uses 37% fewer tokens than MAPPO. On the coding team, all 1,168 continuations, unit tests included, ran in 227 seconds on 1 GPU. On the search team, the 2,768 continuations ran in 108 seconds, and their 5,536 retrievals took 3.0 seconds in total.
The revised Section 6 states how each kind of tool is handled:
If the coding and search results or the cost of restoring environments leave a question open, we will answer it during the discussion.