- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces SAGE, a training-free multi-agent reasoning framework that adapts agents' reasoning strategies and communication structures. It selects a strategy donor based on answer agreement, prefix consistency, and reciprocal peer review, then transfers the donor's strategy to other agents through role-preserving system prompt rewriting. Agents subsequently collaborate through a dynamically updated sparse DAG, and the final answer is selected using weighted pool voting across reasoning stages. Experimental results show improved reasoning accuracy over the evaluated baselines.
Critical strengths
- Role-preserving strategy adaptation: SAGE transfers reasoning strategies from a selected donor to other agents while preserving their original roles. This design allows agents to adapt their reasoning strategies to each problem without sacrificing their distinct specializations.
- Simple and modular strategy transfer: SAGE uses only the selected donor's original system prompt to adapt other agents' prompts, without requiring access to the problem or generated responses. This provides a straightforward approach to strategy transfer while keeping each agent's prompt rewriting independent.
Critical weaknesses
-
Lack of computational cost analysis and cost-matched comparisons: The authors do not provide a comparison of computational costs, such as the number of LLM calls, token consumption, or inference latency. Consequently, it remains unclear whether the improvements primarily result from the proposed framework or from the additional inference budget. A cost-matched comparison with existing multi-agent methods, together with an accuracy–cost trade-off analysis, would strengthen the empirical evaluation.
-
Insufficient justification for the donor selection criteria: The proposed donor selection relies on answer agreement and prefix consistency, but it is unclear why these metrics are reliable indicators of a good strategy donor. In particular, an agent may consistently produce an incorrect answer or agree with other agents on a shared misconception. Such behavior can result in a high donor selection score without necessarily indicating that the agent has a useful reasoning strategy to transfer. Furthermore, while the authors compare their donor selection mechanism against random donor selection, they do not provide ablations isolating the contributions of answer agreement, prefix consistency, and reciprocal peer review. A more detailed analysis of how these metrics correlate with donor quality would strengthen the methodological justification.
-
Insufficient justification for the high–low peer-review strategy: The proposed framework performs reciprocal peer review between high-scoring and low-scoring agents, but the motivation for this specific pairing strategy is not sufficiently justified. Since the objective of peer review is to identify a reliable strategy donor, it is unclear why reviewing high-scoring agents against low-scoring agents would be more effective than reviewing high-scoring agents against high-scoring agents. Comparisons with high–high and random pairing under matched review budgets would help establish whether the proposed high–low pairing is necessary or beneficial.
-
Insufficient justification and ablation of the weighted voting mechanism: The proposed weighted pool voting mechanism involves several hand-designed choices, such as aggregating responses across multiple reasoning stages and assigning a higher weight (1.5) to stage leaders. However, the paper does not sufficiently justify why such a relatively complex voting mechanism is necessary instead of simple majority voting. A comparison with simple majority voting and a sensitivity analysis of the weighting parameter would help justify the proposed design.
Questions for authors
-
Figure 3 shows that SAGE generally achieves larger performance gains with smaller backbone models, while the improvements become smaller or even negative for some larger models. What do you think explains this trend? Could you provide further insights into why SAGE appears to be more effective for smaller models?
-
Could you clarify which system prompt and agent role were used for the Single baseline? Since SAGE employs specialized role prompts, I wonder whether the comparison is fair if the Single baseline uses only a simple, generic prompt. Providing the exact prompt configuration would help assess the fairness of the comparison.
3 comments
Thank you for the detailed responses and additional results. However, some of my main concerns remain unresolved.
-
W1. Computational cost:
My concern is not simply the higher cost of SAGE, but whether it would still outperform other multi-agent baselines “under the same inference budget”. Would SAGE maintain its advantage if the baselines were given the same token budget? -
W2. Donor selection:
I understand that answer agreement and prefix consistency can indicate response reliability. However, why does producing reliable answers imply that an agent has a useful reasoning strategy to “transfer” to others? This connection remains unclear. -
W3. High–low pairing:
I understand the motivation for encouraging diverse perspectives. However, this remains an intuitive justification rather than an empirically validated design choice. Comparisons with high–high or random pairing would help justify this design choice. -
W4. Weighted voting:
The new results show nearly identical accuracy between weighted and unweighted pool voting. I still question why the proposed leader-weighting mechanism is necessary when simple unweighted pool voting performs equally well. -
Q1 and Q2:
Thank you for the clarifications. These questions have been addressed.
I also noticed that there do not appear to have been any revisions to the manuscript so far. Since ICLR allows manuscript revisions during the discussion period, I encourage the authors to incorporate the additional experimental results and clarifications directly into the revised manuscript.
Thank you for the follow-up reply, we really appreciate it. We clarify your concerns below:
W1. Same inference budget
We will add budget-matched versions of the multi-agent baselines to the revised manuscript. Two results already show that SAGE's advantage does not come from extra budget alone.
First, two SAGE ablations keep compute almost fixed. SAGE-RandomDonor uses the same call budget as SAGE, and it is lower on all six benchmarks. SAGE-NoRewrite removes only the short rewrite calls, and it loses about 2 points on both backbones. So at almost the same compute, SAGE's design makes a clear difference.
Second, more tokens do not bring higher accuracy among the baselines. On both backbones, MOC uses more tokens than SelfOrg, but MOC is less accurate. Prior work also shows that more discussion does not always improve answers (Smit et al., 2024). So a larger budget alone does not guarantee better accuracy for the baselines.
W2. Why a reliable agent has a useful strategy to transfer
The link has two steps.
First, prefix consistency tells us which answer is more likely to be correct. Iwase et al. (2026) show that correct reasoning traces are more stable under regeneration than incorrect ones. When a trace is cut partway and regenerated, a correct answer comes back more often than a wrong one. This held in every setting they tested. For example, on FrontierScience-Olympiad with GPT-OSS-120B, correct answers came back 66.0% of the time and wrong answers 22.3% of the time.
Second, agents on the same backbone differ mainly in their system prompts, and each prompt defines a strategy. So when one agent's answer is the most likely to be correct, its prompt is likely the strategy that best suits the problem. That agent becomes the donor.
We also test this link directly in Appendix H.2. There, we replace the selected donor with a random teammate and keep everything else the same. The random donor gives lower accuracy on all six benchmarks. It keeps only about a quarter of the gain from rewriting. So the selected donor's strategy is more useful to transfer than a random teammate's strategy.
W3. High-low pairing
High scores come from answer agreement. So the high-scoring agents often give the same answer. If they only review each other, a different answer from the low-scoring group is never checked.
Appendix F shows this with a real case. On this MMLU question, the four agents first answer A, D, C, and C. The correct answer is D. The two agents that answer C have the highest scores. High-high pairing would only review these two C responses against each other. The D response would never enter the review. With high–low pairing, each high-scoring agent reviews against the low-scoring agents. Here, this review moved the SkepticalVerificationSolver to the correct answer, D. It was then selected as the donor. SAGE answers this question correctly in all three seeds.
W4. Leader weighting
We do not present the leader weight as a source of SAGE's gains. Within each stage, the leader's extra weight is only half a vote. So it mainly affects close votes. This is why weighted and unweighted pooling give almost the same accuracy. The difference is at most 0.03 points on both backbones. This shows that the results in the paper do not depend on this choice. We keep the weight because it adds the prefix-consistency signal to the final vote at no extra cost as these are already computed from our runs. Not reusing them would leave a small gain in accuracy on the table.
Response to Reviewer rTbz
We thank the reviewer for their careful and constructive review. We respond to each of your concerns below.
W1 (cost). Task-model calls and total tokens (thousands) per question, macro-averaged over six benchmarks and three seeds:
A high token cost relative to single-agent sampling is a known property of multi-agent systems as a class, not of SAGE alone. Prior work reports that multi-agent debate does not reliably outperform self-consistency and ensembling once cost is considered (Smit et al., ICML 2024); that a single agent with a strong prompt nearly matches the best discussion methods (Wang et al., ACL 2024); that debate methods often fail to beat CoT and self-consistency even with significantly more inference-time computation (Zhang et al., arXiv:2502.08788); and that majority voting alone accounts for most of the gains attributed to debate (Choi et al., NeurIPS 2025).
We therefore do not claim that SAGE is more compute-efficient than other MAS methods. Our claim is limited to the comparison with communication-based multi-agent methods under the same agent count and round limit.
W2 (donor criteria). Our selection metrics are intended as reliability signals rather than guarantees of correctness, as none of the existing methods can do that, during runtime. [Iwase et al. 2026] show that reasoning traces leading to correct answers are more reproducible under prefix truncation and regeneration than those leading to incorrect answers, providing empirical support for prefix consistency as a signal of answer reliability. Similarly, answer agreement follows the well-established self-consistency principle, where majority voting over independently sampled reasoning paths has been shown to improve reasoning accuracy across multiple benchmarks [Wang et al., 2023]. We acknowledge that correlated errors or shared misconceptions can still lead to high agreement. Thus, these metrics serve as probabilistic indicators of reliable and reproducible strategies, rather than definitive evidence of correctness.
W3 (high–low pairing). Thank you for this point. In our runs, the high-scoring agents often had similar roles and prompts. Because the score is based on answer agreement, they also often gave the same answer. If we paired high-scoring agents with each other, they would mostly review views close to their own, and the review would rarely change anything. Pairing high-scoring agents with low-scoring agents brings in a different role and a different line of reasoning, so each answer is tested against a real alternative before we choose the donor. This is why we chose the high–low design.
W4 (voting). Every SAGE run also saved and graded two alternative outputs from the same trajectories (macro-average, three seeds):
Our design follows Iwase et al. (2026). They showed that prefix consistency was the best predictor of correctness in most of their settings, and that weighting votes by it matches plain majority voting with far fewer samples. This matters in our setting, because a team of four agents gives only a few votes per stage. So we combine both signals instead of using majority voting alone. Each agent’s score adds answer agreement and prefix consistency (λ = 0.5). In a team of four, passing the prefix check therefore counts as much as two extra agreeing agents. The agent with the highest score in each stage leads that stage, and its vote gets weight 1.5 in the final vote. This is how prefix consistency enters the final answer, which plain majority voting would ignore. We pool answers across stages so that a correct answer found early is not lost during revision.
Q1 (Figure 3). We agree that the benefits are concentrated among smaller backbones. We believe this is because stronger backbone models are already sufficiently capable that communication in a multi-agent setting barely provides them any additional/complimentary signal for improving their responses. Weaker agents have greater scope for improvement by leveraging complementary expertise from other agents, which allows them to identify and correct mistakes they would otherwise make independently.
Q2 (Single prompt). For the Single baseline, we send the benchmark question unchanged as the only user message, with no system prompt or role. The CoT baseline is the same, except that we add “Let’s think step by step.” to the end of the question. We will add the exact prompts to Appendix E in the revision.