acceptodds
Under review as a conference paper at ICLR 2027

Self-Adapting Group of Experts for Multi-Agent Reasoning

24 views

Abstract

Multi-agent systems bring together language model agents with different roles to propose, review, and refine solutions. Each agent's response depends on its model's capabilities, the reasoning strategy defined by its system prompt, and the information in its input context. Existing frameworks often adapt communication by changing this context while leaving individual prompts fixed, even when a problem calls for different skills. We study whether agents' initial responses can identify a strategy better suited to the current problem and guide its transfer to other agents. To address this, we introduce **SAGE** (**S**elf-**A**dapting **G**roup of **E**xperts), a training-free framework that uses answer agreement, prefix consistency, and reciprocal peer review to select a strategy donor. **SAGE** transfers the selected donor's reasoning strategy to the other agents while preserving their original roles. This transfer uses only the agents' original system prompts, without access to the problem or generated solutions. After strategy adaptation, agents exchange responses through a dynamic, sparse directed acyclic graph that routes information from higher-scoring agents to lower-scoring agents. Experiments across multiple agent backbones and reasoning benchmarks show that **SAGE** achieves higher average accuracy than the evaluated baselines.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Multi-Agent Reasoning, Multi-Agent Collaboration, Large Language Models, Prompt Adaptation, Test-Time Reasoning, Agent Communication

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_rTbzOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces SAGE, a training-free multi-agent reasoning framework that adapts agents' reasoning strategies and communication structures. It selects a strategy donor based on answer agreement, prefix consistency, and reciprocal peer review, then transfers the donor's strategy to other agents through role-preserving system prompt rewriting. Agents subsequently collaborate through a dynamically updated sparse DAG, and the final answer is selected using weighted pool voting across reasoning stages. Experimental results show improved reasoning accuracy over the evaluated baselines.

Critical strengths
  1. Role-preserving strategy adaptation: SAGE transfers reasoning strategies from a selected donor to other agents while preserving their original roles. This design allows agents to adapt their reasoning strategies to each problem without sacrificing their distinct specializations.
  2. Simple and modular strategy transfer: SAGE uses only the selected donor's original system prompt to adapt other agents' prompts, without requiring access to the problem or generated responses. This provides a straightforward approach to strategy transfer while keeping each agent's prompt rewriting independent.
Critical weaknesses
  1. Lack of computational cost analysis and cost-matched comparisons: The authors do not provide a comparison of computational costs, such as the number of LLM calls, token consumption, or inference latency. Consequently, it remains unclear whether the improvements primarily result from the proposed framework or from the additional inference budget. A cost-matched comparison with existing multi-agent methods, together with an accuracy–cost trade-off analysis, would strengthen the empirical evaluation.

  2. Insufficient justification for the donor selection criteria: The proposed donor selection relies on answer agreement and prefix consistency, but it is unclear why these metrics are reliable indicators of a good strategy donor. In particular, an agent may consistently produce an incorrect answer or agree with other agents on a shared misconception. Such behavior can result in a high donor selection score without necessarily indicating that the agent has a useful reasoning strategy to transfer. Furthermore, while the authors compare their donor selection mechanism against random donor selection, they do not provide ablations isolating the contributions of answer agreement, prefix consistency, and reciprocal peer review. A more detailed analysis of how these metrics correlate with donor quality would strengthen the methodological justification.

  3. Insufficient justification for the high–low peer-review strategy: The proposed framework performs reciprocal peer review between high-scoring and low-scoring agents, but the motivation for this specific pairing strategy is not sufficiently justified. Since the objective of peer review is to identify a reliable strategy donor, it is unclear why reviewing high-scoring agents against low-scoring agents would be more effective than reviewing high-scoring agents against high-scoring agents. Comparisons with high–high and random pairing under matched review budgets would help establish whether the proposed high–low pairing is necessary or beneficial.

  4. Insufficient justification and ablation of the weighted voting mechanism: The proposed weighted pool voting mechanism involves several hand-designed choices, such as aggregating responses across multiple reasoning stages and assigning a higher weight (1.5) to stage leaders. However, the paper does not sufficiently justify why such a relatively complex voting mechanism is necessary instead of simple majority voting. A comparison with simple majority voting and a sensitivity analysis of the weighting parameter would help justify the proposed design.

Questions for authors
  1. Figure 3 shows that SAGE generally achieves larger performance gains with smaller backbone models, while the improvements become smaller or even negative for some larger models. What do you think explains this trend? Could you provide further insights into why SAGE appears to be more effective for smaller models?

  2. Could you clarify which system prompt and agent role were used for the Single baseline? Since SAGE employs specialized role prompts, I wonder whether the comparison is fair if the Single baseline uses only a simple, generic prompt. Providing the exact prompt configuration would help assess the fairness of the comparison.

3 comments
AuthorsOpenReview ↗

Response to Reviewer rTbz

We thank the reviewer for their careful and constructive review. We respond to each of your concerns below.

W1 (cost). Task-model calls and total tokens (thousands) per question, macro-averaged over six benchmarks and three seeds:

MethodQwen callsQwen tokensMinistral callsMinistral tokens
Single10.510.9
SelfOrg13.316.015.134.8
MOC13.020.113.036.2
MAD-M216.024.916.043.8
SAGE36.440.433.060.6

A high token cost relative to single-agent sampling is a known property of multi-agent systems as a class, not of SAGE alone. Prior work reports that multi-agent debate does not reliably outperform self-consistency and ensembling once cost is considered (Smit et al., ICML 2024); that a single agent with a strong prompt nearly matches the best discussion methods (Wang et al., ACL 2024); that debate methods often fail to beat CoT and self-consistency even with significantly more inference-time computation (Zhang et al., arXiv:2502.08788); and that majority voting alone accounts for most of the gains attributed to debate (Choi et al., NeurIPS 2025).

We therefore do not claim that SAGE is more compute-efficient than other MAS methods. Our claim is limited to the comparison with communication-based multi-agent methods under the same agent count and round limit.

W2 (donor criteria). Our selection metrics are intended as reliability signals rather than guarantees of correctness, as none of the existing methods can do that, during runtime. [Iwase et al. 2026] show that reasoning traces leading to correct answers are more reproducible under prefix truncation and regeneration than those leading to incorrect answers, providing empirical support for prefix consistency as a signal of answer reliability. Similarly, answer agreement follows the well-established self-consistency principle, where majority voting over independently sampled reasoning paths has been shown to improve reasoning accuracy across multiple benchmarks [Wang et al., 2023]. We acknowledge that correlated errors or shared misconceptions can still lead to high agreement. Thus, these metrics serve as probabilistic indicators of reliable and reproducible strategies, rather than definitive evidence of correctness.

W3 (high–low pairing). Thank you for this point. In our runs, the high-scoring agents often had similar roles and prompts. Because the score is based on answer agreement, they also often gave the same answer. If we paired high-scoring agents with each other, they would mostly review views close to their own, and the review would rarely change anything. Pairing high-scoring agents with low-scoring agents brings in a different role and a different line of reasoning, so each answer is tested against a real alternative before we choose the donor. This is why we chose the high–low design.

W4 (voting). Every SAGE run also saved and graded two alternative outputs from the same trajectories (macro-average, three seeds):

Output ruleQwenMinistral
Weighted pool vote (paper)58.3175.08
Unweighted pool vote (leader weight 1.0)58.3075.11
Best final-round agent, no vote57.4874.85

Our design follows Iwase et al. (2026). They showed that prefix consistency was the best predictor of correctness in most of their settings, and that weighting votes by it matches plain majority voting with far fewer samples. This matters in our setting, because a team of four agents gives only a few votes per stage. So we combine both signals instead of using majority voting alone. Each agent’s score adds answer agreement and prefix consistency (λ = 0.5). In a team of four, passing the prefix check therefore counts as much as two extra agreeing agents. The agent with the highest score in each stage leads that stage, and its vote gets weight 1.5 in the final vote. This is how prefix consistency enters the final answer, which plain majority voting would ignore. We pool answers across stages so that a correct answer found early is not lost during revision.

Q1 (Figure 3). We agree that the benefits are concentrated among smaller backbones. We believe this is because stronger backbone models are already sufficiently capable that communication in a multi-agent setting barely provides them any additional/complimentary signal for improving their responses. Weaker agents have greater scope for improvement by leveraging complementary expertise from other agents, which allows them to identify and correct mistakes they would otherwise make independently.

Q2 (Single prompt). For the Single baseline, we send the benchmark question unchanged as the only user message, with no system prompt or role. The CoT baseline is the same, except that we add “Let’s think step by step.” to the end of the question. We will add the exact prompts to Appendix E in the revision.

Thank you for the detailed responses and additional results. However, some of my main concerns remain unresolved.

  • W1. Computational cost:
    My concern is not simply the higher cost of SAGE, but whether it would still outperform other multi-agent baselines “under the same inference budget”. Would SAGE maintain its advantage if the baselines were given the same token budget?

  • W2. Donor selection:
    I understand that answer agreement and prefix consistency can indicate response reliability. However, why does producing reliable answers imply that an agent has a useful reasoning strategy to “transfer” to others? This connection remains unclear.

  • W3. High–low pairing:
    I understand the motivation for encouraging diverse perspectives. However, this remains an intuitive justification rather than an empirically validated design choice. Comparisons with high–high or random pairing would help justify this design choice.

  • W4. Weighted voting:
    The new results show nearly identical accuracy between weighted and unweighted pool voting. I still question why the proposed leader-weighting mechanism is necessary when simple unweighted pool voting performs equally well.

  • Q1 and Q2:
    Thank you for the clarifications. These questions have been addressed.

I also noticed that there do not appear to have been any revisions to the manuscript so far. Since ICLR allows manuscript revisions during the discussion period, I encourage the authors to incorporate the additional experimental results and clarifications directly into the revised manuscript.

Thank you for the follow-up reply, we really appreciate it. We clarify your concerns below:

W1. Same inference budget

We will add budget-matched versions of the multi-agent baselines to the revised manuscript. Two results already show that SAGE's advantage does not come from extra budget alone.

First, two SAGE ablations keep compute almost fixed. SAGE-RandomDonor uses the same call budget as SAGE, and it is lower on all six benchmarks. SAGE-NoRewrite removes only the short rewrite calls, and it loses about 2 points on both backbones. So at almost the same compute, SAGE's design makes a clear difference.

Second, more tokens do not bring higher accuracy among the baselines. On both backbones, MOC uses more tokens than SelfOrg, but MOC is less accurate. Prior work also shows that more discussion does not always improve answers (Smit et al., 2024). So a larger budget alone does not guarantee better accuracy for the baselines.

W2. Why a reliable agent has a useful strategy to transfer

The link has two steps.

First, prefix consistency tells us which answer is more likely to be correct. Iwase et al. (2026) show that correct reasoning traces are more stable under regeneration than incorrect ones. When a trace is cut partway and regenerated, a correct answer comes back more often than a wrong one. This held in every setting they tested. For example, on FrontierScience-Olympiad with GPT-OSS-120B, correct answers came back 66.0% of the time and wrong answers 22.3% of the time.

Second, agents on the same backbone differ mainly in their system prompts, and each prompt defines a strategy. So when one agent's answer is the most likely to be correct, its prompt is likely the strategy that best suits the problem. That agent becomes the donor.

We also test this link directly in Appendix H.2. There, we replace the selected donor with a random teammate and keep everything else the same. The random donor gives lower accuracy on all six benchmarks. It keeps only about a quarter of the gain from rewriting. So the selected donor's strategy is more useful to transfer than a random teammate's strategy.

W3. High-low pairing

High scores come from answer agreement. So the high-scoring agents often give the same answer. If they only review each other, a different answer from the low-scoring group is never checked.

Appendix F shows this with a real case. On this MMLU question, the four agents first answer A, D, C, and C. The correct answer is D. The two agents that answer C have the highest scores. High-high pairing would only review these two C responses against each other. The D response would never enter the review. With high–low pairing, each high-scoring agent reviews against the low-scoring agents. Here, this review moved the SkepticalVerificationSolver to the correct answer, D. It was then selected as the donor. SAGE answers this question correctly in all three seeds.

W4. Leader weighting

We do not present the leader weight as a source of SAGE's gains. Within each stage, the leader's extra weight is only half a vote. So it mainly affects close votes. This is why weighted and unweighted pooling give almost the same accuracy. The difference is at most 0.03 points on both backbones. This shows that the results in the paper do not depend on this choice. We keep the weight because it adds the prefix-consistency signal to the final vote at no extra cost as these are already computed from our runs. Not reusing them would leave a small gain in accuracy on the table.

Reviewer_LYGGOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes SAGE, a training-free multi-agent reasoning framework that adapts agents' reasoning strategies and communication structures for individual problems. SAGE selects a strategy donor using answer agreement, prefix consistency, and reciprocal peer review, transfers its reasoning instructions to other agents through prompt rewriting, and coordinates subsequent collaboration using a dynamically updated sparse DAG and weighted answer aggregation. Experiments across six reasoning benchmarks and two primary LLM backbones report improvements over existing multi-agent methods, with additional studies on model scaling, heterogeneous teams, and robustness to corrupted messages.However, I find the evidence insufficient to establish the efficiency and distinct methodological value of the complete framework, particularly given the additional inference operations, the limited validation of the proposed strategy-transfer mechanism, and the lack of isolated evidence for the dynamic graph.

Critical strengths
  1. The paper explores a relevant direction beyond conventional communication optimization by adapting agents' reasoning instructions rather than only modifying which messages they exchange. The distinction between transferring general reasoning guidance and sharing problem-specific solutions is interesting, particularly in a training-free setting.
  2. The main experiments demonstrate encouraging improvements across multiple benchmarks and model backbones. The prompt-rewriting ablation in Appendix H.1 is particularly useful, showing average accuracy gains of approximately 2.1 points on both primary backbones when strategy adaptation is enabled.
  3. The paper includes additional investigations of agent scaling, heterogeneous teams, and corrupted communication. These experiments provide useful evidence about the framework's behavior beyond the standard homogeneous multi-agent setting.
Critical weaknesses
  1. The accuracy improvements are not accompanied by sufficiently convincing computational-cost comparisons. SAGE introduces additional prefix-consistency evaluations, reciprocal peer-review calls, and prompt-rewriting operations, yet the main comparisons only match agent counts and collaboration rounds rather than total model calls or token budgets. Without compute-matched baselines, it is difficult to determine whether the reported improvements justify the substantially more involved inference procedure.
  2. I am not fully convinced that the proposed prompt adaptation demonstrates the claimed problem-specific strategy transfer. According to Eq. (6), the rewriter receives only two original system prompts, without the current problem or generated responses; consequently, the rewriting operation itself is independent of the problem once the donor is selected. The improvements in Appendix H.1 establish that rewriting is beneficial, but do not distinguish genuine strategy transfer from the effect of adding generally useful reasoning or verification instructions.
  3. The reliability of the donor-selection mechanism is not sufficiently established. Answer agreement and prefix consistency measure consensus and reproducibility rather than correctness, while reciprocal review can reflect information obtained from other agents rather than the intrinsic suitability of the selected donor's original strategy. Although Appendix H.2 shows an advantage over random donor selection, it does not directly demonstrate that the proposed scoring mechanism identifies more appropriate or reliable reasoning strategies.
  4. The contribution of the dynamic communication topology remains insufficiently isolated. Lemma 2.1 establishes structural properties of the score-directed DAG, but does not explain why routing information from higher-scoring agents should improve reasoning quality, especially when these scores can favor shared incorrect answers. The existing ablations primarily examine prompt rewriting, donor selection, and answer aggregation, leaving the benefits of the proposed graph construction relative to simpler communication structures unclear.
Questions for authors
  1. Could the authors provide a detailed comparison of total model calls, input/output tokens, and inference time across SAGE and the strongest baselines? It would be particularly helpful to see whether the accuracy advantage persists under approximately matched inference budgets.
  2. Since the prompt rewriter only observes the original role prompts, would a precomputed set of donor–recipient prompt combinations achieve similar performance? I also wonder whether simply adding a shared set of generic reasoning or verification instructions would recover much of the improvement attributed to strategy transfer.
  3. Could the authors provide a more direct evaluation of donor-selection quality, such as the relationship between donor scores, answer correctness, and the subsequent improvements of recipient agents? It would be helpful to understand whether prefix consistency and reciprocal review contribute complementary information or primarily reinforce existing agreement.
  4. Would it be possible to compare the proposed score-directed DAG with fixed, random, or fully connected communication graphs while keeping donor selection, prompt rewriting, and answer aggregation unchanged? Such a comparison would help clarify whether dynamic topology adaptation provides an independent benefit.
2 comments
AuthorsOpenReview ↗

Response to Reviewer LYGG

We thank the reviewer for their careful and constructive review. We respond to each of your concerns below.

Q1 (cost). Task-model calls and input / output tokens (thousands) per question, macro-averaged over six benchmarks and three seeds:

MethodQwen callsQwen in / outMinistral callsMinistral in / out
Single10.15 / 0.3710.13 / 0.73
SelfOrg13.311.8 / 4.215.123.5 / 11.4
MOC13.015.2 / 4.913.025.5 / 10.7
MAD-M216.018.2 / 6.716.029.9 / 14.0
SAGE36.432.0 / 8.433.040.7 / 19.8

A high token cost relative to single-agent sampling is a known property of multi-agent systems as a class, not of SAGE alone. Prior work reports that multi-agent debate does not reliably outperform self-consistency and ensembling once cost is considered (Smit et al., ICML 2024); that a single agent with a strong prompt nearly matches the best discussion methods (Wang et al., ACL 2024); that debate methods often fail to beat CoT and self-consistency even with significantly more inference-time computation (Zhang et al., arXiv:2502.08788); and that majority voting alone accounts for most of the gains attributed to debate (Choi et al., NeurIPS 2025).

We therefore do not claim that SAGE is more compute-efficient than other MAS methods. Our claim is limited to the comparison with communication-based multi-agent methods under the same agent count and round limit. Budget-matched versions of those baselines are needed to make that claim rigorous, and we will add them. Also, the runs used different hardware, so we have no comparable wall-clock times.

Q2 (precomputed rewrites, generic instructions). Your reading of Eq. (6) is correct. Since the rewrite does not depend on the problem, it could be done once for each (target, donor) pair and stored in a table, and with 9 roles there are at most 72 such pairs. We rewrite for each problem to keep some variability in the prompts, and a stored table would remove this variability. We have not run this variant yet, but we expect it to give similar accuracy at a lower token cost, and we will report it in the revision.

Q3 (donor selection). Our selection metrics are intended as reliability signals rather than guarantees of correctness, as none of the existing methods can do that, during runtime. [Iwase et al. 2026] show that reasoning traces leading to correct answers are more reproducible under prefix truncation and regeneration than those leading to incorrect answers, providing empirical support for prefix consistency as a signal of answer reliability. Similarly, answer agreement follows the well-established self-consistency principle, where majority voting over independently sampled reasoning paths has been shown to improve reasoning accuracy across multiple benchmarks [Wang et al., 2023]. We acknowledge that correlated errors or shared misconceptions can still lead to high agreement. Thus, these metrics serve as probabilistic indicators of reliable and reproducible strategies, rather than definitive evidence of correctness.

Q4 (topology). We agree that Lemma 2.1 only shows the graph is sparse and shallow, which limits cost, and that it does not show the graph improves accuracy. We route from higher-scoring agents for the reason given in SelfOrg (Tastan et al., 2026). When wrong answers are spread out, agents agree more often on the correct answer than on any single wrong one, so high-agreement answers are more likely to be correct. We agree this fails when many agents share the same wrong answer. To reduce this risk, agents are told not to copy peer answers blindly, the graph is rebuilt every round, and pooled voting keeps correct answers from earlier stages (Appendix H.3). We also agree that our current ablations do not isolate the graph. In the revision, we will add an ablation that changes only the graph and compares it with no communication, a fully connected graph, a random graph and a graph that is never rebuilt.

Reviewer_LYGGOpenReview ↗

Official Comment by Reviewer LYGG

Thank you for responding to my questions. Your responses clarified some of my concerns. After considering the paper as a whole, I have decided to keep my original score.

Reviewer_zBWLOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes SAGE, a training-free multi-agent reasoning framework that selects a strategy donor using answer agreement, prefix consistency, and reciprocal peer review. It transfers the donor’s prompt-level reasoning guidance to other agents, then coordinates revisions through a dynamic sparse DAG and weighted voting. Experiments cover two main backbones, six reasoning benchmarks, and additional scaling, heterogeneous-team and corruption settings.

Critical strengths
  1. The story is clear. The paper is well-structure and easy to understand.
  2. The empirical analysis is fairly extensive. Beyond evaluating performance across multiple reasoning benchmarks, the paper studies scaling behavior with both backbone size and agent count, as well as heterogeneous-team composition and reliability under corrupted peer messages.
Critical weaknesses
  1. The claimed strategy-transfer mechanism may be overstated. SAGE proposes a concrete and somewhat novel mechanism for selecting a problem-specific donor and transferring its role strategy to other agents. However, Appendix H only shows that selected-donor prompt rewriting improves final accuracy, while it does not establish that the gain comes from transferring a problem-appropriate strategy while preserving complementary roles, rather than from generic prompt strengthening, additional verification instructions, or selecting a globally stronger role.
  2. The main comparisons are not compute-matched. SAGE introduces substantial additional inference workload while the baselines are matched only in agent count and collaboration rounds. Total calls, generated tokens, latency, and cost are not reported. This concern is particularly visible in Figure 3: despite substantially greater inference cost than Single, SAGE provides only modest gains at many larger model sizes and even underperforms Single on GPQA-Diamond at 3B and 32B, leaving its cost-effectiveness and scaling behavior unclear.
Questions for authors
  1. Can the authors directly verify that prompt rewriting transfers a problem-appropriate donor strategy while preserving the target agent’s original role? For example, through human annotation or LLM-judge analysis of rewritten prompts and reasoning traces, together with fixed-donor and universal verification-prompt controls, to distinguish strategy transfer from generic prompt strengthening.
  2. Can the authors report the complete number of model calls and generated tokens to demonstrate that the gains do not primarily result from additional inference computation?
  3. How does SAGE differ from MetaGen[1], which also performs training-free, per-instance adaptation of role prompts and interaction topologies? A direct comparison would help clarify SAGE’s methodological novelty.

[1] MetaGen: Self-Evolving Roles and Topologies for Multi-Agent LLM Reasoning. Arxiv 27 Jan 2026

3 comments
AuthorsOpenReview ↗

Response to Reviewer zBWL

We thank the reviewer for their careful and constructive review. We respond to each of your concerns below.

Q1 (strategy transfer): As per your recommendation, we used LLM to audit all the prompt rewrites of SAGE for every run. Every accepted rewrite of the agent's prompt keeps it's current identity and none of them adopts the donor's entirely. Added text in the prompt is donor-specific. In the runs, agents with rewritten prompts relied less on their peers. When an agent was wrong and none of the peer agent's saw it was right, it fixed its own answer 7.0% of the time, against 4.4% with the original prompt. When it was right and every other peer agent saw it was wrong, it kept its answer 45.5% of the time, against 34.4%.

Q2 (compute). We agree that the comparison is not compute-matched. Task-model calls and total tokens (thousands) per question, macro-averaged over six benchmarks and three seeds:

MethodQwen callsQwen tokensMinistral callsMinistral tokens
Single10.510.9
SelfOrg13.316.015.134.8
MOC13.020.113.036.2
MAD-M216.024.916.043.8
SAGE36.440.433.060.6

A high token cost relative to single-agent sampling is a known property of multi-agent systems as a class, not of SAGE alone. Prior work reports that multi-agent debate does not reliably outperform self-consistency and ensembling once cost is considered (Smit et al., ICML 2024); that a single agent with a strong prompt nearly matches the best discussion methods (Wang et al., ACL 2024); that debate methods often fail to beat CoT and self-consistency even with significantly more inference-time computation (Zhang et al., arXiv:2502.08788); and that majority voting alone accounts for most of the gains attributed to debate (Choi et al., NeurIPS 2025).

We therefore do not claim that SAGE is more compute-efficient than other MAS methods. Our claim is limited to the comparison with communication-based multi-agent methods under the same agent count and round limit. Budget-matched versions of those baselines are needed to make that claim rigorous, and we will add them.

Regarding Figure 3, we agree that the benefits are concentrated among smaller backbones. We believe this is because stronger backbone models are already sufficiently capable that communication in a multi-agent setting barely provides them any additional/complimentary signal for improving their responses. Weaker agents have greater scope for improvement by leveraging complementary expertise from other agents, which allows them to identify and correct mistakes they would otherwise make independently.

Q3 (MetaGen). Thank you for sharing this paper. We will cite and discuss it. Both methods adapt roles and topology per instance, but SAGE differs in three ways. First, SAGE adapts without any external evaluator. The donor selection, routing and voting use only answer agreement, prefix consistency and peer review computed from the agents' own responses. MetaGen's loop stops on an evaluator's pass signal, and its role and edge priors are learned from a pass/cost reward. Second, in SAGE, each problem is solved independently. MetaGen accumulates learned weights and a cache of verified roles across problems, and its own ablation attributes up to 2.4 points to that memory. Third, SAGE adapts roles by transferring instructions between existing peers rather than generating new ones, and its rewriter sees only two role prompts, never the problem, the responses or failure traces, so no solution content or injected text can enter a prompt. SAGE therefore applies where no evaluator or pass signal is available. A direct comparison was not feasible as no code is public, and the paper does not specify how the pass signal is obtained on GSM8K, MMLU or AQuA.

Thank you for the response and the new experiments. However, I still have a few questions.

First, the new strategy-transfer experiments are helpful, but they mainly demonstrate role-name preservation and improved self-correction or resistance to incorrect peers. They do not yet distinguish problem-specific donor transfer from generic prompt strengthening. Please report these results more clearly, including at least the judge models, audit prompts, sample counts, and representative case studies. I encourage adding these details directly to a revised manuscript with the changes clearly marked.

Regarding Q2, SAGE is substantially more expensive than the evaluated MAS baselines, so this concern remains until budget-matched results are provided. More broadly, the modest and sometimes negative gains over the much cheaper single-inference baseline on stronger backbones materially limit the paper’s practical significance and overall contribution.

Thank you for the follow-up reply, we really appreciate it. We clarify your concerns below:

1. Strategy transfer

Appendix H.2 includes analysis on whether the gain comes from the chosen donor or from generic prompt strengthening. In SAGE-RandomDonor, every prompt is still rewritten in the same way. The only change is that the donor is a random teammate. If generic prompt strengthening explained the gain, this variant would keep most of it. It does not. On Qwen2.5-1.5B, the average accuracy is 56.21 without rewriting, 56.72 with a random donor, and 58.31 with SAGE's chosen donor. The random donor is also lower than SAGE on all six benchmarks. So the gain depends on which donor's strategy is transferred.

Appendix F gives a representative case study. We will add the judge model, the full audit prompt, the sample counts and more case studies to the revised manuscript.

2. Compute and practical significance

We will add budget-matched versions of the multi-agent baselines to the revised manuscript. Two ablations already keep the compute budget similar. SAGE-RandomDonor uses the same number of calls as SAGE, and it is lower on all six benchmarks. SAGE-NoRewrite removes only the short rewrite calls, and it loses about 2 points on both backbones. So at almost the same compute, SAGE's design still makes a clear difference.

On the stronger backbone, Ministral-3-3B, SAGE still has the best average accuracy among the multi-agent methods. It scores 75.08, compared with 73.79 for MAD-M2. The gain from prompt rewriting is also about the same on both backbones. So the benefit does not shrink on the stronger backbone.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.