acceptodds
Withdrawn from ICLR 2027

StageMAD: Process-Controlled Communication for Multi-Agent Debate

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Performing multi-step reasoning with large language models based on a single sampled trajectory remains vulnerable to local errors in intermediate steps, as errors may propagate through subsequent reasoning and finally lead to a wrong answer. To mitigate this limitation, multi-agent debate (MAD) samples multiple reasoning trajectories and allows agents to exchange information and revise their reasoning before answer aggregation, which is typically performed through majority or plurality voting. However, most existing MAD methods lack reliability constraints on both the content and propagation of exchanged information, so incorrect reasoning can also spread across agents like correct reasoning. As individual responses are not reliable, unverified erroneous reasoning may influence other agents, suppress an initially correct minority, and even produce an incorrect consensus. To address this problem, we propose StageMAD, a process-guided multi-agent debate framework consisting of three stages: local self-correction, process-evidence-based directed logical guidance, and mutual critique over residual disagreements. By localizing questionable reasoning steps and selectively controlling information propagation, StageMAD enables reliable reasoning to gain broader support while limiting the spread and accumulation of incorrect answers. Experiments on mathematical reasoning and language understanding datasets demonstrate the effectiveness of our proposed framework.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Large Language Models, Multi-Agent Reasoning, Multi-Agent Debate, Process Reward Models, Reasoning Verification

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_87JTOpenReview ↗
Rating
1(−1)
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper addresses the limitations of relying on a single reasoning trace and examines the vulnerabilities of unconstrained interaction within multi-agent debate frameworks. The authors observe that existing MAD approaches generally lack reliability constraints on both the content and the propagation of exchanged information, allowing flawed reasoning to spread across agents alongside valid arguments. To mitigate this issue, the authors introduce StageMAD, a three-stage framework consisting of local self-correction, restricted peer guidance, and mutual verification under close votes. By integrating these three stages, the authors suggest that the proposed method produces more reliable final answers. The authors validate their approach using Qwen2.5-7B-Instruct, Qwen3.5-9B, and DeepSeek-Math-7B across multiple benchmarks.

Critical strengths
  • The authors address an important failure mode in MAD frameworks where correct minority answers are overwritten by peer influence.
  • The authors characterize the theoretical vulnerabilities of multi-agent debate and present a well-motivated framework as a potential remedy.
  • Rather than limiting the evaluation to final accuracy, the authors incorporate diverse analytical metrics, including question-level transitions(w->c / c->w) as well as a frozen-pool continuation setup.
Critical weaknesses
  • The problem setting targeted by the authors appears overly specific. I believe the proposed method should be evaluated against both general MAD frameworks and simple non-debate inference baselines. When compared against these standard baselines, the performance gains achieved through the proposed multi-stage pipeline appear rather marginal.
    • The performance improvement relative to the computational cost is quite limited. For instance, with DeepSeek-Math-7B-Instruct, StageMAD consumes over ten times the token budget of standard CoT while occasionally yielding lower accuracy.
    • Could the authors clarify the exact performance margin of StageMAD over a simple single-pass inference baseline?
  • When proposing a multi-stage framework, conducting a thorough ablation study is essential to justify the necessity of each individual stage. However, the current manuscript seems not provide sufficient ablation experiments to support this design.
    • While Figure 4 presents an ablation analysis, the reported results do not sufficiently justify the inclusion of each stage. The authors need to rigorously verify both the necessity and the reliability of every individual stage.
    • Furthermore, the overall framework appears to rely heavily on Stage 3. Even if arbitrary responses were placed in Stages 1 and 2 and only Stage 3 were executed, the final performance would likely show minimal difference.
  • According to Figure 2, the correction effect of the framework is substantial on relatively straightforward benchmarks such as MATH and GSM8K, whereas the gains on more challenging benchmarks like MMLU-Pro and AIME24 are negligible (or even worse). This disparity suggests that the effectiveness of the method fluctuates considerably depending on task difficulty.
    • In addition, the main experimental results fail to isolate the contribution of controlled communication, which is the central claim of this paper. The non-communicative PRM self-revision baseline performs comparably to or even outperforms StageMAD across several settings. A direct comparison against communication-based baselines equipped with the same PRM is necessary for a fair evaluation.
    • Debate frameworks are primarily needed when solving complex problems. Exhibiting weaknesses on difficult benchmarks undermines the practical utility of StageMAD. Given the increasing complexity of tasks expected from modern LLMs, the authors should more clearly demonstrate whether StageMAD remains effective on demanding tasks.
    • The experiments are restricted to models with relatively small capacities of 9B parameters or fewer. The authors need to demonstrate scalability by verifying whether their findings generalize to more capable models. In particular, the performance gains appear smallest on Qwen3.5, which is the strongest among the tested models. Consequently, it remains uncertain whether this framework provides meaningful benefits for larger models or heterogeneous agent configurations commonly adopted in recent MAD literature.
  • Several baseline results raise questions regarding their validity. With Qwen3.5, CoT-SC@15 achieves only 53.2 on GSM8K, which is lower than the accuracy of the same model on MATH (65.0) and more than 30 points below all other methods in Table 1 (ranging from 85.8 to 95.3), despite all methods starting from independent samples generated by the same task model.
    • As a researcher working extensively with LLMs, I have observed in my own experiments that Qwen3.5 achieves substantially higher performance (than the scores reported by the authors) on these benchmarks even without repeating CoT 15 times. These unexpectedly low figures suggest a potential flaw in the experimental implementation. The authors should inspect their pipeline for issues such as output truncation or answer-parsing failures.
  • The main manuscript is not sufficiently self-contained. Essential details required to understand and reproduce the proposed method are either relegated to the appendix or omitted entirely. Information that is indispensable for comprehending the core methodology must be included in the main text rather than the appendix, as deferring such details merely serves to circumvent the page limit. For example,
    • The number of agents used in the debate
    • Details of the PRM backbone, generation hyperparameters, and evaluation dataset sizes
    • The specific values and selection criteria for the core thresholds α2, δU, and δv in Equations 3 to 5
    • Clear explanations of how group representatives are selected in Stage 3 (Line 280) and what the phrase "with the help of the PRM" specifically entails during source guidance generation (Line 266)
Questions for authors

The terminology used for the stages is inconsistent across the manuscript. The abstract (Lines 22 to 23) refers to the three stages as "local self-correction, directed logical guidance based on process evidence, and mutual critique of residual disagreements," whereas Sections 1 and 4 describe them as "local self-correction, restricted peer guidance, and mutual verification under close votes." Similarly, the messages exchanged in Stage 2 are variously termed "directed logical hints" (Line 105), "constraint-level guidance," "answer-free guidance," and "restricted guidance." Adopting a unified set of terms throughout the paper would improve readability for the reader.

7 comments

Dear Reviewer 87JT:

Thank you for examining our experimental comparisons and the clarity of the manuscript. Your comments prompted us to recheck the CoT baseline more carefully, extend the evaluation, and clarify the implementation details. We address each point below and would appreciate any further questions.

Q1: The problem setting targeted by the authors appears overly specific. I believe the proposed method should be evaluated against both general MAD frameworks and simple non-debate inference baselines. When compared against these standard baselines, the performance gains achieved through the proposed multi-stage pipeline appear rather marginal.

  • The performance improvement relative to the computational cost is quite limited. For instance, with DeepSeek-Math-7B-Instruct, StageMAD consumes over ten times the token budget of standard CoT while occasionally yielding lower accuracy.
  • Could the authors clarify the exact performance margin of StageMAD over a simple single-pass inference baseline?

A1: Table 1 already compares StageMAD with vanilla MAD, MAD-M² (S), MAD-M² (O), and the non-communicative CoT-SC@15 and PRM self-revision baselines. To directly answer your question about the advantage over single-pass inference, we additionally report CoT with one reasoning trajectory per question.

The comparison with single-pass CoT differs from the comparison with CoT self-consistency. On DeepSeek-Math/MATH, StageMAD achieves 55.3% accuracy compared with 41.1% for single-pass CoT, an improvement of 14.2 percentage points. The baseline that achieves higher accuracy at lower token cost in this setting is CoT-SC@15, which reaches 57.7%.

The complete single-pass comparison is shown below. All accuracies are averages over three runs, and the StageMAD results are from Table 1.

ModelDatasetSingle-pass CoTStageMADDifference (pp)
Qwen2.5-7B-InstructMATH62.3%71.7%+9.4
Qwen2.5-7B-InstructGSM8K61.8%93.8%+32.0
Qwen2.5-7B-InstructMMLU-Pro37.4%58.3%+20.9
Qwen2.5-7B-InstructAIME247.8%15.6%+7.8
Qwen3.5-9BMATH60.1%73.6%+13.5
Qwen3.5-9BGSM8K87.53%95.3%+7.8
Qwen3.5-9BMMLU-Pro55.0%71.1%+16.1
Qwen3.5-9BAIME243.3%17.8%+14.5
DeepSeek-Math-7B-InstructMATH41.1%55.3%+14.2
DeepSeek-Math-7B-InstructGSM8K77.3%89.1%+11.8
DeepSeek-Math-7B-InstructMMLU-Pro19.8%34.0%+14.2
DeepSeek-Math-7B-InstructAIME240.0%1.1%+1.1

StageMAD achieves higher accuracy than single-pass CoT in all twelve reported settings. These differences measure the benefit of the complete inference pipeline, including multiple candidates, local correction, and communication. Figure 4 separately examines the contribution of the communication policy by holding the preceding candidate pools fixed.

We agree that StageMAD is not the best use of additional computation in some cases. On DeepSeek-Math/MATH, CoT-SC@15 exceeds StageMAD by 2.4 percentage points while using about one tenth as many tokens. On Qwen2.5-7B-Instruct/MATH, CoT-SC@15 also achieves higher accuracy, 73.8% versus 71.7%, at lower token cost. These are settings where independent sampling provides a better accuracy–cost trade-off.

Our contribution concerns improving communication within MAD, where StageMAD achieves the highest accuracy among the compared MAD methods in 11 of the 12 main settings. The non-communicative baselines establish when this form of interaction is useful relative to simpler inference strategies. We will add the single-pass results and distinguish these comparisons explicitly in the revised manuscript.

Q2: When proposing a multi-stage framework, conducting a thorough ablation study is essential to justify the necessity of each individual stage. However, the current manuscript seems not provide sufficient ablation experiments to support this design.

  • While Figure 4 presents an ablation analysis, the reported results do not sufficiently justify the inclusion of each stage. The authors need to rigorously verify both the necessity and the reliability of every individual stage.
  • Furthermore, the overall framework appears to rely heavily on Stage 3. Even if arbitrary responses were placed in Stages 1 and 2 and only Stage 3 were executed, the final performance would likely show minimal difference.

A2: The stage-wise results do not support the interpretation that Stage 3 accounts for most of the improvement. Table 13 shows that Stage 1 contributes 4.4 and 6.8 percentage points on MATH with Qwen2.5-7B-Instruct and Qwen3.5-9B, respectively, whereas Stages 2 and 3 together contribute 1.8 and 1.1 points. To directly examine your suggestion that the first two stages could be skipped, we additionally evaluated Stage 3 alone from the initial candidate pool.

MethodMATHGSM8KMMLU-ProAIME24
Stage 3 only66.7%92.1%57.1%13.3% (4/30)
StageMAD (Table 1, three-run mean)71.7%93.8%58.3%15.6%

Stage 3 alone scored below the Table 1 averages on all four datasets. It also fell short of the full pipeline on all four datasets in the single-seed comparison from the same R0 candidates. This shows that the preceding stages contribute to the final performance. Figure 4 addresses a complementary question by fixing the R1 or R2 candidate pool and comparing the subsequent communication policies.

Q3: According to Figure 2, the correction effect of the framework is substantial on relatively straightforward benchmarks such as MATH and GSM8K, whereas the gains on more challenging benchmarks like MMLU-Pro and AIME24 are negligible (or even worse). This disparity suggests that the effectiveness of the method fluctuates considerably depending on task difficulty.

  • In addition, the main experimental results fail to isolate the contribution of controlled communication, which is the central claim of this paper. The non-communicative PRM self-revision baseline performs comparably to or even outperforms StageMAD across several settings. A direct comparison against communication-based baselines equipped with the same PRM is necessary for a fair evaluation.
  • Debate frameworks are primarily needed when solving complex problems. Exhibiting weaknesses on difficult benchmarks undermines the practical utility of StageMAD. Given the increasing complexity of tasks expected from modern LLMs, the authors should more clearly demonstrate whether StageMAD remains effective on demanding tasks.
  • The experiments are restricted to models with relatively small capacities of 9B parameters or fewer. The authors need to demonstrate scalability by verifying whether their findings generalize to more capable models. In particular, the performance gains appear smallest on Qwen3.5, which is the strongest among the tested models. Consequently, it remains uncertain whether this framework provides meaningful benefits for larger models or heterogeneous agent configurations commonly adopted in recent MAD literature.

A3: Figure 2 presents the framework overview. If you are referring to the correction–corruption counts in Figure 3, these are raw counts across datasets with different sizes and task characteristics. AIME24, for example, contains only 30 questions. These counts therefore do not isolate task difficulty as the cause of the performance differences. We agree that the gains vary across models and tasks, but the results do not establish a general relationship in which StageMAD becomes less effective as task difficulty increases.

The contribution of controlled communication is examined separately from the full-system comparison. Table 1 compares complete inference methods, whereas Figure 4 fixes the Round-1 or Round-2 candidate pools before comparing subsequent communication policies. These comparisons separate the effect of the communication policy from the quality of the preceding candidates. With the Round-1 pool fixed, StageMAD achieves the highest or tied-highest accuracy among the MAD policies compared in 11 of the 12 settings.

To address your request for a communication baseline equipped with the same PRM, we additionally evaluated MAD-M² (S) with the same GenPRM used by StageMAD on Qwen2.5-7B-Instruct. This variant uses GenPRM to filter the peer reasoning trajectories available to agents. The table reports one run for MAD-M² (S) + GenPRM and the three-run means from Table 1 for the other methods.

DatasetMAD-M² (S)MAD-M² (O)MAD-M² (S) + GenPRMStageMAD
MATH67.4%65.7%66.6%71.7%
GSM8K91.7%90.7%91.3%93.8%
MMLU-Pro58.6%56.9%60.0%58.3%
AIME2413.3%10.0%13.3%15.6%

Access to the same GenPRM does not consistently close the performance gap. StageMAD has higher reported accuracy on MATH, GSM8K, and AIME24, while MAD-M² (S) + GenPRM performs better on MMLU-Pro. Together with the matched-pool experiments, this provides evidence that how process judgments are used to organize communication matters, beyond simply making a PRM available.

To examine whether the findings extend beyond homogeneous teams, we compared StageMAD with vanilla MAD using three Qwen2.5-7B-Instruct agents and two Qwen3.5-9B agents. Both methods used the same team and initial candidates on each dataset. We conducted one run per dataset.

DatasetHeterogeneous vanilla MADHeterogeneous StageMAD
MATH69.4%74.4%
GSM8K94.3%95.6%
MMLU-Pro71.2%70.2%
AIME2413.3% (4/30)20.0% (6/30)

StageMAD improves over vanilla MAD on three of the four datasets with this heterogeneous team, including AIME24, while performing worse on MMLU-Pro. These results provide initial evidence beyond homogeneous teams. They address heterogeneous composition rather than larger model scale; whether the gains extend to substantially larger models remains open. We will distinguish these two questions explicitly in the revised manuscript.

Reviewer_hk4zOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes StageMAD, a three-stage multi-agent debate (MAD) framework regulated by a frozen generative Process Reward Model (GenPRM). To prevent erroneous reasoning from corrupting correct minority answers, StageMAD structures deliberation into: (1) Local self-correction, where agents independently regenerate reasoning suffixes starting from the first PRM-flagged negative step; (2) Restricted peer guidance, where an eligible high-scoring source agent sends answer-free, constraint-level hints to defective targets in lower-scoring groups; and (3) Mutual verification under close votes, triggered when the top-two answer support margin is ≤δv\le \delta_v, allowing group representatives to exchange verifiable step challenges.

Experiments across four benchmarks (GSM8K, MATH, AIME24, MMLU-Pro) and three model families show that StageMAD outperforms existing communication-based MAD baselines in 11 of 12 settings, while exhibiting mixed results against non-communicative PRM self-revision and self-consistency baselines.

Critical strengths

1. Controlled Ablations and Diagnostic Analysis

The authors go beyond end-to-end accuracy by freezing Round-1 and Round-2 candidate pools (Figure 4, Appendix C) and tracking question-level correctness transitions (W→CW \rightarrow C vs. C→WC \rightarrow W, Figure 3, Appendix H). These controlled comparisons help isolate the effects of downstream communication policies from differences in the initial candidate pools.

2. Well-Motivated Restricted Guidance and Selective Verification

Withholding full peer solutions in Stage 2, while masking defective reasoning suffixes, substantially reduces source-answer agreement compared with unrestricted peer guidance (Table 9). Additionally, the Round-3 vote-margin gate reduces unnecessary deliberation and limits harmful revisions on instances with clearer answer-group support (Table 2).

Critical weaknesses

1. Limited Marginal Value of Peer Communication Beyond PRM Self-Correction

Although StageMAD outperforms vanilla MAD, most of its accuracy improvement comes from Stage 1, which performs non-communicative local self-correction. Table 13 shows that Stage 1 provides +4.4 percentage points on Qwen2.5 MATH and +6.8 points on Qwen3.5 MATH, whereas Stages 2 and 3 combined contribute only +1.8 and +1.1 points, respectively. On GSM8K, the additional gains are just +0.3–0.4 points, and on Qwen3.5 MMLU-Pro, peer communication actually reduces accuracy by 0.3 points.

More importantly, the matched-intervention comparison in Table 9 shows that answer-free guidance provides only a modest improvement over local PRM self-revision on MATH (72.0% vs. 71.4%) and performs slightly worse on GSM8K (92.9% vs. 93.0%) and MMLU-Pro (58.7% vs. 59.8%). Furthermore, the standalone PRM self-revision baseline substantially outperforms StageMAD with Qwen3.5 on MATH (79.6% vs. 73.6%), MMLU-Pro (76.5% vs. 71.1%), and AIME24 (41.1% vs. 17.8%).

While the matched-pool results in Figure 4 support the effectiveness of StageMAD relative to alternative MAD communication policies, the limited incremental gains leave the practical value of its additional peer-communication stages insufficiently established relative to simpler, non-communicative revision strategies.

2. Cost-Effectiveness and Reliability Limitations of GenPRM-Controlled Revision

StageMAD evaluates reasoning-step prefixes using a generative PRM, with a maximum of 2,048 generated tokens per verification call. Although the paper includes these costs in its overall token accounting (Appendix A, J), the marginal accuracy gains of Stages 2 and 3 appear relatively small compared with the additional verification and communication overhead. In Table 1, the much simpler CoT-SC@15 baseline achieves higher accuracy than StageMAD on Qwen2.5 MATH (73.8% vs. 71.7%) at substantially lower relative token cost (0.17× vs. 1.12× MAD), with a similar result on DeepSeek-Math MATH (57.7% vs. 55.3%, at 0.18× vs. 1.80×).

The controller is also sensitive to PRM verification errors. When incorrect trajectories are judged entirely positive, StageMAD does not revise them in Stages 1 or 2 and may even select them as guidance sources. This concern is particularly relevant outside mathematical reasoning. On CommonsenseQA (Appendix Tables 11–12), the reported Qwen2.5 run shows more harmful corruptions (C→WC \rightarrow W) than beneficial corrections (W→CW \rightarrow C) in every round, reducing accuracy from 81.2% to 79.9%. Round-3 mutual critique also decreases MMLU-Pro accuracy across all three evaluated model families.

Together, these results raise concerns about whether the complexity and computational cost of PRM-controlled communication consistently translate into meaningful improvements, particularly when process-verification judgments are less reliable.

Questions for authors

Q1. Marginal Benefit of Peer Communication After Stronger Self-Revision

Appendix I.3 shows that PRM self-revision provides feedback to every agent, whereas StageMAD revises only agents with explicitly PRM-identified defects. Could the authors evaluate StageMAD with unconditional PRM self-revision in Stage 1, followed by the original Stages 2 and 3? This would help determine whether peer communication provides meaningful additional gains when starting from a stronger locally revised candidate pool.

Q2. Stage-Wise Token Cost and Compute-Matched Comparisons

Although Appendix A and J account for PRM inference costs, could the authors provide a breakdown of actual token consumption by stage, separating task-model generation from step-level GenPRM verification? More importantly, how does the accuracy–cost trade-off compare with allocating the same total token budget to additional independent self-revision or CoT self-consistency?

Q3. Reliability of the Round-3 Vote-Margin Gate

Table 10 shows that Round-3 mutual critique reduces accuracy in the 3–2 vote stratum on both MATH (63.33% to 56.67%) and GSM8K (86.96% to 69.57%), even though this pattern activates the proposed gate. Meanwhile, other close-vote patterns can benefit from additional verification. Could the authors explain this difference and examine whether a vote-pattern-aware gate, rather than relying solely on the top-two support margin, would improve accuracy and efficiency?

5 comments

Dear Reviewer HK4Z:

Thank you for your careful reading and detailed questions about the value and cost of peer communication. Your suggestions motivated additional experiments and helped us better explain the role of each stage. We present our responses and results below.

Q1: Limited Marginal Value of Peer Communication Beyond PRM Self-Correction

Although StageMAD outperforms vanilla MAD, most of its accuracy improvement comes from Stage 1, which performs non-communicative local self-correction. Table 13 shows that Stage 1 provides +4.4 percentage points on Qwen2.5 MATH and +6.8 points on Qwen3.5 MATH, whereas Stages 2 and 3 combined contribute only +1.8 and +1.1 points, respectively. On GSM8K, the additional gains are just +0.3–0.4 points, and on Qwen3.5 MMLU-Pro, peer communication actually reduces accuracy by 0.3 points.

More importantly, the matched-intervention comparison in Table 9 shows that answer-free guidance provides only a modest improvement over local PRM self-revision on MATH (72.0% vs. 71.4%) and performs slightly worse on GSM8K (92.9% vs. 93.0%) and MMLU-Pro (58.7% vs. 59.8%). Furthermore, the standalone PRM self-revision baseline substantially outperforms StageMAD with Qwen3.5 on MATH (79.6% vs. 73.6%), MMLU-Pro (76.5% vs. 71.1%), and AIME24 (41.1% vs. 17.8%).

While the matched-pool results in Figure 4 support the effectiveness of StageMAD relative to alternative MAD communication policies, the limited incremental gains leave the practical value of its additional peer-communication stages insufficiently established relative to simpler, non-communicative revision strategies.

A1: Stage 1 provides the largest share of StageMAD’s accuracy improvement. The smaller gains from Stages 2 and 3 should be considered together with their incremental cost. Based on the averages across runs in Table 13, Stages 2 and 3 together improve accuracy on MATH by 1.8 and 1.1 percentage points for Qwen2.5-7B-Instruct and Qwen3.5-9B, respectively. On GSM8K, the corresponding gains are 0.3 and 0.4 points.

In the additional Qwen2.5-7B-Instruct cost study, Stages 2 and 3 together consume 16.76 million tokens on MATH, compared with 37.21 million for Stage 1, and 2.82 million on GSM8K, compared with 14.71 million for Stage 1. The later stages therefore use approximately 55% and 81% fewer tokens than Stage 1, respectively, including task-model and GenPRM input and output tokens. They show that the later stages also account for a smaller share of the inference budget. Our response to Q4 provides the full breakdown and comparisons with independent self-revision and additional CoT sampling.

Table 9 addresses a different question: how the revision signal affects the same targets under matched interventions. Answer-free guidance does not consistently outperform local PRM self-revision in this comparison. Its clearer benefit is relative to providing full peer solutions: on MATH, source-answer agreement falls from 76.7% to 20.3%, and on MMLU-Pro, the correct-to-wrong rate among initially correct revision targets falls from 85.6% to 62.9%. These results support restricting peer messages to reduce answer following and harmful revisions, while the comparison with local self-revision shows that introducing peer information is not beneficial in every setting.

Figure 4 further examines which communication policy makes better use of a fixed candidate pool. With the Round-1 candidates held fixed, StageMAD achieves the highest or tied-highest accuracy among the MAD policies compared in 11 of the 12 settings. Thus, its advantage over those policies cannot be attributed solely to the improvement produced by Stage 1.

Together, these results support a practical role for controlled communication: it can provide further gains after local revision while limiting the cost and harmful influence of subsequent exchanges. They do not establish a consistent advantage over independent PRM self-revision, particularly for Qwen3.5-9B. We will make this distinction explicit in the revised manuscript.

Q2: Cost-Effectiveness and Reliability Limitations of GenPRM-Controlled Revision

StageMAD evaluates reasoning-step prefixes using a generative PRM, with a maximum of 2,048 generated tokens per verification call. Although the paper includes these costs in its overall token accounting (Appendix A, J), the marginal accuracy gains of Stages 2 and 3 appear relatively small compared with the additional verification and communication overhead. In Table 1, the much simpler CoT-SC@15 baseline achieves higher accuracy than StageMAD on Qwen2.5 MATH (73.8% vs. 71.7%) at substantially lower relative token cost (0.17× vs. 1.12× MAD), with a similar result on DeepSeek-Math MATH (57.7% vs. 55.3%, at 0.18× vs. 1.80×).

The controller is also sensitive to PRM verification errors. When incorrect trajectories are judged entirely positive, StageMAD does not revise them in Stages 1 or 2 and may even select them as guidance sources. This concern is particularly relevant outside mathematical reasoning. On CommonsenseQA (Appendix Tables 11–12), the reported Qwen2.5 run shows more harmful corruptions (C->W) than beneficial corrections (W->C) in every round, reducing accuracy from 81.2% to 79.9%. Round-3 mutual critique also decreases MMLU-Pro accuracy across all three evaluated model families.

Together, these results raise concerns about whether the complexity and computational cost of PRM-controlled communication consistently translate into meaningful improvements, particularly when process-verification judgments are less reliable.

A2: The manuscript did not distinguish the GenPRM completion limits used in different experimental settings in sufficient detail. The main experiments used a limit of 1,024 tokens per call. For the DeepSeek-Math R1 ablation, we increased this limit to 2,048 because GenPRM frequently exhausted the 1,024-token budget while generating its analysis, before producing a Yes/No verdict. Within each model–dataset pair in that ablation, all variants using GenPRM shared the same GenPRM settings, including the completion limit. We will report these settings explicitly and distinguish the ablation configuration from the main experiments in the revised manuscript.

In the two MATH settings you highlighted, CoT-SC@15 achieves both higher accuracy and lower token cost than StageMAD. The accuracy–cost comparison nevertheless varies across settings. For example, on GSM8K with Qwen2.5-7B-Instruct, StageMAD achieves 93.8% accuracy compared with 93.5% for PRM self-revision, using about 60% fewer total tokens. The stage-wise measurements in our response to Q4 also show that Stages 2 and 3 account for a smaller share of StageMAD’s token cost than Stage 1. These results identify settings where controlled communication offers a useful trade-off, alongside settings where independent sampling or revision is preferable.

Your concern about unreliable process judgments is also relevant to the communication design. If GenPRM judges every step of an incorrect trajectory positively, that trajectory may remain unrevised in Stages 1 and 2 and may become a guidance source. Stage 2 therefore separates selecting a source from accepting its answer. The source provides answer-free guidance about a suspected error in the target’s own reasoning, while withholding its final answer and full derivation. The target uses this guidance to independently recompute the affected reasoning. In Table 9, this restriction reduces source-answer agreement on MATH from 76.7% with full peer solutions to 20.3% with answer-free guidance. It limits direct answer following, although incorrect guidance can still influence revision.

The original Limitations section and Appendix Tables 11–12 already report and analyze the failures on non-mathematical tasks. In the CommonsenseQA run, accuracy falls from 81.2% to 79.9%, and Stage 3 lowers MMLU-Pro accuracy across all three model families. These findings show that the current process verifier and communication rules do not transfer uniformly across task types. We will bring this analysis into the main results discussion and conclusion, making clear that the evidence for StageMAD’s benefits is strongest in the evaluated mathematical reasoning settings and that its accuracy–cost advantages depend on the model and task.

Q3: Marginal Benefit of Peer Communication After Stronger Self-Revision

Appendix I.3 shows that PRM self-revision provides feedback to every agent, whereas StageMAD revises only agents with explicitly PRM-identified defects. Could the authors evaluate StageMAD with unconditional PRM self-revision in Stage 1, followed by the original Stages 2 and 3? This would help determine whether peer communication provides meaningful additional gains when starting from a stronger locally revised candidate pool.

A3: We ran the variant you requested with Qwen2.5-7B-Instruct (1,000 questions per dataset). For each dataset, we apply one round of PRM self-revision to all five agents, then run StageMAD Stages 2 and 3 from the resulting candidate pool. The token column counts additional task-model and GenPRM input and output tokens after the shared R1; it excludes the common R0 and R1 cost.

DatasetPolicy after shared R1Final accuracyGain over R1 (pp)Additional tokens (M)
MATHStop after shared R169.6%0.00.00
MATHRun StageMAD Stage 2 and Stage 370.8%+1.225.44
GSM8KStop after shared R193.7%0.00.00
GSM8KRun StageMAD Stage 2 and Stage 394.0%+0.311.83

Starting from the shared R1 candidate pool, Stage 2 and Stage 3 add 1.2 percentage points on MATH and 0.3 on GSM8K.

Reviewer_t73vOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes StageMAD, a process reward model (PRM)-based multi-agent debate framework designed to mitigate the propagation of erroneous reasoning and prevent correct answers from being overridden by incorrect consensus in conventional multi-agent debate (MAD). StageMAD consists of three stages: (1) identifying erroneous reasoning steps using a PRM and performing localized self-correction; (2) selecting communication partners based on reasoning reliability and exchanging constrained logical hints; and (3) conducting bidirectional challenges to further verify reasoning when voting results are close.

Experiments across three language models and multiple mathematical reasoning and language understanding benchmarks show that StageMAD outperforms existing MAD methods in most settings.

Critical strengths

The paper investigates an important problem in multi-agent communication: how to evaluate reasoning processes, determine which agents require correction, identify reliable sources of guidance, and decide when further communication is necessary. Applying PRMs to error localization and communication control, rather than directly selecting final answers, offers a potentially valuable contribution. Beyond the main performance comparisons, the paper includes shared candidate pool experiments, communication-content ablations, stage-wise analyses, and statistics on correctness transitions, providing useful insights into the effectiveness of the proposed method.

Critical weaknesses
  1. The core algorithmic novelty remains unclear. The three stages primarily integrate existing mechanisms, including PRM-based self-correction, reliability-driven communication, and mutual verification among agents. Although the overall design is reasonable, the paper does not clearly establish its fundamental algorithmic contribution over existing adaptive MAD methods, particularly why the three stages must be combined in the proposed order. What is the most indispensable algorithmic contribution of StageMAD beyond PRM-based self-revision and adaptive MAD?

  2. The benefits of multi-agent communication over PRM-based self-revision are not consistently demonstrated. Table 1 shows that although StageMAD outperforms most MAD baselines in many settings, it does not consistently surpass PRM self-revision. For example, with Qwen3.5-9B, PRM self-revision substantially outperforms StageMAD on MATH, MMLU-Pro, and AIME24. This weakens the claim that multi-agent communication provides additional benefits.

  3. The method's robustness to PRM errors is insufficiently investigated. StageMAD relies on step-level PRM judgments to identify errors, select communication sources, and determine which agents require correction. Incorrectly accepting flawed reasoning or rejecting valid reasoning could lead to error propagation or unnecessary revisions. The paper lacks a sufficiently comprehensive analysis of PRM quality and robustness.

  4. Generalizability beyond mathematical reasoning remains uncertain. StageMAD achieves limited improvements on MMLU-Pro, while the CommonsenseQA results in the appendix even show performance degradation. These findings suggest that the method may be better suited to mathematical reasoning, and its applicability to general language reasoning tasks requires further validation.

  5. The computational overhead and parameter sensitivity require further analysis. StageMAD introduces multiple stages of PRM evaluation and additional model calls. In some settings, its token consumption is substantially higher than that of the baselines. Token counts alone, however, are insufficient to characterize actual inference latency and computational costs. Furthermore, the sensitivity of the communication thresholds and voting-gating parameters requires more thorough investigation.

Questions for authors

See Weakness.

4 comments

Dear Reviewer T73V:

Thank you for your time and valuable comments on the contribution, reliability, and efficiency of StageMAD. Your feedback has helped us clarify our claims and strengthen the experimental analysis. We address your concerns below and would welcome your feedback on these clarifications and additional results.

Q1: The core algorithmic novelty remains unclear. The three stages primarily integrate existing mechanisms, including PRM-based self-correction, reliability-driven communication, and mutual verification among agents. Although the overall design is reasonable, the paper does not clearly establish its fundamental algorithmic contribution over existing adaptive MAD methods, particularly why the three stages must be combined in the proposed order. What is the most indispensable algorithmic contribution of StageMAD beyond PRM-based self-revision and adaptive MAD?

A1: Our contribution combines a communication-control perspective with a mechanism for using peer reasoning without treating a selected peer as an answer authority. We study how we can use process evidence to determine which agents need assistance, who should provide it, what information should be exchanged, and when further discussion is warranted. StageMAD uses PRM judgments to control these decisions while leaving reasoning and revision to the agents. This extends the role of the PRM from identifying errors within individual responses to organizing communication between agents.

We chose the proposed order to make peer communication conditional on what remains unresolved after local self-correction. Stage 1 first allows agents to revise their reasoning using their own trajectories and PRM feedback, without exposure to peer responses. Stage 2 then provides restricted peer guidance for the remaining suspected errors. Stage 3 uses the resulting votes to decide whether close disagreements require mutual verification. Providing guidance before self-correction (Stage 2 -> Stage 1) allocates communication before observing which errors agents can resolve independently. Also, verification before guidance (Stage 3 -> Stage 2) assesses disagreements that subsequent guidance may change. The proposed order implements a progressive communication policy: first allow local self-correction, then provide targeted peer guidance, and finally verify the remaining close disagreements.

The central mechanism of StageMAD is process-guided, answer-free peer guidance in Stage 2. We use a source judged more reliable to provide a concrete check, such as a condition or equation to revisit at the target’s suspected error, without revealing its final answer or full derivation. The target preserves its own reasoning prefix and independently regenerates the masked suffix using this guidance. Thus, selecting a source does not require the target to adopt that source’s answer. This design allows peer reasoning to assist local correction while limiting the direct propagation of potentially incorrect answers. Table 9 supports this mechanism by showing that answer-free guidance reduces agreement between the target’s revised answer and the source’s answer compared with providing full peer solutions.

Q2: The benefits of multi-agent communication over PRM-based self-revision are not consistently demonstrated. Table 1 shows that although StageMAD outperforms most MAD baselines in many settings, it does not consistently surpass PRM self-revision. For example, with Qwen3.5-9B, PRM self-revision substantially outperforms StageMAD on MATH, MMLU-Pro, and AIME24. This weakens the claim that multi-agent communication provides additional benefits.

A2: You mention that PRM self-revision gets higher accuracy than StageMAD in several settings, and we agree that the additional benefits of peer communication over independent PRM self-revision are dependent on the model and the task.

Two questions should be distinguished: whether communication offers a useful accuracy–cost trade-off compared with independent revision, and whether controlled communication improves how agents use the same candidate pool. Our non-communicative baselines address the first question, while the matched-pool experiments in Figure 4 address the second.

For the first question, StageMAD achieves similar or higher average accuracy with fewer tokens in some settings. On GSM8K with Qwen2.5-7B-Instruct, StageMAD achieves 93.8% compared with 93.5% for PRM self-revision, using about 60% fewer total tokens. On MMLU-Pro with DeepSeek-Math-7B, it achieves 34.0% compared with 27.4%, using about 33% fewer total tokens. These results demonstrate a useful trade-off in these settings, rather than a consistent accuracy advantage over independent revision.

For the second question, Figure 4 fixes the Round-1 candidates before comparing subsequent communication methods. StageMAD achieves the highest among the MAD policies compared in 11 of the 12 settings. Its advantage over those policies therefore cannot be attributed solely to Stage 1 producing better initial candidates.

Our main contribution is to improve communication within MAD. In the revised manuscript, we will make this scope explicit and distinguish the improvement over the evaluated MAD policies from the model- and task-dependent benefits over non-communicative inference methods.

Q3: The method's robustness to PRM errors is insufficiently investigated. StageMAD relies on step-level PRM judgments to identify errors, select communication sources, and determine which agents require correction. Incorrectly accepting flawed reasoning or rejecting valid reasoning could lead to error propagation or unnecessary revisions. The paper lacks a sufficiently comprehensive analysis of PRM quality and robustness.

A3: StageMAD uses PRM judgments as process evidence rather than as a final arbiter of correctness. In particular, an incorrect source ranking does not automatically lead to adoption of the source’s answer. Stage 2 separates selecting a guidance source from accepting its conclusion: the source provides answer-free guidance about a suspected error in the target’s reasoning, while the target remains responsible for revising its own trajectory.

To examine the risk of unreliable source selection, we measured how often an incorrect answer group ranked first under the Stage 2 group-selection score with Qwen2.5-7B-Instruct. The analysis includes question instances with both a scored correct group and a scored incorrect group after Round 1.

DatasetIncorrect group ranked first by Stage 2 group-selection score
MATH25.6% (81/317)
GSM8K20.9% (33/158)
MMLU-Pro43.2% (256/593)
AIME2425.0% (2/8)

The results confirm that an incorrect answer group can rank ahead of a correct group, particularly on MMLU-Pro. This motivates restricting what a selected source can transmit. Rather than providing its final answer or full derivation, the source supplies a concrete check concerning the target’s suspected error, such as a condition or equation to revisit. The target preserves its own reasoning prefix and independently regenerates the masked suffix using this guidance. A source with an incorrect final answer may still provide a useful local check, but its conclusion is not directly supplied as the answer to follow.

Table 9 evaluates this message restriction under matched interventions, holding the Round-1 candidate pools, PRM scores, and revision targets fixed. Compared with full peer solutions, answer-free guidance reduced agreement between the target’s revised answer and the source’s answer on MATH from 76.7% to 20.3%. On MMLU-Pro, it reduced the correct-to-wrong rate among initially correct revision targets from 85.6% to 62.9%. These results show that the communication design affects how strongly a selected source influences the target: restricting the message reduces answer following and harmful revisions in these comparisons.

Two remaining failure modes are incorrect guidance influencing the target’s revision and erroneous trajectories escaping revision when GenPRM judges all their steps positively. We will discuss these alongside the source-ranking analysis in the revised manuscript. Together, the analyses explain why StageMAD combines process-based source selection with restrictions on the information that sources can transmit.

Q4: Generalizability beyond mathematical reasoning remains uncertain. StageMAD achieves limited improvements on MMLU-Pro, while the CommonsenseQA results in the appendix even show performance degradation. These findings suggest that the method may be better suited to mathematical reasoning, and its applicability to general language reasoning tasks requires further validation.

A4: We agree that the evidence is more consistent for mathematical reasoning. The original Limitations section (Lines 481–483) discusses the limits of a process reward model geared toward mathematics on language-oriented benchmarks. Appendix I.2 (Tables 11–12) reports the MMLU-Pro and CommonsenseQA results and their stage-wise analysis.

In the Qwen2.5-7B-Instruct run on CommonsenseQA, accuracy fell from 81.2% after the initial vote to 79.9% at the end. In each of the three stages, more questions went from correct to wrong than from wrong to correct. In the MMLU-Pro run we analyzed, the final net gain was 10 correct answers, but Stage 3 alone led to a net loss of 7. The agent-level analysis in the appendix also shows that local self-correction sometimes changed correct answers to wrong ones on both datasets before peer communication began.

We will include these results in the main results discussion. The gains are more consistent on mathematical reasoning tasks and vary by model and task on the language benchmarks we evaluated. The current results provide limited evidence for broader generalization beyond mathematics.

Q5: The computational overhead and parameter sensitivity require further analysis. StageMAD introduces multiple stages of PRM evaluation and additional model calls. In some settings, its token consumption is substantially higher than that of the baselines. Token counts alone, however, are insufficient to characterize actual inference latency and computational costs. Furthermore, the sensitivity of the communication thresholds and voting-gating parameters requires more thorough investigation.

A5: We supplemented the token counts with observed batch wall-clock times for Qwen2.5-7B-Instruct. Within each dataset, StageMAD and PRM self-revision start from the same Round-0 candidates. The totals include Round 0 and all three subsequent rounds, and token counts include both task-model and GenPRM input and output tokens. The recorded times exclude model initialization, dataset loading, and checkpoint writes.

DatasetStageMAD tokens (M)PRM self-revision tokens (M)StageMAD time (min)PRM self-revision time (min)
MATH57.3394.24269.8576.6
GSM8K19.5649.06105.7257.7
MMLU-Pro66.0397.46472.6421.5
AIME244.475.1118.524.5

StageMAD uses fewer tokens in all four runs and has shorter observed execution time on MATH, GSM8K, and AIME24. We will include both measurements in the revised manuscript. Our response to HK4Z, Q4 provides the breakdown by round and separates task-model and GenPRM tokens.

We also varied α2\alpha_2 and δU\delta_U one at a time with Qwen2.5-7B-Instruct, starting from the same frozen Round-1 candidate pool within each dataset. For MATH, GSM8K, and MMLU-Pro, we ran Stages 2 and 3 once per distinct guidance configuration. The table reports final accuracy after Stage 3.

α2\alpha_2δU\delta_UMATHGSM8KMMLU-ProAIME24
0.25 (default)0.0271.8%93.0%58.4%10.0%
0.000.0270.3%93.3%58.8%10.0%
0.500.0271.2%93.3%58.7%10.0%
0.250.0170.4%93.1%58.4%16.7%
0.250.0471.3%93.1%58.5%13.3%

Across the tested settings, accuracy varies by 1.5 percentage points on MATH, 0.3 on GSM8K, and 0.4 on MMLU-Pro. On these three datasets, changing α2\alpha_2 generally alters more guidance routes than changing δU\delta_U. On AIME24, accuracy ranges from 10.0% to 16.7% (3–5 of 30 questions).

Table 2 compares the default δv=1\delta_v=1 gate with ungated Stage 3 from the same frozen Round-2 candidate pools. The gate reduces the share of questions that enter Stage 3 from 52.1% to 24.8% on MATH and from 18.4% to 4.7% on GSM8K, while improving final accuracy on both. Our response to HK4Z, Q5 further examines gating by the full vote pattern.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.