acceptodds
Under review as a conference paper at ICLR 2027

Beyond Answer Sharing: TrueMAD for Targeted Peer Engagement in Multi-Agent Debate

Abstract

Multi-agent debate (MAD) aims to improve large language model reasoning by exposing agents to alternative solutions and enabling iterative revision. However, standard broadcast-style MAD makes peer answers visible without requiring agents to engage with a particular peer claim. As a result, reliable reasoning may be suppressed by conflicting or majority responses, limiting achievable performance, while erroneous reasoning may persist and degrade final predictions without actionable feedback. We propose **TrueMAD**, a backbone-agnostic test-time plug-in that introduces one-to-one, free-form peer advice between answer broadcasting and self-revision. Each source agent responds to a target's current reasoning through support, rebuttal, refinement, or further questions, while the target retains control over how to revise. The same mechanism can augment standard MAD and its improved variants without parameter updates. Across AIME, GPQA-Diamond, and TruthfulQA with *Qwen3-30B-A3B*, **TrueMAD** improves agent-level and strict-majority accuracy over MAD by and percentage points on average, while **TrueDMAD** improves DMAD by and points, respectively. Analyses at the utterance, agent-transition, and group levels further show more peer-specific deliberation, more correction-oriented revisions, and more favorable consensus dynamics. Our code is available at https://anonymous.4open.science/r/TrueMAD-ICLR/.

Primary area:
generative models
Keywords:
Multi-Agent Debate, Multi-Agent System, Large Language Model

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_fFLTOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents TrueMAD, a framework that creates different channels between a targeted agent and its peer. Different from existing multi-agent debate protocols, which often broadcast the conversation and reasoning to every agent, TrueMAD aims to address conformity (suppressed by the majority answer) and unaddressed reasoning in existing MAD.

Critical strengths
  1. Separating the channel and avoiding broadcasting everything to every agent is a sound and intuitive design.

  2. Multi-agent debate has been well explored in the community, making it hard to achieve consistent improvement, especially with less hand-designed or post-hoc heuristics. However, the evaluation on AIME, GPQA and TruthfulQA shows that TrueMAD is effective across models and datasets.

Critical weaknesses
  1. The ordered source-target pair, noted in line 225, makes the framework expensive. If I understand correctly, assuming 3 agents in a debate, this will form 6 ordered pair, i.e., (1,2), (1,3), (2,3), (3,2), (3,1), (2,1) and this can go for multiple rounds. So scaling this framework is harder than existing work (for example, if we want 10 agents to participate in a debate).

  2. Several representative baselines are missing. For example, Reconcile [1] and Mixture-of-Agent [2] should be compared. Most importantly, it is not clear whether all the baselines are compute-matched. That is, if TrueMAD uses 3 model instances with 5 rounds, the baselines should also get up to 15 LLM calls. Or the token usage should roughly match.

  3. The authors should also report compute-matched self-consistency [3] and Best-of-N [4]. I would expect these to be strong but easy-to-implement baselines, and that would justify whether we need a much more complex pipeline.

  4. In section 4.4, there is an analysis about "auxiliary-call budget", but I didn't fully get it. What does "auxiliary-call budget is allocated to feedback" mean? My understanding is that this is different from the LLM call or token budget I mentioned in points 2 and 3. But if not, please clarify. Also in this budget-matched comparison, the margin is smaller than the main result table, suggesting that the main result table is not budget-matched.

References:

Questions for authors
  1. In lines 175-179, the authors mention: "Increasing the number of agents or debate rounds expands answer exposure, but does not ensure that any agent examines a specific peer claim. Strengthening a generic revision prompt may elicit more elaborate responses while leaving the source and target of feedback implicit. Imposing fixed adversarial roles guarantees opposition rather than evidence-sensitive engagement, and may unnecessarily challenge valid reasoning." I am curious whether there are any empirical results or prior work that can support these hypotheses.

  2. Can we use different model backbones in the debate, instead of using the same model multiple times?

4 comments
AuthorsOpenReview ↗

Response to Reviewer fFLT (1/2)

We thank the reviewer for the constructive feedback. We added the requested baselines and token-budget comparisons and clarified TrueMAD's computational overhead and motivation.

1. Scalability and compatibility with different backbones (W1, Q2)

Your interpretation is correct. The fully connected advice stage requires N(N−1) calls per debate update. Our four-agent setting is consistent with recent MAD studies. ReConcile [1] and DebUnc [2] use three agents, and Free-MAD [3] evaluates three- and four-agent configurations.

Our objective is a simple, effective plug-in that makes peer engagement explicit while preserving the underlying debate protocol. The all-pairs implementation has quadratic advice-call overhead. Extending it to larger groups is an important direction. Pruning redundant interactions or routing advice to a bounded number k of peers per agent would reduce advice calls to O(Nk) per update while retaining directed feedback and avoiding exhaustive pairwise communication.

The text-based advice interface can accommodate mixed-backbone teams. Our TrueDMAD experiments demonstrate compatibility with heterogeneous reasoning strategies.

2. Additional representative MAD baselines (W2)

Following your suggestion, we added ReConcile, MoA [4], and Free-MAD. Comparisons use Qwen3-30B-A3B, the main experiments' evaluation data and debate-round setting, and three independent seeds. ReConcile and MoA use same-backbone implementations. Metrics follow the main table, with values reported as mean ± standard deviation (%).

MethodAIME Agent ACCAIME Vote ACCGPQA-Diamond Agent ACCGPQA-Diamond Vote ACC
MAD29.81 ± 2.4929.63 ± 2.5754.12 ± 1.2952.86 ± 1.27
ReConcile13.83 ± 2.8325.93 ± 3.2152.69 ± 2.0452.69 ± 2.04
MoA29.63 ± 1.7029.63 ± 1.7054.04 ± 1.8253.91 ± 2.07
Free-MAD31.76 ± 0.1630.74 ± 0.6454.38 ± 0.2954.59 ± 1.67
TrueMAD35.37 ± 2.3634.81 ± 1.7058.00 ± 0.6056.73 ± 0.29

TrueMAD outperforms all three added baselines on both metrics and datasets. TrueMAD exceeds the strongest added baseline in Vote ACC by 4.07 points on AIME and 2.14 points on GPQA-Diamond. These comparisons cover confidence-based consensus, layered aggregation, and consensus-free debate. The token-matched comparisons below address the budget question.

Rebuttal time constraints limit the current comparisons to AIME and GPQA-Diamond. We will include TruthfulQA results in the updated manuscript once completed.

AuthorsOpenReview ↗

Response to Reviewer fFLT (2/2)

3. Token-matched Self-Consistency and Best-of-N (W3)

We agree that these are important controls. We added Self-Consistency [5] and Best-of-N [6], selecting candidate counts K to match TrueMAD’s total-token budget through independent sampling and candidate selection. Post-experiment token accounting confirmed comparable consumption. Both baselines use Qwen3-30B-A3B and three independent seeds, consistent with the main experiments.

MethodAIME Final-answer accuracy (%)GPQA-Diamond Final-answer accuracy (%)
Self-Consistency29.26 ± 0.6454.71 ± 1.05
Best-of-N32.22 ± 1.9253.87 ± 0.58
TrueMAD34.81 ± 1.7056.73 ± 0.29

Under comparable token budgets, TrueMAD exceeds the stronger baseline on AIME and GPQA-Diamond by 2.59 and 2.02 percentage points, respectively. This supports targeted peer engagement as an effective use of inference compute relative to additional independent sampling or candidate selection.

4. Clarifying the auxiliary-call budget and supporting the motivation (W4, Q1)

For W4, “auxiliary calls” are additional LLM calls that generate feedback before answer revision. Section 4.4 matches their number across TrueMAD, ComputeMAD, and Self-AdviceMAD while varying feedback generation. This matches calls, and token consumption can differ. The main table matches agent counts and response rounds, while the new SC/BoN comparisons match total-token budgets. We clarified these distinctions in the revised Section 4.4.

For Q1, we refined the motivation and added supporting literature to distinguish protocol-level observations from empirical findings. See the revised manuscript's motivation discussion (lines 181–189).

We hope these responses address your concerns and look forward to your further feedback.

[1] Chen J, Saha S, Bansal M. ReConcile: Round-Table Conference Improves Reasoning via Consensus among Diverse LLMs. ACL 2024.

[2] Yoffe L, Amayuelas A, Wang W Y. DebUnc: Improving Large Language Model Agent Communication with Uncertainty Metrics. Findings of EMNLP 2025.

[3] Cui Y, Fu H, Zhang H, et al. Free-MAD: Consensus-Free Multi-Agent Debate. Findings of ACL 2026.

[4] Wang J, Wang J, Athiwaratkun B, et al. Mixture-of-Agents Enhances Large Language Model Capabilities. ICLR 2025.

[5] Wang X, Wei J, Schuurmans D, et al. Self-Consistency Improves Chain of Thought Reasoning in Language Models. ICLR 2023.

[6] Lightman H, Kosaraju V, Burda Y, et al. Let's Verify Step by Step. ICLR 2024.

Dear Reviewer,

We have carefully addressed all your comments and provided point-by-point responses. As the rebuttal period is nearing its end, we would be grateful if you could take a moment to review our replies.

Please let us know if you have any further concerns or suggestions.

Thank you very much.

Thank the authors for their detailed responses. Whilte they have mostly addressed my questions, the scalability issue remains. Therefore, after careful thought, I decided to keep my original score.

Reviewer_Gt81OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

TrueMAD takes conventional multi-agent debate and adds a peer-review stage, where before every round each agent gets an opportunity to critique each other agent's answer, following which each agent can revise their answer. This is evaluated on math science and truthfulness benchmarks, where the method outperforms the provided baseline debate methods.

Critical strengths

Adding a deliberate 'review' stage to the debate process is a very intuitive idea where agents are able to cross-check each others' reasoning, moving debate past an aggregation of reasoning traces and consensus based decision making to a more 'multi-agent self-refine' approach.

Critical weaknesses

Missing compute matched comparisons with key consensus-based, debate, and peer review baselines, including (but not limited to): https://arxiv.org/abs/2203.11171 https://arxiv.org/abs/2311.08152v2 (very similar conceptually to this work; please clarify novelty with respect to other multi-agent peer review works) https://arxiv.org/abs/2509.20502 Also worthwhile to include consensus free methods like https://aclanthology.org/2026.findings-acl.1600/

There are several works that target how to improve MAD protocol, this work would be stronger with a more thorough comparison against contemporary consensus and consensus-free methods. Adding the peer review stage increases the number of model calls available to the system and this should be matched among the various external methods compared.

Questions for authors

Please see weaknesses; would be good to add some comparisons against recent debate methods.

3 comments
AuthorsOpenReview ↗

Response to Reviewer Gt81

We thank the reviewer for valuing deliberate peer review and suggesting comparisons. We added the recommended peer-review and consensus-free methods and other baselines, and clarify TrueMAD's motivation, design, and evaluation relative to MAPR below.

1. Additional comparisons with peer-review, consensus-based, and consensus-free methods

We added MAPR [1], MARS [2], and Free-MAD [3], alongside Mixture-of-Agents (MoA) [4] and ReConcile [5]. Experiments use Qwen3-30B-A3B on AIME and GPQA-Diamond, reporting mean ± standard deviation (%). ReConcile and MoA use same-backbone implementations.

MethodAIME Agent ACCAIME Final ACCGPQA-Diamond Agent ACCGPQA-Diamond Final ACC
MAD29.81 ± 2.4929.63 ± 2.5754.12 ± 1.2952.86 ± 1.27
MAPR32.04 ± 0.6432.22 ± 1.1152.69 ± 2.3952.48 ± 2.10
Free-MAD31.76 ± 0.1630.74 ± 0.6454.38 ± 0.2954.59 ± 1.67
MoA29.63 ± 1.7029.63 ± 1.7054.04 ± 1.8253.91 ± 2.07
MARS—22.96 ± 1.28—54.71 ± 0.77
ReConcile13.83 ± 2.8325.93 ± 3.2152.69 ± 2.0452.69 ± 2.04
TrueMAD35.37 ± 2.3634.81 ± 1.7058.00 ± 0.6056.73 ± 0.29

Final ACC denotes the reported Vote ACC for methods with agent-level predictions and final-answer accuracy for MARS. Rebuttal time constraints limit comparisons to AIME and GPQA-Diamond. We will include TruthfulQA results in the updated manuscript when ready.

The suggested Self-Consistency [6] baseline achieves final-answer accuracies of 29.26 ± 0.64% on AIME and 54.71 ± 1.05% on GPQA-Diamond.

2. Distinction from MAPR in motivation, interaction design, and evaluation

Motivation. MAPR addresses limitations of individual self-correction by providing peer perspectives that help agents reconsider their reasoning. TrueMAD addresses a failure within existing MAD, where multiple reasoning paths are available, yet interaction can fail to exploit them or overturn a correct answer already present. For example, when one agent is correct and three are wrong, debate may persuade the correct agent to abandon its answer without helping others identify their errors. This motivates targeted help for erroneous reasoning and support for reliable reasoning during revision, enabling the group to better use its existing reasoning.

Interaction design. Our advice stage augments an existing MAD protocol while preserving its initialization, reasoning procedures, and final aggregation. Each message is generated independently from the question and current source–target response pair, then routed to the target's existing revision procedure. In MAPR's released implementation, a source sequentially reviews targets while accumulating preceding review dialogue. Our feedback can support a sound argument, identify an error, or request clarification according to the target's reasoning. Applying the same intervention to standard MAD and strategy-diverse DMAD lets us examine its effects across reasoning procedures.

Evaluation. Our contribution connects failures of group interaction to a composable intervention, evaluated at three levels covering peer-engagement specificity and independence, individual correction and harmful revision, and recovery from erroneous majorities or expansion of correct minorities. This links the motivation and method to observable debate dynamics alongside final accuracy.

These distinctions show how TrueMAD’s focus on group-interaction failures shapes its design and evaluation. We hope they clarify its contribution relative to MAPR.

3. Results analysis

TrueMAD achieves the highest accuracy among the completed multi-agent comparisons on both datasets. It exceeds MAPR on AIME by 3.33 points in Agent ACC and 2.59 points in Final ACC, and MARS, the strongest completed added baseline on GPQA-Diamond, by 2.02 points in Final ACC. Gains over MAD and DMAD and the Pairwise-AnalysisMAD ablation support the advice stage's applicability and the value of recipient-oriented feedback. Consistently improved error correction further connects these gains to our motivation of making group interaction more productive.

We welcome your further feedback and discussion.

[1] Xu Z, Shi S, Hu B, et al. Towards Reasoning in Large Language Models via Multi-Agent Peer Review Collaboration. arXiv:2311.08152, 2023.

[2] Wang X, Wang J, Wang Y, et al. MARS: Toward More Efficient Multi-Agent Collaboration for LLM Reasoning. arXiv:2509.20502, 2025.

[3] Cui Y, Fu H, Zhang H, et al. Free-MAD: Consensus-Free Multi-Agent Debate. Findings of ACL 2026.

[4] Wang J, Wang J, Athiwaratkun B, et al. Mixture-of-Agents Enhances Large Language Model Capabilities. ICLR 2025.

[5] Chen J, Saha S, Bansal M. ReConcile: Round-Table Conference Improves Reasoning via Consensus among Diverse LLMs. ACL 2024.

[6] Wang X, Wei J, Schuurmans D, et al. Self-Consistency Improves Chain of Thought Reasoning in Language Models. ICLR 2023.

Dear Reviewer,

We have carefully addressed all your comments and provided point-by-point responses. As the rebuttal period is nearing its end, we would be grateful if you could take a moment to review our replies.

Please let us know if you have any further concerns or suggestions.

Thank you very much.

I thank the authors for their rebuttal and for adding more comparisons, however I still have concerns about how well this method actually scales and works across various tasks and models; as such, have decided to keep my original score thanks.

Reviewer_64wcOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes TrueMAD, a training-free plug-in for multi-agent debate (MAD) that introduces targeted peer feedback before answer revision. Unlike conventional MAD, which broadcasts peer responses without requiring explicit engagement, TrueMAD enables agents to provide one-to-one feedback through support, rebuttal, refinement, or further questions. The method can be integrated into both standard MAD and DMAD without parameter updates. Experiments on AIME, GPQA-Diamond, and TruthfulQA demonstrate consistent improvements in agent-level and majority-vote accuracy. Additional analyses of reasoning quality, answer transitions, and consensus dynamics suggest that targeted feedback promotes more effective error correction and collective decision-making.

Critical strengths

[#1] Interaction-Level Limitation of Conventional MAD Introducing explicit source–target feedback is a straightforward and interpretable intervention that addresses this interaction-level limitation without modifying the underlying model parameters. The compatibility with both MAD and DMAD also makes the proposed mechanism practically relevant.

[#2] Consistent Empirical Improvements The proposed method achieves improvements over its corresponding MAD and DMAD baselines across all three primary benchmarks. The evaluation goes beyond final accuracy by examining round-wise performance, initial correctness conditions, individual answer transitions, and group-level consensus behavior.

Critical weaknesses

[#1] Insufficient isolation of targeted peer engagement from additional pairwise reasoning and inference overhead.

The paper's central contribution is the introduction of explicit source–target engagement as an interaction mechanism for multi-agent debate. However, TrueMAD introduces N(N−1)N(N-1) additional advice-generation calls per debate transition, resulting in 48 auxiliary calls per question under the reported four-agent, five-round configuration.

The authors partially address this concern through ComputeMAD and Self-AdviceMAD in Section 4.4, which match the number of auxiliary calls. These controls are valuable, and TrueMAD consistently achieves higher mean accuracy across the three benchmarks. Nevertheless, they do not fully isolate the effect of targeted engagement. ComputeMAD generates feedback from the entire answer pool without specifying a recipient, whereas Self-AdviceMAD generates critiques of an agent's own response. Consequently, these comparisons vary not only the targeting mechanism but also the information available to the feedback generator and the structure of the reasoning task.

A more direct comparison would preserve the same source–target response pairs while removing the explicit recipient-oriented feedback instruction. Furthermore, matching the number of calls does not guarantee comparable token consumption or computational cost, neither of which is quantitatively reported.

Therefore, the experiments demonstrate the effectiveness of the complete TrueMAD protocol, but provide limited evidence that explicit target-oriented engagement, rather than additional pairwise reasoning or differences in feedback generation, is responsible for the observed improvements. This distinction is important because the paper's primary contribution is presented as a novel interaction protocol.

[#2] Limited direct evidence connecting targeted advice to the claimed correction and consensus mechanisms.

The paper introduces several diagnostic metrics, including Correction Preference Lift (CPL), Asymmetric Majority Escape (AME), and Correct-Minority Propagation (CMP1v3), to characterize how targeted feedback affects debate dynamics. While these metrics provide useful evidence of favorable behavioral changes, they do not directly establish the mechanisms underlying those changes.

[2-1] CPL summarizes the relative frequency of wrong-to-correct and correct-to-wrong transitions through their ratio. However, reporting only this ratio obscures whether improvements arise primarily from correcting incorrect answers, preserving correct answers, or both.

[2-2] AME measures the relative tendency to leave erroneous versus correct majorities. Although this captures an important aspect of consensus dynamics, leaving an erroneous majority does not necessarily imply that an agent has identified and corrected the underlying reasoning error.

[2-3] CMP1v3 measures whether a state with exactly one correct agent transitions to a state with multiple correct agents. This indicates an increase in the number of correct responses but does not establish whether the initially correct agent's reasoning was preserved or whether its reasoning was responsible for the improvement.

These distinctions are particularly relevant because the paper motivates TrueMAD through two specific failures: suppression of reliable reasoning and insufficient correction of erroneous reasoning. The current analyses are consistent with improvements in these behaviors, but do not directly trace how generated advice influences the recipient's subsequent reasoning and answer revision.

Additionally, the utterance-level analysis relies on a single LLM judge to evaluate Rebuttal Precision, Evidence Independence, and Novel Contribution, without reported independent validation of these judgments.

Overall, the empirical analyses support improvements in answer accuracy and debate dynamics, but the evidence connecting these improvements to the proposed mechanisms of targeted error correction and preservation of reliable reasoning remains indirect.

Questions for authors

Q1. Isolating the contribution of targeted peer engagement

The current auxiliary-call-matched baselines (ComputeMAD and Self-AdviceMAD) do not fully isolate the contribution of explicit source–target engagement, as they differ from TrueMAD in both the information provided and the feedback-generation structure. Could the authors introduce a more controlled comparison that preserves the same pairwise inputs and computational budget while removing the explicit target-oriented feedback instruction? This would help establish whether the improvements are attributable to targeted engagement itself rather than additional pairwise reasoning.

Q2. Computational efficiency and practical scalability

TrueMAD requires (N(N-1)) additional advice-generation calls per debate transition, introducing quadratic communication complexity with respect to the number of agents. Although Section 4.4 controls for the number of auxiliary calls, the actual token consumption and inference overhead remain unclear. Could the authors report total input/output tokens, inference latency, and accuracy under matched token or computational budgets? These results are important for determining whether the performance gains justify the additional inference cost.

Q3. Validating the claimed correction mechanism

The proposed CPL, AME, and CMP1v3 metrics characterize favorable answer-transition and consensus patterns, but do not directly establish that targeted advice causes error correction or preserves correct minority reasoning. Could the authors provide the underlying wrong-to-correct and correct-to-wrong transition probabilities, together with an analysis of whether the generated advice identifies specific reasoning errors that are subsequently corrected by the recipients? Such evidence would more directly substantiate the paper's central mechanistic claims.

Q4. Reliability of the deliberation-quality evaluation

The utterance-level analysis relies on a single LLM judge to assess Rebuttal Precision, Evidence Independence, and Novel Contribution. However, the reliability of these judgments is not independently established. Could the authors validate these metrics using human assessments or an alternative LLM judge and report the agreement between evaluations? This would strengthen the claim that TrueMAD improves substantive peer deliberation rather than merely producing responses that receive higher model-based evaluation scores.

6 comments
AuthorsOpenReview ↗

Response to Reviewer 64wc (1/2)

We thank the reviewer for the constructive feedback. We address token-matched comparisons and a pairwise ablation (W1, Q1–Q2), refined behavioral analyses (W2, Q3), and validation with an alternative LLM judge (Q4).

1. Token-budget comparisons and isolation of target-oriented feedback (W1, Q1–Q2)

Following your suggestion, we added Pairwise-AnalysisMAD, changing only TrueMAD's feedback-generation instruction to third-person analysis of two agents' responses. The pairwise input structure and revision protocol are preserved, while response texts can differ as debate trajectories evolve. This tests recipient-oriented feedback beyond generic pairwise reasoning.

We also added Self-Consistency [1] and Best-of-N [2], selecting candidate counts K to match TrueMAD's total-token budget through independent sampling and candidate selection. Post-experiment token accounting confirmed comparable consumption for all three baselines. These comparisons use Qwen3-30B-A3B and three independent seeds, consistent with the main experiments. Rebuttal time constraints limit the current comparisons to AIME and GPQA-Diamond. We will include TruthfulQA results in the updated manuscript once completed.

MethodAIME Agent ACC (%)AIME Final ACC (%)GPQA-Diamond Agent ACC (%)GPQA-Diamond Final ACC (%)
Self-Consistency—29.26 ± 0.64—54.71 ± 1.05
Best-of-N—32.22 ± 1.92—53.87 ± 0.58
Pairwise-AnalysisMAD30.37 ± 1.6731.11 ± 1.1154.42 ± 0.9151.35 ± 0.29
TrueMAD35.37 ± 2.3634.81 ± 1.7058.00 ± 0.6056.73 ± 0.29

On AIME and GPQA-Diamond, TrueMAD exceeds Pairwise-AnalysisMAD by 3.70 and 5.38 percentage points in final accuracy and 5.00 and 3.58 points in agent accuracy, respectively. It also exceeds the stronger of Self-Consistency and Best-of-N on each dataset by 2.59 and 2.02 points in final accuracy.

These results support recipient-oriented feedback over third-person pairwise analysis in both mean agent-level and final accuracy. The SC/BoN results further support its effectiveness relative to independent sampling or candidate selection under comparable total-token budgets.

[1] Wang X, Wei J, Schuurmans D, et al. Self-Consistency Improves Chain of Thought Reasoning in Language Models. arXiv:2203.11171.

[2] Lightman H, Kosaraju V, Burda Y, et al. Let's Verify Step by Step. ICLR 2024.

AuthorsOpenReview ↗

Response to Reviewer 64wc (2/2)

2. Decomposing the correction and consensus diagnostics (W2, Q3)

We agree that the original aggregate metrics leave distinct behavioral explanations unresolved. We therefore examine events corresponding to your concerns, including wrong-to-correct and correct-to-wrong transitions, recovery of erroneous-majority supporters, and retention of the originally correct agent as others improve. These answer-level diagnostics should be interpreted separately from tracing the transmission of a particular reasoning step.

Each entry below shows baseline → augmented method for MAD → TrueMAD or DMAD → TrueDMAD. Values are percentages.

DatasetComparisonP(W→C | W) ↑P(C→W | C) ↓Wrong-majority supporter → correct ↑
AIMEMAD → TrueMAD4.22 → 5.754.96 → 2.940.78 → 0.37
AIMEDMAD → TrueDMAD4.93 → 6.946.15 → 6.880.42 → 1.35
GPQA-DiamondMAD → TrueMAD6.71 → 12.245.24 → 7.432.20 → 4.59
GPQA-DiamondDMAD → TrueDMAD12.70 → 15.648.90 → 7.835.64 → 7.95
TruthfulQAMAD → TrueMAD4.00 → 10.590.93 → 2.200.86 → 4.69
TruthfulQADMAD → TrueDMAD13.65 → 17.303.68 → 3.157.08 → 9.57

Consistently stronger error correction (CPL). P(W→C | W) increases in all six comparisons, showing consistently improved recovery from incorrect answers. A possible explanation for the increased P(C→W | C) in some settings is that targeted feedback encourages closer reconsideration of specific reasoning steps, creating correction opportunities while allowing imperfect peer advice to overturn correct answers. Together with higher CPL in all six comparisons, these results support a stronger correction preference, with correct-answer retention varying by setting.

Recovery from erroneous majorities (AME). To address whether leaving a wrong majority leads to a correct answer, we measure how often erroneous-majority supporters answer correctly in the next round. This probability increases in most comparisons, including 2.20% → 4.59% on GPQA-Diamond and 0.86% → 4.69% on TruthfulQA for MAD → TrueMAD. The refined diagnostic thus supports improved recovery from erroneous majorities in most evaluated settings.

Preserving the originally correct agent as others improve (CMP). To assess whether higher CMP accompanies retention of the originally correct agent, we track transitions from states with exactly one correct agent. H denotes that agent remaining correct, and S denotes at least one other agent becoming correct. Their joint occurrence, H∩S, captures retention of the original correct agent as correctness expands to others.

DatasetComparisonH∩S (%)CMP (%)
AIMEMAD → TrueMAD20.3 → 48.325.0 → 55.0
AIMEDMAD → TrueDMAD22.8 → 29.640.4 → 47.9
GPQA-DiamondMAD → TrueMAD24.3 → 24.725.4 → 32.4
GPQA-DiamondDMAD → TrueDMAD12.2 → 30.036.0 → 47.3
TruthfulQAMAD → TrueMAD12.8 → 20.714.1 → 25.0
TruthfulQADMAD → TrueDMAD38.2 → 38.147.4 → 54.8

H∩S increases or remains numerically comparable across all six comparisons. Together with consistently higher CMP, this indicates a maintained or increased frequency of group improvement in which the originally correct agent stays correct while another becomes correct. These results strengthen the evidence that targeted feedback supports group improvement while retaining the original correct agent.

These finer-grained diagnostics clarify the answer-level changes accompanying TrueMAD’s gains and sharpen the interpretation of the correction and consensus results.

3. Validation with an alternative LLM judge (Q4)

We added GPT-6-luna as a second judge alongside DeepSeek-V4-Flash. Both assign higher scores to TrueMAD than MAD and to TrueDMAD than DMAD on Rebuttal Precision (RP), Evidence Independence (EI), and Novel Contribution (NC) across all three datasets.

The table reports score differences as ΔRP / ΔEI / ΔNC.

DatasetComparisonOriginal judgeGPT-6-luna
AIMETrueMAD − MAD+0.66 / +0.99 / +0.36+0.450 / +0.516 / +0.186
AIMETrueDMAD − DMAD+0.63 / +0.43 / +0.27+0.964 / +0.525 / +0.447
GPQA-DiamondTrueMAD − MAD+1.00 / +1.44 / +0.87+0.444 / +0.615 / +0.293
GPQA-DiamondTrueDMAD − DMAD+0.45 / +0.34 / +0.33+0.812 / +0.653 / +0.491
TruthfulQATrueMAD − MAD+1.58 / +2.44 / +2.01+0.890 / +1.366 / +1.038
TruthfulQATrueDMAD − DMAD+0.11 / +0.36 / +0.42+0.733 / +0.811 / +0.731

The judges agree on the direction of improvement in all 18 aggregate comparisons across three datasets, two protocol pairs, and three dimensions, supporting the robustness of the baseline-relative trends across the two judge models.

We hope this response addresses your concerns and look forward to further discussion.

Thank you for the detailed rebuttal and the additional experiments. I appreciate the authors' efforts to address all four questions raised in my review.

The new Pairwise-AnalysisMAD baseline provides a substantially more direct comparison for isolating recipient-oriented feedback, and the token-matched Self-Consistency and Best-of-N experiments strengthen the empirical justification for the additional inference budget. I also find the decomposed answer-transition analyses and the alternative LLM judge evaluation useful.

Overall, these additions address a substantial portion of my original concerns.

I have two remaining questions regarding the interpretation of the new results.

1. Correction versus preservation of initially correct answers.

The new transition analyses show that wrong-to-correct transitions increase across all six comparisons, which strengthens the evidence for improved error correction. However, correct-to-wrong transitions also increase in some settings, including GPQA-Diamond (MAD → TrueMAD: 5.24% → 7.43%) and TruthfulQA (MAD → TrueMAD: 0.93% → 2.20%).

Could the authors clarify under what conditions targeted feedback causes an initially correct agent to change to an incorrect answer? In particular, is this associated with incorrect peer advice, majority pressure, or the recipient's inability to evaluate the advice?

The new H∩S analysis provides useful evidence that correct answers can be preserved while other agents improve, but it does not directly establish which specific feedback messages cause successful corrections.

2. Practical inference efficiency.

The token-matched comparisons with Self-Consistency and Best-of-N are valuable and demonstrate that TrueMAD achieves better accuracy under comparable total-token budgets on AIME and GPQA-Diamond.

However, total token consumption does not fully capture end-to-end inference latency, particularly when pairwise feedback introduces dependencies between debate stages. Could the authors clarify the practical latency and computational overhead relative to the strongest baselines?

These remaining questions concern the boundaries of the proposed mechanism and its practical efficiency, rather than the overall validity of the additional experimental results.

Thank you again for the thorough response.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.