acceptodds
Under review as a conference paper at ICLR 2027

GCL-MAD: Graph-guided Conflict Localization for LLM Multi-Agent Debate

Abstract

Multi-agent debate (MAD) has emerged as a promising paradigm for improving the reasoning capabilities of large language models. However, final decisions in existing MAD may be influenced by arguments presented with greater fluency or confidence rather than the actual correctness of reasoning. Moreover, intermediate reasoning errors are often embedded within plausible complete reasoning trajectories, making it difficult to identify which specific steps are responsible for an incorrect conclusion. To address these limitations, we propose GCL-MAD, a structured multi-agent debate framework that shifts debate from exchanging complete reasoning trajectories to reasoning about verifiable epistemic components. Specifically, outputs from agents with complementary roles are organized into structured fields, such as an answer, assumptions, or skeptical points or boundary conditions, with each populated field represented as one epistemic component node. Based on these components,a graph-construction agent organizes potential relationships among these components into an Epistemic Conflict Graph, where conflict edges are associated with explicit rationales describing their underlying causes. Guided by this graph, GCL-MAD identifies conflict-related edges and their associated components, enabling targeted verification only within these regions rather than re-verifying complete reasoning trajectories. Experiments across six reasoning benchmarks and two backbone models show that GCL-MAD consistently improves reasoning performance and supports more reliable answer correction through localized verification.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Multi-Agent Debate, LLM agents, Multi-Agent Reasoning, Agent Collaboration, Graph-Structured Reasoning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_M7A9OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes GCL-MAD, a multi-agent reasoning framework that identifies conflicts between structured agent outputs and selectively verifies them before adjudication. Solver, Skeptic, and Edge-case agents generate typed components, which are connected through reviewed conflict relations. Evaluation covers six reasoning benchmarks and two backbone models, supplemented by ablations, answer-transition analyses, and controlled routing comparisons. The results suggest that selective verification can improve answer correction, although the evidence for a distinct graph contribution and broader generalization remains limited.

Critical strengths

The controlled routing comparison is the strongest part of the evaluation. Reusing initial agent outputs and matching verifier-call counts helps distinguish target selection from the benefit of additional inference. The answer-transition analysis also measures both successful corrections and harmful revisions, providing a more informative assessment of reliability than final accuracy alone. These experiments support selective conflict verification as a useful design choice.

Critical weaknesses

The evaluation is too narrow to substantiate the framework’s generality. Only two backbones are tested, leaving unclear whether conflict identification, confidence-based filtering, and conservative adjudication remain effective across models with different reasoning capabilities. Although six benchmarks are included, they concentrate on mathematical reasoning and knowledge or commonsense questions with short, objectively scored answers. They provide limited coverage of tasks requiring sustained reasoning, external evidence, or execution-based verification. Several datasets are also subsampled to 300 questions, and small gains lack paired uncertainty estimates. These limitations weaken the evidence that the observed benefits extend beyond the selected models and task formats.

The graph-specific contribution is insufficiently isolated. Nodes represent complete output fields, and the graph routes pairwise verification without propagation or path-based reasoning. The closest alternative, Flat Conflict Ranking, performs within 0.67 percentage points on MATH and 0.89 on GPQA. Without uncertainty estimates and separate controls for consolidation and structural filtering, the results do not establish a substantial advantage over structured conflict review. I therefore lean toward rejection despite the promising routing results.

Questions for authors

For clarification of generalization, what evidence indicates that the routing benefit is stable across both backbones, rather than specific to GPT-5-mini in the controlled comparison? Were model choices, benchmark subsets, and the confidence threshold fixed before examining evaluation outcomes?

For clarification of the central contribution, do the differences from Flat Conflict Ranking remain under paired uncertainty estimates, and which graph-specific operation accounts for them? Broader model and task coverage would be valuable future work, but the immediate assessment depends on the robustness and attribution of the gains already reported.

5 comments
AuthorsOpenReview ↗

Response part 1 [W1]

We thank the reviewer for the constructive feedback. We address the concerns regarding evaluation scope, dataset subsampling, and statistical reliability through additional experiments and clarifications, as detailed below.

  1. Extended Evaluation Across Backbones and Challenging Benchmarks.

    GCL-MAD is built on a novel idea: localizing epistemic conflicts in multi-agent debate and addressing them through targeted verification, with its effectiveness demonstrated on established reasoning benchmarks such as MATH. We further evaluate on two challenging benchmarks, HMMT (Feb. 2025, 2026) and AIME (2024 I/II, 2025, 2026), with a third backbone, Qwen-Flash. Across all six settings in Table R1, GCL-MAD outperforms all baselines and ablated variants by up to 12.70 percentage points, proving effective across different difficulties and models.

    Table R1. Main results and component ablation on challenging reasoning benchmarks (Accuracy %).

    DatasetBackboneMADMAD-ASDMADCGCL-MAD (Full)w/o Skepticw/o Edge-casew/o ECG
    HMMTGPT-5-mini25.4028.5725.4041.2739.6838.1038.10
    DeepSeek-V4-Flash80.9579.3773.0282.5476.1974.6073.02
    Qwen-Flash34.9233.3338.1044.4441.2739.6839.68
    AIMEGPT-5-mini53.3352.2243.3358.8955.5655.5657.78
    DeepSeek-V4-Flash77.7875.5675.5682.2262.2267.7858.89
    Qwen-Flash62.2267.7862.2270.0062.2264.4460.00
  2. Dataset Subsampling and Paired Uncertainty Estimates.

    Full-dataset evaluation is computationally expensive. For example, evaluating a single baseline on the complete MATH test set requires nearly 10 days in our setup. Given the multiple baselines, backbones, ablations, and random seeds, we therefore randomly sample 300 questions per run on certain datasets, using fixed seeds and identical question subsets across compared methods. Subset-based evaluation is also adopted in related work:

    [1] Wu et al., ACL 2026 Main, Beyond Examples: Towards Automated Thought-level In-Context Reasoning for Large Language Models.

    [2] Guo et al., ICLR 2025, Temporal Consistency for LLM Reasoning Process Error Identification

    [3] Choi et al., NeurIPS 2025, Debate or Vote: Which Yields Better Decisions in Multi-Agent Large Language Models?

    To quantify the reliability of the observed improvements, we additionally compute paired bootstrap 95% confidence intervals using matched question-level predictions. As shown in Table R2, all confidence intervals exclude zero, providing statistical evidence that GCL-MAD outperforms the strongest competing MAD baseline on the evaluated subsets.

    Table R2. Paired bootstrap confidence intervals for GCL-MAD versus the second-best MAD baseline.

    BackboneDatasetΔAcc. (pp)95% CI (pp)
    GPT-5-miniMATH+1.00[0.44, 1.67]
    GPT-5-miniGPQA+0.82[0.15, 1.71]
    DeepSeek-V4-FlashMATH+2.78[1.11, 4.67]
    DeepSeek-V4-FlashGPQA+0.82[0.37, 1.34]

    ΔAcc. denotes the accuracy difference between GCL-MAD and the second-best MAD baseline, in percentage points.

    The 95% CI is computed using paired bootstrap resampling of question-level predictions.

AuthorsOpenReview ↗

Response part 2 [W2&Q1&Q2]

We thank the reviewer for the constructive comments. We address the concerns regarding graph-specific contributions, generalization, and experimental reliability below.

  1. Graph-Specific Contributions (W2&Q2).

    As shown in Table R3, removing conflict consolidation reduces accuracy by 2.00 percentage points, primarily due to fewer successful corrections. Removing edge rationales decreases accuracy by 1.11 points, with fewer successful corrections and more erroneous revisions. Together with Table 3, these results demonstrate that conflict consolidation improves verification-target selection, while explicit relational information enhances verification reliability, providing more direct evidence for the effectiveness of ECG's structured routing mechanism.

    Table R3. Ablation of conflict consolidation and edge rationales on DeepSeek-V4-Flash (GPQA).

    MethodAcc. (%)W→C (%)C→W (%)
    Full ECG61.603.570.23
    w/o Conflict Consolidation (matched verifier calls)59.601.560.22
    w/o Edge Rationale (fixed verification targets)60.492.680.45
  2. Cross-Backbone Generalization (Q1&Q2). To examine whether ECG's benefits depend on the backbone model, we extend the controlled comparison with Flat Conflict Ranking to DeepSeek-V4-Flash.

    As shown in Table R4, ECG achieves larger improvements over Flat Ranking on DeepSeek-V4-Flash. The consistent improvements across both backbones support the effectiveness of ECG-guided verification routing.

    Table R4. Cross-backbone comparison of ECG Routing and Flat Conflict Ranking (%).

    BackboneMethodGPQA Acc.GPQA W→CGPQA C→WMATH Acc.MATH W→CMATH C→W
    GPT-5-miniFull ECG75.225.800.0088.671.330.67
    Flat Conflict Ranking74.335.130.2288.001.331.33
    DeepSeek-V4-FlashFull ECG61.603.570.2383.892.670.00
    Flat Conflict Ranking59.822.230.6781.550.330.00
  3. Paired Uncertainty Estimates (Q2).

    To assess the statistical reliability of ECG's improvements over Flat Conflict Ranking, we compute paired bootstrap 95% confidence intervals using matched question-level predictions.

    As shown in Table R5, ECG shows positive accuracy gains across both backbones and datasets. Three paired confidence intervals exclude zero, while the remaining interval reaches zero at the reported precision, providing additional statistical support for ECG-guided routing.

    Table R5. Paired bootstrap confidence intervals for ECG versus Flat Conflict Ranking.

    BackboneDatasetECG − Flat (Δ\DeltaAcc.)95% CI
    GPT-5-miniMATH+0.67[0.00, 1.67]
    GPT-5-miniGPQA+0.89[0.22, 1.79]
    DeepSeek-V4-FlashMATH+2.34[0.67, 4.00]
    DeepSeek-V4-FlashGPQA+1.78[0.67, 3.12]
  4. Experimental Settings (Q1&Q2).

  • We selected intermediate-capability backbone models to investigate the benefits of structured multi-agent debate.
  • Benchmark subsets were randomly sampled using fixed seeds shared across methods, without performance-based selection.
  • The confidence threshold of τ = 3 was chosen based on preliminary sensitivity analysis to filter weakly supported conflicts that could cause unnecessary answer revisions, and was kept fixed in the main comparisons.

We welcome your further feedback and would be happy to address any remaining questions.

As the rebuttal period is drawing to a close, we would appreciate your reviewing our latest response. If these address your concerns, we would be grateful if you could consider raising your score.

Reviewer_QnEaOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper focused on the communication and reflection mechanism of multi-agent systems. The authors pointed out that the widely used fully communication manners in the current multi-agent debate systems where each agent directly takes the full reasoning trajectories of other agents is too coarse, as the specific source of failure is hidden in a long trajectory and therefore not easy to be detected. To solve this, the authors proposed a more fine-grained mechanism, where the trajectory of different agents will be decomposed into small components. These components will be used as nodes to construct a graph, and only conflictive nodes will be kept for reflection.

Critical strengths
  1. The proposed method provides a way to dive deep into the communication of multi-agent systems, where we not only enable communication among agents, but also allow the location of specific errors and conflicts by trajectory decomposition.

  2. The proposed method does not rely on a naive majority voting manner to derive the final answer, but instead used an evidence-grounded way, which should be more reliable.

Critical weaknesses
  1. From the main results we did not see a significant improvement over the compared baseline methods, which kind of cannot fully demonstrate the effectiveness of the proposed method.

  2. Similarly, in the ablation study, some ablated version does not look to be much inferior to the original version, and sometimes it’s even better than the fully designed version. This raises concerns about whether all the components in the proposed method are necessary.

Questions for authors

The main question or concern is whether using a more challenging dataset will help us to see a) more accuracy gain over baselines; b) bigger difference between the full model and ablated ones.

1 comment
AuthorsOpenReview ↗

Response to Reviewer QnEa

We thank the reviewer for the suggestion. We extend our evaluation to a third backbone, Qwen-Flash, and two challenging reasoning benchmarks, HMMT (February 2025 and February 2026) and AIME (2024 I/II, 2025, and 2026). As shown in Table R1, GCL-MAD consistently outperforms all baselines and ablated variants across all six settings, supporting its effectiveness on challenging reasoning tasks.

Table R1. Main results and component ablation on challenging reasoning benchmarks (Accuracy %).

DatasetBackboneMADMAD-ASDMADCGCL-MAD (Full)w/o Skepticw/o Edge-casew/o ECG
HMMTGPT-5-mini25.4028.5725.4041.2739.6838.1038.10
DeepSeek-V4-Flash80.9579.3773.0282.5476.1974.6073.02
Qwen-Flash34.9233.3338.1044.4441.2739.6839.68
AIMEGPT-5-mini53.3352.2243.3358.8955.5655.5657.78
DeepSeek-V4-Flash77.7875.5675.5682.2262.2267.7858.89
Qwen-Flash62.2267.7862.2270.0062.2264.4460.00

We welcome your further feedback and would be happy to address any remaining questions.

Reviewer_1sm4OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes GCL-MAD, which decomposes multi-agent reasoning into structured epistemic components and constructs a conflict graph to selectively verify potentially inconsistent components. Experiments across six benchmarks and two backbone models demonstrate improved reasoning accuracy and answer correction reliability.

Critical strengths
  1. The motivation is clear, and conflict-localized verification offers an intuitive alternative to full-trajectory debate. The use of role-specific epistemic components and selective verification provides a coherent and interpretable framework.

  2. The experimental evaluation is relatively comprehensive, including strong baselines, ablations, matched-budget routing comparisons, and answer-transition analyses. The results demonstrate promising improvements in reasoning accuracy and correction reliability across multiple benchmarks.

Critical weaknesses

1. Limited graph-specific novelty.

The proposed graph primarily serves as an LLM-guided conflict filtering and verification-routing mechanism, without graph propagation or multi-hop reasoning. Although Table 3 provides matched-budget routing comparisons, ECG Routing improves over Flat Conflict Ranking by only 0.67 and 0.89 percentage points on MATH and GPQA, respectively. Moreover, the ablation without ECG removes both graph construction and conflict-guided verification. Thus, the contribution of the graph representation itself remains insufficiently isolated from LLM-based conflict identification and filtering.

2. Coarse-grained error localization.

Each node represents an entire structured field, potentially containing multiple reasoning steps. Thus, the method does not necessarily identify individual erroneous steps. Moreover, the reported answer-transition analyses evaluate correction outcomes rather than the accuracy of error localization itself, leaving the effectiveness of component-level localization insufficiently validated.

3. Uncertain robustness and efficiency.

The method relies on uncalibrated, LLM-assigned ordinal confidence scores and shows considerable sensitivity to the conflict threshold. For example, Table 5 shows that GPQA accuracy drops from 75.22% to 68.53% when the threshold changes from 3 to 1. In addition, the matched-budget comparisons control the number of verifier calls rather than total inference cost, while the overall framework does not consistently reduce token consumption. Some accuracy gains are also relatively modest.

Questions for authors

Q1. What benefits arise specifically from the graph representation, beyond structured conflict identification and filtering? Given the relatively small improvements over Flat Conflict Ranking in Table 3, could the authors further isolate and quantify the contribution of graph structure itself?

Q2. How accurately does the framework localize actual reasoning errors, rather than merely improving final-answer correctness? In particular, how robust is its performance when relevant conflicts are missed, agents share incorrect assumptions, or edge-confidence estimates are unreliable?

4 comments
AuthorsOpenReview ↗

Response part 1 [W1&Q1]

We thank the reviewer for this insightful comment. We clarify the following points:

  1. Core Contribution. GCL-MAD shifts multi-agent debate from trajectory-level interaction to structured epistemic conflict localization. ECG organizes epistemic components into typed, rationale-bearing relations to guide targeted verification and evidence-grounded correction. Its contribution lies in structured conflict localization and verification routing, without requiring graph propagation or multi-hop reasoning.

  2. Interpreting Flat Conflict Ranking. For a fair comparison, Table 3 assigns Flat Ranking the same per-example verification budget as ECG Ki=∣EiECG∣K_i=|E_i^{\mathrm{ECG}}|, isolating verification-target selection from the number of verifier calls. Otherwise, an appropriate KiK_i (i.e., the number of selected conflicts) would have to be specified separately for Flat Ranking. Moreover, Flat Ranking is a routing variant of GCL-MAD that retains its core methodological innovation, including epistemic decomposition, conflict localization, targeted verification, and evidence-grounded adjudication. It differs primarily in conflict selection. Their performance gap therefore reflects the incremental contribution of ECG routing, not of GCL-MAD overall. Flat Ranking’s competitiveness against the other routing baselines further supports the shared conflict-driven verification framework. To examine backbone dependence, we extend this comparison to DeepSeek-V4-Flash.

    As shown in Table R1, ECG achieves larger improvements over Flat Ranking on DeepSeek-V4-Flash than on GPT-5-mini, suggesting that its benefits vary across backbones. Its consistent advantage across both models and datasets further supports the effectiveness of ECG-guided verification routing.

    Table R1. Cross-backbone comparison of ECG Routing and Flat Conflict Ranking (%).

    BackboneMethodGPQA Acc.GPQA W→CGPQA C→WMATH Acc.MATH W→CMATH C→W
    GPT-5-miniFull ECG75.225.800.0088.671.330.67
    Flat Conflict Ranking74.335.130.2288.001.331.33
    DeepSeek-V4-FlashFull ECG61.603.570.2383.892.670.00
    Flat Conflict Ranking59.822.230.6781.550.330.00
  3. Graph-Guided Operations. Table 3 evaluates the combined contribution of structural filtering and conflict consolidation through Flat Ranking, which removes both operations. We further conduct controlled ablations of conflict consolidation and edge rationales to examine their individual effects.

    As shown in Table R2, removing conflict consolidation reduces accuracy by 2 percentage points, primarily through fewer successful corrections (W→C: 3.57% → 1.56%). Removing edge rationales decreases accuracy by 1.11 points, with fewer successful corrections and more erroneous revisions. These results indicate that both conflict consolidation and explicit relational information contribute to the effectiveness of ECG-guided verification.

    Table R2. Ablation of conflict consolidation and edge rationales on DeepSeek-V4-Flash (GPQA).

    MethodAcc. (%)W→C (%)C→W (%)
    Full ECG61.603.570.23
    w/o Conflict Consolidation (matched verifier calls)59.601.560.22
    w/o Edge Rationale (fixed verification targets)60.492.680.45
AuthorsOpenReview ↗

Response part 2 [W2&W3&Q2]

  1. Scope of Conflict Localization. GCL-MAD is fundamentally a structured multi-agent debate framework, where debate is conducted through targeted verification. As described in Section 2.1, ECG guides this verification-driven debate by identifying answer-relevant conflicts at the field level, without pinpointing individual erroneous reasoning steps.

  2. Shared Incorrect Assumptions. Shared incorrect assumptions are a common challenge in multi-agent debate, particularly when agents use the same backbone model. While this issue cannot be completely eliminated, GCL-MAD mitigates it through complementary reasoning roles, explicit conflict identification, targeted verification, and evidence-grounded adjudication, thereby improving correction reliability.

  3. Confidence Threshold. The confidence scores are ordinal signals for conflict filtering, not calibrated probabilities. As shown in Table 5, lowering the threshold increases successful corrections from 5.80% to 7.14%, but also raises erroneous revisions from 0.00% to 8.04%. This explains the accuracy decline and supports conservative conflict filtering.

  4. Robustness Analysis. We further conduct controlled experiments on DeepSeek-V4-Flash (GPQA) by comparing ECG with matched-budget random routing, randomly removing 30% of retained conflicts, and shuffling edge-confidence scores before filtering. All settings reuse the same role-agent outputs, with detailed experimental procedures provided in the revised manuscript.

    As shown in Table R3, GCL-MAD consistently achieves higher accuracy than these variants, demonstrating the importance of reliable conflict selection.

    Table R3. Automated evaluation of verification routing and robustness on DeepSeek-V4-Flash (GPQA).

    SettingFinal Acc. (%)W→C (%)C→W (%)
    Original GCL-MAD61.603.570.23
    Random routing with matched verifier calls59.822.010.45
    Randomly remove 30% of retained conflicts59.151.560.67
    Shuffle confidence scores before filtering59.822.230.67
  5. Computational Efficiency. Our focus is effective verification with controlled computational cost. Table 3 shows higher accuracy under matched verifier-call budgets, while Table 7 reports approximately 61% lower token consumption than three-round MAD on MATH. We acknowledge that efficiency benefits vary across datasets.

We welcome your further feedback and would be happy to address any remaining questions.

As the rebuttal period is drawing to a close, we would appreciate your reviewing our latest response. If these address your concerns, we would be grateful if you could consider raising your score.

Dear Reviewer 1sm4,

Another reviewer raised similar concerns about the graph-specific contribution, the comparison with Flat Conflict Ranking, and the isolation of ECG’s graph-guided operations. We have addressed these shared concerns through additional experiments and clarifications. We note that the other reviewer has since raised their score after reviewing our response. If our response has addressed your concerns as well, we would be grateful if you could consider raising your score. We would be happy to clarify any remaining questions.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Global Response

Dear PC, SAC, AC, and Reviewers,

We sincerely thank you for your constructive feedback and expert insights, which have substantially strengthened the manuscript.

We are encouraged that the reviewers recognized the novelty of our approach, its clear motivation, and the reliability of the proposed framework.

Based on your feedback, we have made the following updates to the paper:

  1. Expanded Evaluation. We add a third backbone, Qwen-Flash, and two challenging reasoning benchmarks, HMMT (February 2025 and 2026) and AIME (2024 I/II, 2025, and 2026). GCL-MAD outperforms all compared baselines and ablated variants across all six settings. (Appendix C.4)

  2. Paired Uncertainty Estimates. We provide paired bootstrap 95% confidence intervals for the main MAD comparisons and ECG versus Flat Conflict Ranking, offering additional statistical evidence for the observed improvements. (Appendix C.5)

  3. Additional ECG Ablations. We extend the matched-budget ECG versus Flat Ranking comparison to DeepSeek-V4-Flash and introduce controlled ablations of conflict consolidation and edge rationales, further clarifying their contributions to verification routing. (Appendix C.6)

  4. Robustness Analysis. We conduct additional experiments involving matched-budget random routing, missing conflicts, and shuffled confidence scores, demonstrating the importance of effective conflict selection and confidence-based filtering. (Appendix C.7)

Key clarifications

  • Interpreting Flat Conflict Ranking (Reviewers 1sm4 and M7A9). Flat Conflict Ranking is a routing variant of GCL-MAD that retains its core innovations. In other words, the small gap on GPT-5-mini captures only the marginal benefit of ECG routing and does not represent the overall contribution of GCL-MAD. Our additional experiments show larger ECG gains on DeepSeek-V4-Flash, while controlled ablations further demonstrate the contributions of conflict consolidation and edge rationales. (Section 3.5)

We provide further details in our point-by-point responses and have incorporated all additions into the revised manuscript.

We are pleased that our rebuttal has successfully addressed the main concerns of all reviewers. We would particularly like to highlight that Reviewers 1sm4 and M7A9 raised several closely related concerns, including the graph-specific contribution of ECG, the relatively modest improvements over Flat Conflict Ranking, and the need to better isolate the effectiveness of graph-guided operations. Notably, Reviewer M7A9 explicitly acknowledged that their concerns had been addressed and raised their score from 2 to 3.

Thank you again for your time and dedication in reviewing our submission.

Sincerely,

The Authors of Submission #30663

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.