ARGORA: Orchestrated Argumentation with Counterfactual Diagnostics for Multi-LLM Consensus

Existing multi-expert LLM systems gather diverse perspectives but resolve them through simple aggregation or voting, obscuring which arguments drove the final decision. We introduce $\text{ARGORA}$, a framework that compiles multi-expert discussions into explicit graphs of supporting and attacking arguments. By formalizing the deterministic aggregation procedure over these graphs as a structural model that admits edge-local interventions, ARGORA can systematically remove individual arguments and recompute outcomes, identifying which argument chains were decisive and whether decisions would change under targeted modifications. We further introduce a correction mechanism that aligns the framework's internal consensus with external judgments when they disagree. Across diverse benchmarks and an open-ended use case, ARGORA achieves competitive accuracy and demonstrates corrective behavior: when experts initially disagree, the framework resolves arguments toward correct answers more often than it introduces new errors. Our implementation is publicly available at https://anonymous.4open.science/r/ARGORA-iclr.