acceptodds
Under review as a conference paper at ICLR 2027

Interaction Destroys the Diagnosticity of Agreement Where Truth is Scarce

Abstract

While multi-agent consensus is frequently used as an epistemic proxy for reliability, agreement justifies reliance only when derived from independent evidence. We find that peer interaction compromises this decision-relevant independence, triggering cascades that decouple accuracy from diagnosticity. When three heterogeneous agents deliberate on questions where their independent baseline answers disagree, the resulting two-of-three majorities are false 74.8% (86/115) of the time, versus 23.1% (31/134) for independent votes (Fisher ). A registered blind-revision control separates visibility from revision: private revision keeps conditional falsity inside its registered bound, whereas visibility manufactures 14.2 percentage points [8.3-20.7] more false agreed answers per question on MMLU-hard (McNemar ; replicated at points), without a detected worsening of the majorities that form. We formalize this as the Diagnosticity-of-Agreement Law: agreement signals truth only to the extent agents could have agreed without interacting. Replications indicate that this collapse, which arrives predominantly at the first exchange (62.9% of the increase), tracks a panel's operative ability to verify. A weak panel collapses on GSM8K (80.0% vs. 21.2%) and MMLU (84.3% vs. 57.4%); a mid-capability panel resists on GSM8K ( point) but collapses on MMLU-hard (44.8% vs. 28.9%). Crucially, although deliberation is at least as accurate as self-consistency at a matched number of completions (34.7% vs. 31.0%), it severely degrades diagnosticity (79.6% vs. 46.2% conditional false agreement), indicating that rising accuracy does not by itself warrant reliance on agreement. Finally, a predicted Coverage Law () bounds mitigation on panels that appear able to verify retrieved evidence; on a weaker panel it failed, and checkable evidence restored the bound (4.0% and 2.7% residual).

open until 14 Dec 2026

est. 32% chance this paper gets accepted at ICLR 2027.

Reject 68%Accept 32%

What do you think this paper will get?

All positions stay anonymous.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.