EviCon: How Evidence Quality Shapes Beneficial and Harmful Conformity in LLM-Based Multi-Agent Systems

Large language model (LLM) based multi-agent systems increasingly rely on peer interaction to improve collective decision making. However, conformity during such interaction can have opposing consequences. Peer influence may correct erroneous decisions, but it may also steer correct judgments toward incorrect or biased conclusions. Existing conformity evaluations mainly rely on answer-only protocols or unconstrained debate, leaving the role of evidence quality insufficiently understood. To address this gap, we propose EviCon, an evaluation framework for systematically studying how evidence quality shapes beneficial and harmful conformity. EviCon compares three peer-message conditions, namely Answer-Only, Grounded Evidence, and Erroneous Evidence, across five heterogeneous LLMs, two pressure directions, and two contextual ambiguity levels. Our experiments show that grounded evidence generally strengthens peer assisted correction, whereas erroneous evidence can amplify harmful conformity toward biased peer decisions. Further analysis reveals that these effects depend not only on whether models recognize and map relevant evidence, but also on how they reconcile contextual evidence with peer opinions. Based on these findings, we develop ARBITER, a structured reasoning strategy that separately evaluates peer answers and supporting evidence. Across five LLMs, ARBITER better preserves peer assisted correction while strengthening resistance to erroneous peer influence.