acceptodds
Under review as a conference paper at ICLR 2027

LLM Cognitive Bias Exploitation and Emergent Consensus Capture in Connected Agents

Abstract

LLM agents increasingly deliberate in connected panels, where every agent sees the verdicts of the others. This turns a cognitive bias of the models themselves, their tendency to defer to peers, into an attack surface: an adversary who can post one verdict may pull the honest agents with it. We ask whether that failure can be predicted before deployment from a measurement on a single agent. It can, in closed form. If an agent abandons its own evidence once a unanimous peer margin reaches , then public deliberation acquires regularities that belong to no individual agent, including an error floor of , where is the accuracy of an agent's own evidence and , that does not fall as the panel grows and capture of a panel of any size by early adversaries. Both and are measurable, so the prediction has no fitted parameters. We test it over 4,770 panel episodes on OpenAI and Anthropic models. Measured with the classic Asch item, where an agent commits and is then contradicted, the prediction fails badly (calibration slope 0.300, where 1 is perfect, against a pre-specified ). Measured the way deliberation actually presents peers, before the agent commits, the same prediction is calibrated (slope 0.827, ). Conformity recovered from 4,980 decisions inside the panels shows why: real agents behave as the in-context measurement says, not as the Asch one does, and they defer gradually rather than at a threshold. Biases also compound, but selectively. An adversary that adds one unverifiable claim of superior evidence to its verdict doubles the damage and captures panels of the most deferential configuration 93% to 97% of the time, whereas claims of rank or of certainty add almost nothing: these agents are moved by what resembles evidence and not by status. The same asymmetry supplies defenses. Dissent that merely disagrees is largely ignored, while the identical dissent carrying an evidence claim halves mean error, and cuts it from 0.933 to 0.233 for GPT-4.1-mini; sealed voting removes the mechanism by construction; and panels of reasoning models largely resist the attack. Code, items, and all 61,706 cached model responses are in the supplementary material.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
LLM agents, agentic AI, cognitive bias, psychometrics, AI safety and security, role drift, multi-agent systems, conformity, agent evaluation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_DuJ7OpenReview ↗
Rating
4
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This work investigates the vulnerability of teams of LLM agents to conformity bias and proposes a closed-form method of approximating the error-floor for a particular agent team. To accomplish this, the authors set up an experiment in which each LLM agent has access to private information which it must use to provide a verdict. Importantly, one (or more) of the agents on the team is an attacker who intentionally gives the incorrect verdict in an attempt to bias the other agents. This experimental paradigm is used to test the predictions of the threshold model, with the key discovery being that the model is more accurate at predicting capture when conformity is measured in-context rather than post-hoc, and this accuracy can be further improved by simulating panel decisions using the full experimentally-derived conformity curve. Lastly, the authors also extend their study to investigate different forms of attack (e.g., asserting access to privileged evidence), as well as potential defenses, and the generalizability of their approach to other agents/LLM providers.

Critical strengths
  1. The authors do a good job addressing a timely and important topic: Agent teams have become increasingly used for approaching complex problems, and the tendency of LLM agents towards agreeableness is a well documented weakness. The present work therefore not only addresses an important and timely topic, but does so through an experimental paradigm which effectively isolates the bias in question and quantifies its possible detriment to agent team performance. This could be useful for future research focused on developing team's of agents which are robust to attack.

  2. The proposed model is compelling and applicable: The main contribution of the paper is an elegant model for estimating the error rate for a team of agents whose conformity bias is under attack. The model depends only on estimating the behavioral parameters of a single agent: namely that agent's accuracy pp and their conformity threshold kk (or for even more improved accuracy, by using the measured conformity curve). Importantly, the model does not require any fitting or fine-tuning to predict team behavior. The flexibility of this approach could prove useful for providing a preliminary evaluation on the robustness of LLM teams to this form of attack without expensive experiments involving teams of agents.

Critical weaknesses
  1. Presentation of results impedes readability: In various parts of the main text, the results are presented either (a) in prose, or (b) with references to the Appendix. In my opinion, this significantly hinders the readability of the manuscript and at times makes it too difficult to appreciate the key outcomes. For example, L305 states: "pooled over configurations it rises through 0.095, 0.164, 0.380, 0.479, 0.645, 0.796 as the opposing δ\delta goes from 0 to 5." This is just one example. It is my strong belief that it would be more appropriate to either include the figure in the main manuscript or to state the main takeaway more plainly. Similarly, Figure 2 appears to be the main results-focused figure in the main text, but its small text makes it very difficult to read.
Questions for authors
  1. In the setting with one attacker and one defender, each claiming access to privileged information, did you investigate whether the ordering has any bearing on the outcome of the attack? In other words, is there a difference between the attacker going first and defender second and the defender going first and attacker second? If there is a discovered sensitivity to ordering, this could possibly be interesting to report.
2 comments
AuthorsOpenReview ↗

Response to DuJ7: §5 restructured and Fig. 2 re-rendered; what the cascade model and the existing data say about attacker/defender order

Thank you for the positive assessment and for the concrete presentation suggestions. We agree with both.

W1: results in prose and in the appendix. We will restructure §5 so that each finding can be read without the appendix:

  • a short results-at-a-glance table at the start of §5 (finding, key number, where it is shown);
  • a one-sentence takeaway in plain words at the start of each subsection;
  • the sentence you quote (L304–306) becomes: "In real panels conformity rises smoothly with the opposing margin, from about 0.1 to 0.8, with no jump larger than 0.22; a threshold model would need a jump near 1." The six values move into a figure (current Fig. 5a, promoted to §5.5);
  • the calibration scatter (current Fig. 4c) moves into §5.4 next to Table 1;
  • §5.2, §5.7 and §5.9, which carry many inline numbers, are condensed in the same way.

W1: Figure 2. The figure was drawn on a 13.6 × 3.2 in canvas and scaled to text width, which shrinks the fonts to about 40%. We re-render it at print size as a 2 × 2 grid with text of at least 8 pt and larger markers.

Q1: does attacker/defender order matter? We did not test this. In the defender study (Table 22, L394–398) the attacker always speaks first and the defender second. There, a plain defender lowers mean panel error only from 0.600 to 0.542, and the same defender carrying an evidence claim lowers it to 0.308. Two related results are already in the paper:

Existing evidence on orderPanel error
Lone bare adversary speaking 1st / 3rd / 5th (Table 14, L357–360)0.217 / 0.156 / 0.167 (p = 0.22, 0.31)
Evidence-claim attack: no robust agents / two robust agents right after the attacker / two robust agents last (Table 17, L422–425)0.933 / 0.500 / 0.833

The cascade model of §3 makes only a weak prediction here. Once both have spoken, the visible margin is 0 whatever the order, so it predicts no order effect for the agents who follow. But it ignores the content of claims and how an LLM weighs earlier against later messages in its context (primacy vs recency). A swap (defender first, attacker second, with and without the defender's evidence claim, on the four Table 22 configurations) would isolate exactly that. We have not run it. We will state the prediction in §5.8 and list the swap as a limitation; if we can complete it before the discussion closes, we will post it here.

Thank you again. We hope these changes make the results much easier to follow, and we would be glad to hear any further suggestions.

AuthorsOpenReview ↗

Follow-up to DuJ7 Q1: we ran the attacker/defender order swap; order matters, and the margin-only model misses it

Thank you again for Q1. We said we would post the order swap if we could complete it before the discussion closes; it is now done.

Setup. This is the defender study of Table 22 with one change: the defender speaks first and the attacker second. Everything else is the same: four configurations, n = 9, the same 30 batches, and an attacker who posts the wrong verdict together with the evidence claim. We re-ran the attacker-first conditions through our response cache, and they reproduce Table 22 exactly.

Result: the first speaker has a large advantage. Panel error:

ConfigurationNo defenderPlain defender, attacker firstPlain defender, defender firstEvidence defender, attacker firstEvidence defender, defender first
GPT-4.1-mini, neutral0.9330.8330.4330.2330.000
GPT-4.1-mini, deferential0.9670.9000.5330.6670.133
GPT-5-mini, neutral0.2330.2000.1670.2330.133
GPT-5-mini, deferential0.2670.2330.2670.1000.133
Pooled (120 panels)0.6000.5420.3500.3080.100

We compared the two orders on the same 120 panels, using an exact McNemar test on the panels where the order changed the outcome:

  • plain defender: defender-first was right where attacker-first was wrong in 26 panels, and the reverse in 3 (p = 1.5 × 10⁻⁵);
  • defender with the evidence claim: 27 panels vs 2 (p = 1.6 × 10⁻⁶).

The effect comes from GPT-4.1-mini. For GPT-5-mini, order changes panel error by at most 0.10.

The margin-only cascade model does not predict this. In either order, the first honest agent sees one attacker and one defender, a visible margin of 0, so the model predicts the same behaviour. It does not behave the same:

  • with a plain defender, it votes wrong 0.61 of the time when the attacker spoke first and 0.39 when the defender did;
  • with the evidence claim, 0.38 and 0.18;
  • among agents whose own sample pointed to the right answer (96 of 120), 0.53 and 0.26 with a plain defender.

So the first post carries more weight than the margin alone implies: a primacy effect.

What we will change. In §5.8 we will:

  • report the swap and state that the cascade model ignores order effects beyond the visible margin;
  • list this as a limitation of the margin-only model;
  • note the practical consequence: a cheap defense is to seat a trusted agent first.

Thank you for the question. It exposed an effect our model misses.

Reviewer_m3GYOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies how vulnerable multi-agent systems are when an attacker introduces an incorrect answer before other agents respond. The authors model this behavior using the gambler's ruin framework and predict team-level error rates based on individual agent accuracy and susceptibility to peer influence. They also find that agents are particularly vulnerable to unverifiable claims about having access to additional evidence, while claims based on authority or confidence are less effective.

Critical strengths
  • Important problem: As multi-agent systems become increasingly common, understanding how malicious agents can influence collective decisions is an important security problem. I think the paper addresses a relevant and timely research question.

  • Interesting insights: The observation that agents are more easily influenced by claims of additional evidence than by authority or confidence is interesting. The paper also provides some useful insights into how information sharing and agent independence affect collective decisions.

Critical weaknesses
  • Limited generalizability of the experimental setting: My main concern is that the experimental setup feels somewhat artificial. Agents are deliberately given limited information and must make decisions based on incomplete evidence. It is unclear why this particular setting is representative of real-world multi-agent systems, especially when agents could potentially access the same information. Moreover, this setup may naturally favor attacks that claim access to additional evidence. I would like to see whether the findings hold under more realistic settings where agents have access to complete information.

  • Limited novelty in the prediction method: The main improvement seems to come from matching the measurement procedure to the actual interaction setting. If agents observe others' answers before making their own decisions, it is not particularly surprising that measuring their behavior under the same conditions leads to better predictions. I am not convinced that this demonstrates a meaningful advantage of the proposed framework over existing approaches. It would also be useful to evaluate whether the alternative measurement procedure performs better in later rounds, where agents have already formed their own opinions.

  • Insufficient justification for the threshold: The choice of a 50% threshold for defining the susceptibility parameter seems arbitrary. I could not find a convincing justification for this choice, and it is unclear how sensitive the theoretical conclusions are to different thresholds. Since the final predictions rely on the full susceptibility curve, the practical significance of this threshold-based characterization is also unclear.

  • Limited evidence from frontier models: Although the paper includes some experiments with stronger models, most of the main results rely on relatively small models. Interestingly, the attacks appear much less effective against stronger models such as GPT-5 and o4-mini. This raises questions about how practically relevant the identified vulnerabilities are for more capable multi-agent systems. More extensive experiments with frontier models would strengthen the paper's claims.

Questions for authors
  1. Why do the experiments assume that agents have access to only a limited portion of the available evidence? Would the proposed attacks remain effective if all agents had access to the same complete information?

  2. The improvement in prediction accuracy seems to come primarily from matching the measurement procedure to the actual multi-agent interaction. Could the authors clarify what the proposed framework contributes beyond this alignment? Also, have the authors evaluated whether the alternative measurement procedure performs better in later discussion rounds, where agents have already formed their own answers?

4 comments
AuthorsOpenReview ↗

Response to m3GY (1/2): why partial evidence — conformity sits in agents with ambiguous evidence, and we scope the claims accordingly; the 2×2 ablation; later rounds

Thank you for your review and for the specific questions, which point to the main limits of our setting. Here we address the setting (W1/Q1), and novelty and later rounds (W2/Q2); Part 2 covers the threshold (W3) and frontier models (W4).

W1/Q1: why partial evidence? We will state three reasons in §2:

  1. Panels add most when members hold different information (shards, retrievals, tool outputs or samples). With shared complete inputs, errors are more correlated, though debate can still help there (Du et al., 2024).
  2. A normative answer. The hypergeometric design gives a closed-form Bayes ceiling (p* = 0.8485, Eq. 1, L145–157), so conformity is measured as departure from the agent's own evidence (Definition 1).
  3. Fixed ground truth, with items resampled, so none can appear in a training corpus (L148–149).

We agree the setting fits distributed-evidence panels better than shared-input panels such as LLM judges, and we will narrow the motivation (L043–044). We also agree that the "reviewed all 12 transactions" claim is credible partly because evidence is partial; the rank and certainty cues, which do not compound (Table 19), were tested in the same setting. A new check on cached in-panel decisions facing an opposing majority supports your concern:

Agent's own sample (lines over the limit, of 5)Conformityn
Decisive (0 or 5)0/63 (95% upper bound ≈ 0.06)63
Strong (1 or 4)0.10475
Ambiguous (2 or 3)0.51963

Decisive samples are our closest analogue to complete information, so we expect the attacks to weaken sharply when evidence is complete and verifiable. We have not run a complete-information condition; we will state that our results hold for partial evidence. If we can complete one before the discussion closes, we will post it here.

W2/Q2: is the gain just matching measurement to interaction? Matching the format is the main step. Using the full curve also matters, though the evidence for that is weaker than our text implied. Calibration slopes [95% CI] from the 2×2 ablation (Table 1; band [0.7, 1.3]; row B corrected, see our correction comment):

Threshold kFull curve in the cascade
Post-hoc probeA: 0.300 [0.22, 0.38]C: 0.352 [0.27, 0.44]
In-context probeB: 1.341 [1.05, 1.63]D: 0.827 [0.75, 0.90]

Only row D lies wholly inside the band. Row B's interval overlaps it, but B fits worse (R² 0.777, MAE 0.131 vs 0.949, 0.054) and its slope moves from 0.46 to 1.46 with the threshold level (Part 2). We will rewrite L268–269 in these terms.

Which format matters was not settled in advance: in people, committing first reduces conformity (Deutsch & Gerard, 1955; L700–701), while sycophancy results suggest the reverse; here it rose from 0.21 to 0.91 (Table 7). The probes agree at one peer for 5 of 6 OpenAI models (Table 5) but diverge at 2–3 peers for 3 of them (Table 6), so a deployer must measure. Beyond the match, the framework gives:

  • a closed-form error floor and adversary count for capture (Prop. 1);
  • out-of-sample predictions: adversary doses (Table 15, slope 0.697 [0.55, 0.84], just below the band, vs 0.286), panels up to 21 (MAE 0.069 vs 0.165), where capture appears (Table 20, Spearman −0.880);
  • a mechanism check on 4,980 decisions (Table 12);
  • order-dependent defenses (Table 17).

Related models of biased consensus fit influence to the debate trajectories they describe (L452–454); ours is measured beforehand.

Later rounds. Our 3-round debate (Table 21) re-asks agents without their own verdict, so it does not test committed agents. It does show that re-asking follows the in-context curve: facing a 3–1 opposing majority, agents abandon their sealed verdict 0.46 [0.39, 0.54] of the time (n = 164); the in-context curve predicts 0.38 (net margin) or 0.49 (opposing count), the post-hoc curve 0.78. A committed debate is untested and we state it as a limitation.

AuthorsOpenReview ↗

Response to m3GY (2/2): the ½ threshold is the closest step approximation (sensitivity table); frontier models — no false alarms, and what that does not show

W3: the 50% threshold. We agree this needed justification.

  1. Why ½. For a non-decreasing measured curve φ, the deterministic rule 1{δ ≥ k} that disagrees least with the measured behaviour, minimising Σ_δ w_δ |φ(δ) − 1{δ ≥ k}| for any positive weights w_δ, switches exactly where φ first exceeds ½. Each margin contributes φ(δ) if the rule says "do not conform" and 1 − φ(δ) otherwise, so each term is minimised separately at that switch. So k in Definition 1 is the best threshold approximation of the curve, not a free parameter. We will add this after Definition 1.
  2. What depends on it. Only the deterministic rows of Table 1 (A and B) and the interpretation of Proposition 1 (error floor, adversaries needed). The calibrated predictor (row D) uses the full curve and has no threshold. We already recommend reporting curves (L238, L470–471, App. F) and keep k because it makes Proposition 1 closed-form.

Sensitivity (new). Rows A and B recomputed from the cached curves, with k_τ the smallest n_p at which φ > τ. Each cell gives slope [95% CI] and R².

τRow A (post-hoc)Row B (in-context)
0.30.301 [0.22, 0.38], 0.7020.457 [0.34, 0.58], 0.704
0.40.301 [0.22, 0.38], 0.7021.425 [1.24, 1.61], 0.902
0.50.300 [0.22, 0.38], 0.7021.341 [1.05, 1.63], 0.777
0.60.300 [0.22, 0.38], 0.7021.461 [1.11, 1.81], 0.737
0.70.300 [0.22, 0.38], 0.7021.461 [1.11, 1.81], 0.737

MAE is 0.192–0.193 for row A and 0.119–0.143 for row B, against 0.054 for row D.

  • Row A does not move. Both GPT-4.1-mini configurations have φ_post(1) ≥ 0.82, so k_post = 1 at every τ. The post-hoc failure does not depend on the choice of ½.
  • Row B is unstable. Its slope swings from 0.46 to 1.46, and its point estimate is inside [0.7, 1.3] at no τ, although its interval reaches into the band for τ ≥ 0.4. That instability is why we recommend the curve, which row D uses without any threshold.

W4: frontier models. We agree this matters. A pre-deployment predictor should not raise false alarms for robust models. o4-mini and GPT-5 have low in-context conformity (0.00 and 0.15 to one peer, L400–401), and their panels resist the evidence-claim attack (0.067 and 0.200, against 0.933 for GPT-4.1-mini; Table 23). From their measured curves, with nothing fitted:

n = 9 panel error (Table 23)PredictedObserved (30 panels)
GPT-5, no adversary0.0540.100
GPT-5, bare adversary0.1680.200
o4-mini, no adversary0.0040.000
o4-mini, bare adversary0.0150.033

All four predictions lie inside the exact 95% intervals, as would those of a no-conformity baseline. These cells therefore show only that the predictor raises no false alarm for robust models; they do not test the format comparison or discriminate between predictors. Two qualifications on practical relevance:

  • Capability does not remove the bias. Claude Sonnet 5 conforms in context to three peers at 0.51 (Table 8), about the same as GPT-4.1-mini at the neutral stance (0.50; 0.72 deferential, Table 7). A bare adversary raises GPT-5 panel error from 3 to 6 of 30 panels, a difference this sample cannot resolve.
  • Smaller models are a practical choice. Panels multiply inference cost by panel size, so smaller models are often used; the predictor tells a deployer which are safe.

We have not run panels for other providers' frontier models and state this as a limitation. The reasoning endpoints were run at minimal reasoning effort (L999–1001).

Thank you again for raising these points. They have helped us state the scope of our claims more precisely, and we would welcome any further questions.

AuthorsOpenReview ↗

Follow-up to m3GY Q1/Q2: complete-information panels and a debate with committed agents

Thank you again for Q1 and Q2. We said we would post these experiments if we could complete them; both are now done. They use the paper's code, prompts and the four configurations of Tables 21–22. Conditions that already appear in the paper were re-run through our cache and reproduce the published numbers exactly.

Q1: complete information. We used the same 30 batches (n = 9). Each honest agent now sees all 12 transactions, so every member has the same input, and the prompt says the other auditors were given the same batch. Policy, stance and answer format are unchanged. Pooled panel error over the four configurations (120 panels per cell):

No attack1 bare attacker1 attacker + evidence claim4 bare attackers
Partial evidence (paper)0.1000.2500.6000.842
Complete information0.3000.2670.3170.442

Three findings:

  1. Where the agents read the batch correctly, every attack failed. On the batches with 8 of 12 lines over the limit, the models made no errors in any condition: 0 of 60 panels were wrong, even with four attackers, and no first honest agent voted wrong. On those same batches, the evidence-claim and four-attacker conditions made 50% and 70% of panels wrong under partial evidence.
  2. Where they misread it, the attacks still work. On batches with 4 of 12 lines over the limit, the models answer in one word and wrongly flag the batch 43% of the time with no attacker present. The attacks push toward that same error. They raise the first honest agent's error to 0.48 (bare), 0.62 (evidence claim) and 0.70 (four attackers). This matches the pattern in our first reply: conformity concentrates where an agent's own reading of the evidence is unreliable.
  3. A shared input removes the panel's advantage and most of the evidence claim's force. With no attack, the first agent's error is about the same in both settings (0.21 partial, 0.22 complete). The panel lowers it to 0.10 under partial evidence but raises it to 0.30 under complete information, because members share one input and therefore the same mistakes, and the visible sequence passes them on. The evidence claim adds 0.35 to panel error over a bare attacker under partial evidence, but only 0.05 under complete information.

So, to your question: the attacks did not work when the agents read the complete information correctly. They kept working where the model's own judgment was unreliable, and complete information did not remove that. We will add this as a new subsection, present partial evidence as the setting where the claim-based attacks are strongest, and scope the claims accordingly.

Q2: later rounds, with agents that have formed their own answers. We used the same debate as Table 21 (n = 5, 3 rounds, one bare adversary, 30 panels per configuration), with the same sealed round 1. In rounds 2–3, each honest agent now keeps its own conversation (its evidence, its earlier verdicts, then the other four verdicts), so it must revise an answer it has committed to. The table covers agents facing a 3–1 opposing majority in rounds 2–3:

Conformity [95% CI]nPeer-first curve predictsAnswer-revision curve predicts
Re-asked without own verdict (paper)0.46 [0.39, 0.54]1640.38–0.490.78
Committed (new)0.70 [0.62, 0.77]1510.38–0.480.75

Over all opposed decisions (n = 174–178), the Brier score favours the answer-revision curve for committed agents (0.12 vs 0.19–0.26) and the peer-first curve for re-asked agents (0.22–0.23 vs 0.35). The separation is sharpest for GPT-4.1-mini: committed agents conform 1.00 (neutral) and 0.97 (deferential), against 1.00 from the answer-revision curve and 0.43–0.73 from the peer-first curve. For GPT-5-mini the two curves lie closer together and the data separate them less clearly (neutral: 0.24 observed, 0.23–0.28 peer-first, 0.31 answer-revision; deferential: 0.62, 0.38–0.47, 0.78).

This answers your second question directly. In later rounds where agents hold their own answers, the alternative (answer-revision) measurement does predict better. The framework's point is therefore not that one probe is right, but that the probe must match how the deliberation presents peers. The same agents follow one probe or the other depending on whether they hold a committed answer. We will add the committed debate to §5.6 and replace the corresponding limitation.

Thank you for pressing on both points; the paper is clearer for it.

Thank you for the rebuttal and for transparently sharing the new results, even when they show the limits of your setup. That kind of honesty is really helpful for the research community. However, these results confirm my main concerns. Under complete information, attacks fail on clear batches (0% error) and the extra impact of the evidence-claim attack almost disappears (+0.35 to +0.05), which shows the high capture rates rely heavily on hiding information from agents. Also, the committed-debate experiment shows the answer-revision probe works better when agents keep their prior answers. Along with dropping the Asch benchmark claim, the main takeaway comes down to matching the test format to the interaction format, which feels expected. While I appreciate your open and thorough response, I will keep my current score due to the narrowed scope and limited novelty.

Reviewer_yufxOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies whether conformity measured on a single LLM can predict errors in connected panels of agents. The authors use a sequential cascade model to relate private accuracy and deference to previous votes to collective error and adversarial capture. They compare a post-hoc probe, in which an agent revises its answer after seeing conflicting peers, with an in-context probe, in which peer verdicts are shown before its first response. The post-hoc measurement overpredicts panel error, while the in-context conformity curve gives closer predictions in the main OpenAI experiments. Additional experiments examine adversarial claims, speaking order, panel size and composition, defenses, and a smaller Claude replication.

Critical strengths
  1. The paper makes a useful connection between single-agent measurement and collective outcomes. The conformity curves are measured before the panel experiments, rather than being fitted to panel behavior. The 2×2 comparison of probe format and cascade model also helps distinguish measurement mismatch from the limitations of a fixed conformity threshold.
  2. The experimental evaluation includes relevant controls beyond the main attack setting. Sealed voting, different attacker counts and positions, panel size, and model composition help clarify when errors result from peer influence rather than majority voting alone. The reported release of prompts, generated items, and cached responses also supports reproducibility.
Critical weaknesses
  1. The novelty claim relies in part on treating the post-hoc probe as the classic or standard Asch-style conformity benchmark. This characterization is questionable. In the original Asch procedure, participants heard the confederates before responding, and Weng et al. (ICLR 2025), which the paper cites, also presented peer answers before the subject responded. Moreover, Section 5.4 shows that the large format gap arises mainly when a model's previous answer is challenged. The experiments therefore establish a difference between answer revision and peer-informed initial decisions, but do not establish that standard peer-first conformity benchmarks are invalid.
  2. The proposed pre-deployment predictor is evaluated mainly within a closely matched setting. The 26 main panel conditions come from only two base OpenAI models, each with three system-prompt stances. The smaller Claude study is less conclusive: both predictors underestimate panel error, while in-panel conformity falls between the two measurements. The stochastic model also applies conformity measured with unanimous opposing peers to mixed-vote panels using only the net opposing margin. It is unclear whether the same margin produces the same response when the number and composition of earlier votes differ.
Questions for authors
  1. Which existing LLM conformity benchmarks use the post-hoc, prior-commitment protocol considered here? How does the benchmark-failure claim account for peer-before-answer protocols such as Weng et al. (ICLR 2025)?
  2. For the same opposing margin, do agents behave similarly when earlier votes differ in number or composition (e.g., 1–0 versus 3–2)? Does the strength of private evidence change this relationship?
  3. How broadly can the pre-deployment prediction be interpreted given the small number of independent base models and the less conclusive Claude panel results?
3 comments
AuthorsOpenReview ↗

Response to yufx (1/2): you are right about Asch — the two probes are renamed and the claim narrowed; where commit-then-challenge is used

Thank you for a precise and fair review. We agree with W1 and will change the framing. We explain how below and then answer Q1; Part 2 covers generality and mixed votes.

W1: the post-hoc probe is not "the classic Asch" benchmark. You are right on both counts. Asch's participants heard the confederates before answering (our own App. B.1 says so, L687–688), and BenchForm (Weng et al., 2025) presents peer answers before the subject responds. It was inaccurate to describe the post-hoc format as "the classic Asch design" (L023, L070, L100–101, L162–163, L918) and as "the standard benchmark" (L100, L123–124), and to say it has "the longer pedigree" (L169–170). We also read §5.4 the way you do: the gap comes from contradicting the agent's own previous turn (Table 24, L284–292).

What the experiments establish is therefore what you describe. Answer revision and a peer-informed initial decision are different quantities, and they can differ by a factor of four (GPT-4.1-mini: 0.91 vs 0.21 to one peer, Table 7). The final version changes three things:

  • Names. "Post-hoc" becomes the answer-revision (commit-then-challenge) probe and "in-context" becomes the peer-first probe. A closer human analogue of the former is the commitment conditions of Deutsch & Gerard (1955), in which participants recorded their judgment before hearing the group.
  • Claim. Contribution 2 becomes: an answer-revision score (the protocol of sycophancy challenge evaluations and of debate revision rounds) mispredicts sequential peer-first panels about threefold (Table 1, rows A vs D), and the error traces to contradicting the agent's own prior answer (Table 24). We remove "benchmark failure" and add: we do not claim that peer-first benchmarks such as BenchForm are invalid; our results support them. A peer-first score fed through the cascade of Proposition 1 predicts panels with nothing fitted (row D, R² = 0.949) and recovers in-panel conformity (MAD 0.031 vs 0.244, Table 12).
  • Edits. Abstract, Fig. 1, L100–102, L123–126, Def. 2, L169–170, §8, App. E, the Table 5 caption, and Related Work, which will credit BenchForm as the peer-first precedent our in-context probe follows.

The claim is narrower, and we think it remains useful: the practical question is which single-agent score to use for a given deliberation protocol, and how to turn it into panel risk. The matching test in the other direction would be a debate in which each agent's own previous verdict is in context, the revision setting where the answer-revision probe should be the right measurement. Our existing 3-round debate (Table 21) re-asks agents without showing their own earlier verdict, so it does not test this. We will state that explicitly and list the committed debate as a limitation.

Q1: which benchmarks use the post-hoc protocol? For conformity to peers, the peer-first protocol of Weng et al. is, to our knowledge, the usual design. The commit-then-challenge structure appears where answer reversal is studied: sycophancy evaluations that challenge a model's answer (Sharma et al., 2024), and multi-agent debate, in which agents answer independently and then revise after seeing their peers (Du et al., 2024). Our claim is no longer about benchmarks in general. It is that revision-style scores should not be used to screen models for sequential panels, whereas peer-first scores can be.

AuthorsOpenReview ↗

Response to yufx (2/2): generality stated at the level of the evidence; mixed votes on 4,620 decisions — the net margin is an approximation, and evidence strength decides who is swayed

W2/Q3: generality. We agree that the panel calibration rests on two OpenAI base models with three stances each, and a stance is not an independent model (App. H, L1070–1073). The format evidence is broader but mixed. At one peer the two probes agree for 5 of 6 OpenAI models (Table 5). At 2–3 peers the answer-revision probe exceeds peer-first by 0.21–0.38 for GPT-4.1, GPT-5.4-mini and GPT-5-mini (Table 6). The post-hoc probe flags 4 of 6 models as cascade-prone and peer-first flags 1, so the choice of probe changes the screening decision for 3 of 6. The gap also appears for Claude Haiku 4.5 and Sonnet 5 (Table 8) and in a second domain (Table 18). Zero-parameter predictions for the existing GPT-5 and o4-mini panels (Table 23) fall inside the exact 95% intervals, but so would a no-conformity baseline, so they show only that the predictor raises no false alarm for robust models. We have not run panels for further base models.

We will state the claim at that level: panel prediction is calibrated for the OpenAI configurations tested, and remains open elsewhere. We agree the Claude Haiku panels are inconclusive (Table 9; in-panel conformity lies between the probes, Table 10).

W2/Q2: mixed votes. You are right that this was an untested assumption (L200–203). Each in-panel decision records its position and margin. We recovered the composition of all 4,620 cached decisions with earlier peers and re-traced each agent's private sample from the response cache (identical decisions, no model calls). Pooled, all cells with n ≥ 20:

δOpp–supConformityn
+11–0 / 2–1 / 3–2 / 4–30.10 / 0.26 / 0.27 / 0.18385 / 204 / 48 / 33
+22–0 / 3–1 / 4–2 / 5–30.27 / 0.47 / 0.31 / 0.68176 / 140 / 29 / 31

Q2 directly: at δ = +1, does 2–1 or 3–2 differ from 1–0? Yes, but only for agents whose own evidence is ambiguous:

Agent's own sample (lines over the limit, of 5)1–02–13–2
Ambiguous (2 or 3)0.19 (n=209)0.36 (132)0.50 (24)
Strong (1 or 4)0.00 (158)0.09 (56)0.04 (24)
Decisive (0 or 5)0/180/16—

The 244 decisive samples never conform at any margin. On the other 4,376 decisions we fit a logistic model with configuration fixed effects, evidence strength, panel size and adversary count, with errors clustered by episode. An opposing verdict multiplies the odds of conforming by 2.15, and a supporting one by 0.59. The net-margin restriction is rejected (p = 7×10⁻⁵): at a fixed margin, each additional earlier vote raises the odds 1.13×.

At the panel level the net margin still calibrates best on slope, R² and MAE among variants of the same model (26 conditions). For individual decisions the variants are close (Brier 0.0994 vs 0.0992 and 0.1093):

Panel-level modelSlope [95% CI]R²MAE
Net margin (row D)0.827 [0.75, 0.90]0.9490.054
Opposing count, opposing majority0.704 [0.62, 0.79]0.9230.059
Opposing count, any opposition0.561 [0.47, 0.65]0.8500.092

We will (i) state in §3 that the net margin is an approximation and report these tables and the regression, (ii) add that supporting voices protect less than opposing voices harm, consistent with our lone-dissenter result (L362–365), and (iii) disclose that the model sets conformity to zero at non-positive margins, whereas at a 1–1 tie we observe 0.12 (n = 373).

Thank you again for these questions. The mixed-vote analysis in particular has made the model's assumptions more explicit, and we would welcome any further comments.

AuthorsOpenReview ↗

Follow-up to yufx: the committed-debate test in the other direction, and GPT-5.4-mini as a new base model

Thank you again for your review. In our reply we described "the matching test in the other direction": a debate in which each agent's own earlier verdict stays in context. That is the revision setting, where the answer-revision probe should be the right measurement. We also said we had not run panels for further base models. Both are now done. They use the paper's code and prompts. Conditions that already appear in the paper were re-run through our cache and reproduce the published numbers exactly.

1. Committed debate: the result supports your reading. We used the same debate as Table 21 (n = 5, 3 rounds, one bare adversary, 30 panels for each of 4 configurations), with the same sealed round 1. In rounds 2–3, each honest agent now keeps its own conversation: its evidence, its earlier verdicts, then the others' verdicts. The table covers agents facing a 3–1 opposing majority:

Conformity [95% CI]nPeer-first curve predictsAnswer-revision curve predicts
Re-asked without own verdict (paper)0.46 [0.39, 0.54]1640.38–0.490.78
Committed (new)0.70 [0.62, 0.77]1510.38–0.480.75

Over all opposed decisions, the Brier score favours the answer-revision curve for committed agents (0.12 vs 0.19–0.26) and the peer-first curve for re-asked agents (0.22–0.23 vs 0.35). So, as you argued, the answer-revision probe is the right measurement where agents revise a committed answer, and the peer-first probe where they decide with peers visible. This tests the narrowed claim of our reply in both directions; we will add it to §5.6.

2. A new base model: GPT-5.4-mini. We ran the paper's pipeline unchanged on this model:

  • p on 150 items, and both curves on 100 items, for three stances;
  • visible panels with n ∈ {5, 9}, n_c ∈ {0, 1}, plus n_c = k_post where it fits; 30 panels per cell;
  • 16 new panel cells in all.

The measured thresholds are k_post = 2, 2 and none, and k_ctx = 5, 3 and none (neutral, deferential and independent stances).

Calibration slope [95% CI] uses the Table 1 statistic (band [0.7, 1.3]). "Inside 95% CI" counts the new cells whose exact 95% interval for observed error contains the prediction:

PredictorNew 16 cellsInside 95% CIPooled with the 26 original cells
A: answer-revision threshold0.61 [0.50, 0.71]10/160.42 [0.34, 0.51]
B: peer-first threshold1.41 [0.98, 1.84]4/161.41 [1.17, 1.65]
C: answer-revision curve0.81 [0.69, 0.92]14/160.50 [0.40, 0.61]
D: peer-first curve1.06 [0.91, 1.21]14/160.97 [0.88, 1.05] (R² 0.93, MAE 0.061)
No conformity (sealed)1.39 [0.88, 1.89]4/161.43 [1.14, 1.71]

What this shows, and what it does not:

  • Shows. The full peer-first curve calibrates on a base model that was not part of the original calibration. Pooled over 42 cells, it is the only predictor in the band.
  • Shows. Unlike the GPT-5 and o4-mini check (Table 23), this test discriminates: the no-conformity baseline fails, with only 4 of 16 cells inside.
  • Does not show. On GPT-5.4-mini alone, the answer-revision curve does about as well (mean absolute error 0.072 for both). This model's two curves are close at one peer (0.17 vs 0.21 for neutral), so these panels do not by themselves separate the two probes.
  • Does not show. Row D's two misses (neutral, n = 9) are underpredictions.
  • Does not show. Both threshold predictors fail at two attackers. The answer-revision threshold predicts certain capture (1.00 vs 0.63–0.80 observed), and the peer-first threshold predicts too little.

We will add GPT-5.4-mini to Table 1 and App. H with these caveats. Thank you again; your reframing suggested the committed-debate test, and it held up.

Comments on the paper

2 comments
AuthorsOpenReview ↗

General response: the two probes renamed (answer-revision vs peer-first), new analyses of cached decisions, and what remains open

We thank the reviewers for careful and specific reviews. We are glad they found the zero-fitted-parameter link from a single-agent measurement to panel outcomes useful (DuJ7, yufx), the controls and reproducibility valuable (yufx), and the evidence-versus-status asymmetry interesting (m3GY).

Summary of concerns and what we did

ConcernRaised byWhat we didStatus
The post-hoc format is not "the classic Asch design"yufx W1/Q1Renamed the two probes and narrowed the claim (below)In final version
Novelty beyond matching measurement to interactionm3GY W2/Q22×2 ablation re-read; what the framework adds beyond the matchNew analysis
Why a ½ threshold?m3GY W3It is the closest step approximation of the measured curve; τ-sensitivity for rows A and BNew analysis
Mixed votes and the net marginyufx W2/Q2Composition of 4,620 cached decisions: the net margin is an approximation, yet the best panel-level predictorNew analysis
Why partial evidence?m3GY W1/Q1Rationale; conformity sits in agents with ambiguous evidence (0 of 63 with decisive evidence conformed to an opposing majority)New analysis; complete information is a limitation
Generality and frontier modelsyufx W2/Q3, m3GY W4Claim scoped to the evidence; zero-parameter predictions for GPT-5 and o4-mini panelsNew analysis; limitation
Later rounds, committed agentsm3GY W2/Q2The existing debate follows the in-context curve (0.46 observed, 0.38–0.49 predicted)New analysis; committed debate is a limitation
Results in prose; Fig. 2 unreadableDuJ7 W1Results-at-a-glance table, takeaways, Fig. 2 at print sizeIn final version
Attacker/defender orderDuJ7 Q1What the model predicts and what the paper already showsLimitation

The reframing (yufx). In Asch (1956), participants heard the confederates before answering (our App. B.1, L687–688), as in BenchForm (Weng et al., 2025). Calling our post-hoc format "the classic Asch design" or "the standard benchmark" (L023, L070, L100–101, L123–124, L162–163) was inaccurate. In the final version:

  • the post-hoc format becomes the answer-revision (commit-then-challenge) probe, the structure of sycophancy challenges (Sharma et al., 2024) and of debate revision rounds;
  • the in-context format becomes the peer-first probe, the structure of Asch and of BenchForm;
  • the claim becomes: revising a committed answer and following peers when first deciding are different quantities (0.91 vs 0.21 to one peer for GPT-4.1-mini, Table 7), and in our sequential panels only the latter predicts outcomes (Tables 1, 12). We do not claim that peer-first benchmarks are invalid; our results support them. "Benchmark failure" is removed.

The following are unchanged: Proposition 1, the zero-parameter calibration (Table 1 row D), the held-out tests (Table 15, §5.9), the in-panel mechanism check (Table 12), the localisation of the gap to contradicting one's own prior turn (Table 24), and the evidence-claim results (§5.6–5.8).

A correction to Table 1 row B and four Table 5 thresholds, found while preparing the revision, is posted as a separate comment. No conclusion changes.

What remains open. We have not tested a complete-information condition, a debate in which each agent sees its own earlier verdict, an attacker/defender order swap, or panels for further base models. We state these as limitations and scope our claims to partial evidence, sequential panels and the configurations tested. If we can complete any of them before the discussion closes, we will post it here. The final version will contain everything marked "In final version" and the new analyses.

AuthorsOpenReview ↗

Correction: Table 1 row B and four Table 5 thresholds used an outdated estimator (conclusions unchanged)

While preparing the revision we found that Table 1 row B used an outdated in-context threshold for one configuration: GPT-4.1-mini, deferential, with k_ctx = 3 instead of 2. The value 3 came from an earlier logistic-fit estimator. The direct estimator of Definition 1, which App. F says we use throughout, gives 2, matching the k_ctx = 2 already listed in Tables 20 and 26. The stored in-context curve is 0.25, 0.56, 0.72, … for 1, 2, 3, … opposing peers, so it first exceeds ½ at two peers. Recomputed with the corrected threshold:

Table 1, row B (in-context probe, threshold)Slope [95% CI]R²MAE
Submitted1.462 [1.11, 1.81]0.7380.143
Corrected1.341 [1.05, 1.63]0.7770.131
  • Rows A, C and D are unchanged. Row D (0.827 [0.75, 0.90], R² 0.949, MAE 0.054) remains the only row whose interval lies inside the pre-specified band [0.7, 1.3].
  • Row B's point estimate stays above the band and, as before, its interval overlaps it.
  • Four thresholds in Table 5 (three post-hoc, one in-context) came from the same earlier estimator and are corrected. That table's conclusion rests on φ(1) and does not change. App. F's remark that the earlier estimator "changed no post-hoc threshold" holds only for the six stance configurations, and we correct it.

The final version and the supplement will carry the corrected values. We apologize for the error.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.