- Rating
- 4
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This work investigates the vulnerability of teams of LLM agents to conformity bias and proposes a closed-form method of approximating the error-floor for a particular agent team. To accomplish this, the authors set up an experiment in which each LLM agent has access to private information which it must use to provide a verdict. Importantly, one (or more) of the agents on the team is an attacker who intentionally gives the incorrect verdict in an attempt to bias the other agents. This experimental paradigm is used to test the predictions of the threshold model, with the key discovery being that the model is more accurate at predicting capture when conformity is measured in-context rather than post-hoc, and this accuracy can be further improved by simulating panel decisions using the full experimentally-derived conformity curve. Lastly, the authors also extend their study to investigate different forms of attack (e.g., asserting access to privileged evidence), as well as potential defenses, and the generalizability of their approach to other agents/LLM providers.
Critical strengths
-
The authors do a good job addressing a timely and important topic: Agent teams have become increasingly used for approaching complex problems, and the tendency of LLM agents towards agreeableness is a well documented weakness. The present work therefore not only addresses an important and timely topic, but does so through an experimental paradigm which effectively isolates the bias in question and quantifies its possible detriment to agent team performance. This could be useful for future research focused on developing team's of agents which are robust to attack.
-
The proposed model is compelling and applicable: The main contribution of the paper is an elegant model for estimating the error rate for a team of agents whose conformity bias is under attack. The model depends only on estimating the behavioral parameters of a single agent: namely that agent's accuracy and their conformity threshold (or for even more improved accuracy, by using the measured conformity curve). Importantly, the model does not require any fitting or fine-tuning to predict team behavior. The flexibility of this approach could prove useful for providing a preliminary evaluation on the robustness of LLM teams to this form of attack without expensive experiments involving teams of agents.
Critical weaknesses
- Presentation of results impedes readability: In various parts of the main text, the results are presented either (a) in prose, or (b) with references to the Appendix. In my opinion, this significantly hinders the readability of the manuscript and at times makes it too difficult to appreciate the key outcomes. For example, L305 states: "pooled over configurations it rises through 0.095, 0.164, 0.380, 0.479, 0.645, 0.796 as the opposing goes from 0 to 5." This is just one example. It is my strong belief that it would be more appropriate to either include the figure in the main manuscript or to state the main takeaway more plainly. Similarly, Figure 2 appears to be the main results-focused figure in the main text, but its small text makes it very difficult to read.
Questions for authors
- In the setting with one attacker and one defender, each claiming access to privileged information, did you investigate whether the ordering has any bearing on the outcome of the attack? In other words, is there a difference between the attacker going first and defender second and the defender going first and attacker second? If there is a discovered sensitivity to ordering, this could possibly be interesting to report.
2 comments
Follow-up to DuJ7 Q1: we ran the attacker/defender order swap; order matters, and the margin-only model misses it
Thank you again for Q1. We said we would post the order swap if we could complete it before the discussion closes; it is now done.
Setup. This is the defender study of Table 22 with one change: the defender speaks first and the attacker second. Everything else is the same: four configurations, n = 9, the same 30 batches, and an attacker who posts the wrong verdict together with the evidence claim. We re-ran the attacker-first conditions through our response cache, and they reproduce Table 22 exactly.
Result: the first speaker has a large advantage. Panel error:
| Configuration | No defender | Plain defender, attacker first | Plain defender, defender first | Evidence defender, attacker first | Evidence defender, defender first |
|---|---|---|---|---|---|
| GPT-4.1-mini, neutral | 0.933 | 0.833 | 0.433 | 0.233 | 0.000 |
| GPT-4.1-mini, deferential | 0.967 | 0.900 | 0.533 | 0.667 | 0.133 |
| GPT-5-mini, neutral | 0.233 | 0.200 | 0.167 | 0.233 | 0.133 |
| GPT-5-mini, deferential | 0.267 | 0.233 | 0.267 | 0.100 | 0.133 |
| Pooled (120 panels) | 0.600 | 0.542 | 0.350 | 0.308 | 0.100 |
We compared the two orders on the same 120 panels, using an exact McNemar test on the panels where the order changed the outcome:
- plain defender: defender-first was right where attacker-first was wrong in 26 panels, and the reverse in 3 (p = 1.5 × 10⁻⁵);
- defender with the evidence claim: 27 panels vs 2 (p = 1.6 × 10⁻⁶).
The effect comes from GPT-4.1-mini. For GPT-5-mini, order changes panel error by at most 0.10.
The margin-only cascade model does not predict this. In either order, the first honest agent sees one attacker and one defender, a visible margin of 0, so the model predicts the same behaviour. It does not behave the same:
- with a plain defender, it votes wrong 0.61 of the time when the attacker spoke first and 0.39 when the defender did;
- with the evidence claim, 0.38 and 0.18;
- among agents whose own sample pointed to the right answer (96 of 120), 0.53 and 0.26 with a plain defender.
So the first post carries more weight than the margin alone implies: a primacy effect.
What we will change. In §5.8 we will:
- report the swap and state that the cascade model ignores order effects beyond the visible margin;
- list this as a limitation of the margin-only model;
- note the practical consequence: a cheap defense is to seat a trusted agent first.
Thank you for the question. It exposed an effect our model misses.
Response to DuJ7: §5 restructured and Fig. 2 re-rendered; what the cascade model and the existing data say about attacker/defender order
Thank you for the positive assessment and for the concrete presentation suggestions. We agree with both.
W1: results in prose and in the appendix. We will restructure §5 so that each finding can be read without the appendix:
W1: Figure 2. The figure was drawn on a 13.6 × 3.2 in canvas and scaled to text width, which shrinks the fonts to about 40%. We re-render it at print size as a 2 × 2 grid with text of at least 8 pt and larger markers.
Q1: does attacker/defender order matter? We did not test this. In the defender study (Table 22, L394–398) the attacker always speaks first and the defender second. There, a plain defender lowers mean panel error only from 0.600 to 0.542, and the same defender carrying an evidence claim lowers it to 0.308. Two related results are already in the paper:
The cascade model of §3 makes only a weak prediction here. Once both have spoken, the visible margin is 0 whatever the order, so it predicts no order effect for the agents who follow. But it ignores the content of claims and how an LLM weighs earlier against later messages in its context (primacy vs recency). A swap (defender first, attacker second, with and without the defender's evidence claim, on the four Table 22 configurations) would isolate exactly that. We have not run it. We will state the prediction in §5.8 and list the swap as a limitation; if we can complete it before the discussion closes, we will post it here.
Thank you again. We hope these changes make the results much easier to follow, and we would be glad to hear any further suggestions.