acceptodds
Under review as a conference paper at ICLR 2027

Endogenous Byzantine Faults: Measuring and Containing Fault Contagion in LLM Multi-Agent Systems

Abstract

LLM multi-agent systems use voting, debate, and peer review to obtain reliability from redundancy, on a premise inherited from Byzantine fault tolerance (BFT): an honest majority outvotes a faulty minority. The premise treats the set of faulty agents as fixed by the adversary. We show that in LLM systems the fault set is endogenous: an honest agent becomes faulty by reading a protocol message, so the fault spreads through ordinary use of the protocol. We formalize a contagious Byzantine fault model with three measurable quantities, the per-hop infection rate , the payload replication rate , and the inter-replica failure correlation , under which an honest majority survives fewer than adversaries under independent compromise, i.e. none once ; contagion across rounds adds a cascade condition and correlated replicas an irreducible failure floor. The vote share these quantities imply tracks debate accuracy over 62 systems (, held-out , no better than alone) and places 53 of them on the correct side of the majority threshold without fitting. In about 780k agent calls over 11 models from seven families and 1300 questions, a message that only claims coordinator authority flips 97% of Llama-3.3-70B agents on arithmetic they can verify themselves, while a worked argument flips 0%. Within the model families we test, larger models are harder to persuade but easier to command, and some models forward the payload at every hop of a relay chain. One adversary among five drives debate accuracy from 74–90% to 2% or less for four large models. Mixing vulnerable model families does not help; only members that resist the instruction channel do. A provenance gate that admits peer evidence but not peer directives restores accuracy to 72–87% and, unlike a warning prompt, mostly preserves the rate at which agents accept legitimate corrections; an adaptive attacker is left with the weaker evidence channel, which the gate does not address.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
multi-agent LLM systems, Byzantine fault tolerance, fault contagion, prompt injection, multi-agent debate, agent security

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 4 reviews, above 68% of the others.
Reviewer_xhAZOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper studies the contagion Byzantine faults in LLM multi-agent systems. The paper provides condition on f/n (the fraction of initial faults) under which we can operate without breakdown and proposes a technique for fault containment to control the infection rate (rate of recruitment by Byzantine agents). Some theory is presented to support the claims and empirical study shows the efficacy of the proposed scheme.

Critical strengths
  1. The contagious Byzantine fault model is interesting. Faulty-behavior propagation through message passing is a real and under-studied phenomenon in agentic systems, and it needs more attention.
  2. The connection to epidemic modeling is informative, and framing contagion among LLM agents in those terms is a useful perspective.
Critical weaknesses
  1. The abstract is poorly organized. The second half does not correspond to the first half: after the contagion model and the threshold n(1−2β)/(2(1−β))n(1-2\beta)/(2(1-\beta)) are introduced, the text jumps straight into experimental results that do not make much sense at that point, they come too soon and without the setup needed to interpret them. Parts of the abstract also read as LLM-generated without any human proofreading.
  2. The contributions are stated in a hand-wavy manner. The claims in the Contributions paragraph are asserted rather than argued, and the quantities are used before they are properly defined - making it difficult to get the sense of the results.
  3. Theorems are stated without interpretation and novelty is unclear. Theorem 1 and Theorem 2 are given as bare mathematical statements; the paper does not explain what they show. Since the contagion process is essentially a Reed–Frost/noisy-OR epidemic model, it is also not made clear which parts of these results are new relative to the epidemic and correlated-jury literature.
  4. The agent-side resistance is not detailed enough. The containment filtering in Section 6 is not formally presented and difficult to grasp.
  5. Informal statements and forward references. Key conditions are not defined formally. For instance, the requirement in Theorem 2 that the compromise indicators be “exchangeable, extendable to arbitrarily many replicas”. The notion of an agent and its operating framework is likewise never formalized. The text also refers to (Q3) on line 159 before the research questions have been introduced; they appear only in Section 5.
  6. The empirical results appear to indicate the weakness (fragility) of the attack than the strength of the defense. In Figure 2, the provenance gate restores accuracy to near the no-adversary level, which suggests the threat is brittle.
Questions for authors
  1. What exactly is an agent in this framework, and what is the operating framework assumed for it?
  2. Why can’t honest agents perform checks on incoming messages? More generally, why not make the “recruitment” robust?
  3. How is the proposed contagion model different, technically speaking, from standard epidemic modeling?
  4. What are Theorems 1 and 2 showing? Could the authors provide an interpretation of each statement.
  5. Could the exchangeability and extendability condition in Theorem 2 be defined formally?
Reviewer_uKNFOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper argues that the Byzantine fault tolerance premise behind voting and debate in LLM multi-agent systems fails because faults are endogenous: an honest agent becomes faulty by reading a peer message. It defines three measurable quantities: per-hop infection rate β\beta, payload replication rate βrep\beta^{rep} and inter-replica correlation ρ\rho. From these it derives a static adversary threshold, a contagion chain and a redundancy floor. Across 11 open models and 1,300 questions, a forged coordinator notice compromises most models far more than a persuasive argument does, and one adversary among five collapses debate accuracy.

Critical strengths
  1. Clear framing and well-separated measurements. Distinguishing the instruction channel from the evidence channel, and answer infection from payload replication, yields findings that are easy to act on. Examples are the 0% vs. 97% flip rate on verifiable arithmetic, and Ministral-14B with β=98%\beta = 98\% but βrep=10%\beta^{rep} = 10\%. The benign-correction control measures what each defense costs, which most robustness studies omit.

  2. Careful and honest empirical practice. The study is large (≈720k agent decisions) and uses cluster-level intervals, held-out prediction against baselines, explicit invalid-reply rules, and controls such as faithful paraphrase and a sender tag. The authors openly report where their own model adds little, for example that β\beta alone predicts as well as the vote share and that the contagion chain does not beat persistence.

Critical weaknesses
  1. The theoretical contribution is thin. The authors acknowledge that Proposition 1 is linearity of expectation, the chain is Reed–Frost, and Theorem 2 is a correlated-jury result. The held-out analysis shows that the vote-share statistic predicts no better than β\beta alone (MAE 9.8 vs. 9.1), and the contagion chain does not beat carrying round 1 forward. Some assumptions are contradicted by the data (conditional independence vs. measured ρ\rho; an absorbing state vs. agents that revert).

  2. The threat model is narrow, and the empirical novelty is moderate. The central attack is a forged coordinator notice in a setup with no sender authentication, hidden reasoning disabled, 60-word answers and short output budgets. The two closed frontier models tested resist it. The main finding therefore largely measures the instruction-following of open models under minimal prompting, which is close to known prompt-injection and prompt-infection results (Greshake et al., Lee et al., Cui & Du). All system-level results use n=5n = 5 on multiple-choice tasks.

  3. The defense evaluation is limited. The gate's filter model (Qwen3.7-flash) was selected on the evaluation questions themselves. Swapping in DeepSeek-V4-flash raises infection from 1% to 26%, so the result depends on one filter choice. The gate is essentially an LLM-based injection filter and is not compared with published defenses (spotlighting, instruction hierarchy, injection classifiers). Only one hijack line against the filter is tested, and an adaptive attacker on the evidence channel largely bypasses it.

  4. Statistical and experimental control. Debate and committee cells use 100 questions with one outcome each (±10-point intervals), so many between-cell differences cannot be ranked. Models are accessed through a routing service with uncontrolled versions and quantization. The scale claim ("harder to persuade, easier to command") rests on two families, and in one of them size is confounded with release, while it fails across families.

Questions for authors
  1. How do the infection and collapse results change with hidden reasoning enabled and larger output budgets, which are typical deployment settings?

  2. With sender authentication, how much of the attack survives through payload replication by honest members alone?

  3. How does the gate compare with spotlighting, instruction-hierarchy models or a trained injection classifier, and how robust is it to stronger, iterated attacks on the filter itself?

  4. Can the filter model be chosen on held-out questions, and how does performance vary across several filter models?

  5. Do the threshold and redundancy-floor predictions hold for larger nn (e.g., 9 or 15 agents)?

Reviewer_3TwNOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how malicious peer messages can spread errors through multi-agent LLM systems and undermine group decisions. Across eleven models and three task sets, it measures wrong-answer adoption, attack-message forwarding, and correlated failures, and uses a simplified contagion model to interpret the results. The authors also evaluate a message-filtering defense that reduces instruction-based attacks while largely preserving legitimate corrections, but remains vulnerable to adaptive attacks using misleading reasoning.

Critical strengths
  1. The study distinguishes instruction-based attacks from misleading arguments and separates wrong-answer adoption from attack-message forwarding.
  2. Comparisons within the tested Llama and Ministral families reveal that larger models better resist misleading arguments but are more susceptible to forged notices.
  3. The provenance gate restores debate accuracy to 72–87% under the evaluated forged-notice attacks, while largely preserving legitimate corrections.
Critical weaknesses
  1. The experimental section needs clearer organization: Q1–Q7 use inconsistent heading levels and are spread across sections, making the narrative difficult to follow.
  2. Q1, Q3, and Q7 overlap with prior studies of instruction-based attacks, malicious propagation, and security–collaboration trade-offs, such as Multi-Agent Security Tax (https://ojs.aaai.org/index.php/AAAI/article/view/34970). Without a clearer account of the additional findings, the originality of this part of the study is difficult to establish.
  3. The central phenomenon in Q4, that one manipulated agent can mislead group decisions, was already studied in Can an Individual Manipulate the Collective Decisions of Multi-Agents? (https://aclanthology.org/2025.emnlp-main.611/). This reduces the novelty of the empirical observation unless the analysis adds explanatory or predictive value beyond prior work.
  4. The defense provides a limited methodological contribution. Its filter model is selected using the evaluation questions, its protection against adaptive attackers is limited, and comparisons with established defenses are missing. These limitations weaken the evidence for a generalizable defensive advantage.
Questions for authors
  1. What new insights do Q1, Q3, and Q7 provide beyond those established in Multi-Agent Security Tax?
  2. Beyond showing that one manipulated agent can mislead group decisions, what additional explanatory or predictive value does Q4 provide?
  3. Does the gate’s performance generalize to held-out questions and tasks when the filter model is selected on a separate development set?
  4. How does the provenance gate compare with existing defenses discussed in this paper such as INFA-Guard and SAC under comparable attack settings, considering infection rate, legitimate correction, and computational cost?
1 comment

Thank you for the detailed response and the additional experiments. The first three questions are largely addressed.

My remaining concern is Q4: the comparison uses a simplified version of INFA-Guard and omits mentioned SAC. Could you include SAC and report infection rate, legitimate correction rate, and computational cost for all methods under the same conditions?

In addition, could you clarify how the “independent answers” accuracy in Table 43 was computed, including how votes were aggregated? For each model and attacker count, could you also report the percentage of questions that at least one non-adversarial agent answered correctly before exchanging messages? This would give an oracle upper bound for selecting among the initial answers.

Reviewer_mWkJOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies how adversarial messages spread through LLM multi-agent systems. It separates answer adoption from payload replication, relates measured susceptibility and correlated failures to voting reliability, and evaluates a provenance gate that removes peer directives while retaining evidence.

Critical strengths
  1. The paper systematically investigates fault contagion through clearly defined research questions. Complementary exposure, relay, and debate experiments provide useful insights into message susceptibility and fault propagation in the evaluated settings.
Critical weaknesses
  1. The main debate experiments omit explicit trust-boundary instructions for peer messages. Warnings are evaluated only under single-agent exposure, where they substantially change susceptibility. This leaves uncertainty about whether the system-level findings and the gate's benefits persist with this basic safeguard.
  2. The evaluation provides limited evidence that the gate itself resists adaptive attempts to override its filtering instructions. Table 7 includes a fixed filter-hijack test, while the adaptive attacks primarily exploit reasoning that the gate is designed to retain.
Questions for authors
  1. How do the main debate results change when all honest agents receive the warning in Appendix I? What additional benefit does the gate provide over this warning-only baseline?
  2. How robust is the gate to adaptive attempts to make it follow peer instructions and retain directives it should remove? Can these filter failures be evaluated separately from adversarial reasoning that passes through by design?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.