- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper studies the contagion Byzantine faults in LLM multi-agent systems. The paper provides condition on f/n (the fraction of initial faults) under which we can operate without breakdown and proposes a technique for fault containment to control the infection rate (rate of recruitment by Byzantine agents). Some theory is presented to support the claims and empirical study shows the efficacy of the proposed scheme.
Critical strengths
- The contagious Byzantine fault model is interesting. Faulty-behavior propagation through message passing is a real and under-studied phenomenon in agentic systems, and it needs more attention.
- The connection to epidemic modeling is informative, and framing contagion among LLM agents in those terms is a useful perspective.
Critical weaknesses
- The abstract is poorly organized. The second half does not correspond to the first half: after the contagion model and the threshold are introduced, the text jumps straight into experimental results that do not make much sense at that point, they come too soon and without the setup needed to interpret them. Parts of the abstract also read as LLM-generated without any human proofreading.
- The contributions are stated in a hand-wavy manner. The claims in the Contributions paragraph are asserted rather than argued, and the quantities are used before they are properly defined - making it difficult to get the sense of the results.
- Theorems are stated without interpretation and novelty is unclear. Theorem 1 and Theorem 2 are given as bare mathematical statements; the paper does not explain what they show. Since the contagion process is essentially a Reed–Frost/noisy-OR epidemic model, it is also not made clear which parts of these results are new relative to the epidemic and correlated-jury literature.
- The agent-side resistance is not detailed enough. The containment filtering in Section 6 is not formally presented and difficult to grasp.
- Informal statements and forward references. Key conditions are not defined formally. For instance, the requirement in Theorem 2 that the compromise indicators be “exchangeable, extendable to arbitrarily many replicas”. The notion of an agent and its operating framework is likewise never formalized. The text also refers to (Q3) on line 159 before the research questions have been introduced; they appear only in Section 5.
- The empirical results appear to indicate the weakness (fragility) of the attack than the strength of the defense. In Figure 2, the provenance gate restores accuracy to near the no-adversary level, which suggests the threat is brittle.
Questions for authors
- What exactly is an agent in this framework, and what is the operating framework assumed for it?
- Why can’t honest agents perform checks on incoming messages? More generally, why not make the “recruitment” robust?
- How is the proposed contagion model different, technically speaking, from standard epidemic modeling?
- What are Theorems 1 and 2 showing? Could the authors provide an interpretation of each statement.
- Could the exchangeability and extendability condition in Theorem 2 be defined formally?
Thank you for the detailed response and the additional experiments. The first three questions are largely addressed.
My remaining concern is Q4: the comparison uses a simplified version of INFA-Guard and omits mentioned SAC. Could you include SAC and report infection rate, legitimate correction rate, and computational cost for all methods under the same conditions?
In addition, could you clarify how the “independent answers” accuracy in Table 43 was computed, including how votes were aggregated? For each model and attacker count, could you also report the percentage of questions that at least one non-adversarial agent answered correctly before exchanging messages? This would give an oracle upper bound for selecting among the initial answers.