acceptodds
Under review as a conference paper at ICLR 2027

SEIAR-D: Quantifying Error Propagation in LLM Multi-Agent Systems

Abstract

Complex collaborative tasks have driven the rapid development of LLM-based multi-agent systems (LLMAS). However, intensive interaction also introduces systemic risks. A local error can propagate, be reconstructed, and be amplified across agents, eventually leading to system-level failure. Existing studies have begun to examine error cascades and collective error retention, while the internal propagation process remains less understood. To address this gap, we develop a theoretical framework from four perspectives: relay propagation, semantic amplification, population-level criticality, and error consolidation risk. It models error evolution at both the agent and system levels. We study how normal agents become new propagation sources, how semantic amplification strengthens propagation, and when errors shift from decay to sustained expansion. We further define consolidation risk from propagation states and transitions for early warning. Experiments show that errors among LLM agents are not isolated or static events, but dynamic processes involving propagation, amplification, recovery, and critical transitions. Our framework provides a unified view for understanding and predicting error risks in LLMAS.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM Multi-Agent Systems, Error Propagation, Semantic Amplification, Error Dynamics

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_W9MMOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes SEIAR-D, a hierarchical model for error propagation in LLM multi-agent systems. It distinguishes six agent-level states, including error exposure, acceptance, amplification, deviation, and recovery. The authors analyze relay propagation, semantic amplification, population-level criticality, and early prediction of error consolidation. Experiments across four LLMs suggest that agents can amplify errors through new supporting reasoning and that intermediate propagation states can help predict final system failures.

Critical strengths
  1. Distinguishing error repetition from semantic amplification is meaningful. LLM agents can generate new reasoning that reinforces an incorrect claim, and the proposed state taxonomy captures this behavior more explicitly than binary propagation models.

  2. The empirical evaluation is relatively extensive, covering four LLMs and multiple intervention settings. The early-warning results are also promising, suggesting that intermediate agent states contain useful predictive signals.

Critical weaknesses
  1. The overall novelty is limited, as the framework mainly integrates existing ideas rather than introducing fundamentally new insights.

  2. Most of the theorems follow directly from standard probability or dynamical-systems results. For example, the multi-path result follows from the monotonicity of event unions, the amplification result assumes nondecreasing propagation probabilities, and the criticality condition uses the standard spectral-radius criterion. Their specific contribution to understanding LLM behavior is therefore unclear.

  3. The framework relies heavily on automatically distinguishing states such as Infected, Amplifying, and Deviated, but the reliability of these labels is not adequately evaluated. The assumption that aggregate state counts form a sufficient Markov state is also strong.

  4. Moreover, the static-state baseline already achieves very high AUROC, making the added benefit of transition modeling relatively modest.

Questions for authors
  1. Can the authors report human annotation agreement, particularly for Infected, Amplifying, and Deviated?
  2. What evidence supports the Markov assumption on aggregate state counts? Would including topology or interaction history improve prediction?
  3. Can the authors isolate the contribution of transition modeling over the strong static-state baseline and report statistical significance?
  4. Since the amplification theorem assumes nondecreasing acceptance probabilities, can this assumption be validated empirically rather than imposed?
3 comments
AuthorsOpenReview ↗

We appreciate the reviewer's insightful comments, which have motivated us to strengthen the theoretical analysis and provide more rigorous empirical validation. We address the key concerns below.

1. LLM-Specific Theoretical Insights

Our theoretical analysis builds on standard mathematical tools. Its main contribution is to characterize how error acceptance (I) and semantic amplification (A) interact in LLM-based multi-agent systems.

New Proposition: Cross-State Semantic Feedback Criticality

Consider the SEIAR-D next-generation matrix:

k_{II} & k_{IA}\\\\ k_{AI} & k_{AA} \end{pmatrix}\geq 0,$$ where $k_{uv}$ is the expected number of new state-$u$ carriers generated by a state-$v$ carrier. When $k_{II}<1$ and $k_{AA}<1$, neither state can sustain propagation alone. However, $\rho(K)>1$ if and only if $$\boxed{k_{IA}k_{AI}>(1-k_{II})(1-k_{AA})}.$$ **Theoretical Insight:** Error acceptance and semantic amplification can reinforce each other. Agents may accept an error, generate new supporting arguments, and induce further error acceptance. This cross-state feedback can sustain propagation even when neither state can do so independently. This proposition provides a theoretical explanation of how newly generated reasoning can contribute to persistent errors in LLM multi-agent systems. We will include its proof in the revised paper. ### 2. Reliability of Human Annotations We use a Rule-first Hybrid Evaluator. Rules identify S, E, D, and R, while an LLM distinguishes I from A. We stratified 360 samples from four LLMs. Two human annotators labeled them independently and resolved disagreements through arbitration. Overall human agreement reached 92.2%, with Cohen's $\kappa=0.906$. Class-level results for I, A, and D are: | State | Human Agreement (%) | Cohen's κ | Evaluator F1 (%) | |---|---:|---:|---:| | I (Infected) | 88.8 | 0.857 | 89.2 | | A (Amplifying) | 88.1 | 0.854 | 90.0 | | D (Deviated) | 95.6 | 0.945 | 88.3 | Human Agreement denotes positive agreement. Class-level $\kappa$ is computed using one-vs-rest labels. Against the arbitrated labels, the evaluator achieved 93.3% accuracy and 93.3% Macro-F1. In addition, 115 of 120 I/A samples received identical labels across three independent evaluations, showing 95.8% stability. These results support the reliability of our annotations. We will add these results and clarify the I/A/D labeling criteria in the revised paper. ### 3. Markov Assumption, History, and Network Topology The first-order Markov assumption is an approximation of the system's macro-state dynamics. We conduct two additional experiments to evaluate its effectiveness. (1) Impact of Historical Information | Method | Average AUROC | |---|---:| | Binary Propagation | 0.9109 | | First-order SEIAR-D | 0.9895 | | Second-order SEIAR-D | 0.9936 | | Full-history | 0.9932 | First-order SEIAR-D outperforms Binary Propagation by 0.0786 AUROC. Second-order and Full-history provide only 0.0041 and 0.0037 additional gains. This suggests that first-order states capture most predictive information. (2) Impact of Network Topology and Interaction Exposure Our experiments use a fixed, fully connected topology, so structural features remain constant. We therefore add interaction exposure features, including the Target Agent's error-message exposure, new supporting arguments received, and cumulative error exposure. | Method | Average AUROC | |---|---:| | First-order SEIAR-D | 0.9895 | | First-order SEIAR-D + Interaction Exposure | 0.9923 | Interaction exposure improves AUROC by only 0.0028. These results support first-order SEIAR-D as an effective and compact approximation. We will include these results and clarify the scope of our assumptions in the revised paper. ### 4. Independent Contribution of State-Transition Modeling We introduce a Matched Static State baseline on MiniMax-M3. It uses the same input features as SEIAR-D but does not explicitly model state transitions. | Method | AUROC ↑ | F1 (%) ↑ | Warning Round ↓ | |---|---:|---:|---:| | Static State | 0.9884 | 90.00 | 2.39 | | Matched Static State | 0.9879 | 89.80 | 2.45 | | SEIAR-D (Ours) | **0.9914** | **93.62** | **1.23** | Compared with Matched Static State, SEIAR-D improves AUROC by 0.0035 and F1 by 3.82 percentage points, while providing warnings 1.22 rounds earlier. Paired bootstrap analysis (2,000 resamples) and paired significance tests confirm statistically significant improvements: | Metric | 95% CI | p-value | |---|---:|---:| | ΔAUROC | [0.0006, 0.0067] | 0.018 | | ΔF1 (pp) | [0.9, 6.4] | 0.012 | These results confirm that explicit state-transition modeling improves prediction performance and early warning, even with matched input features.
AuthorsOpenReview ↗

Additional Analysis

5. Empirical Validation of the Monotonicity Assumption

We thank the reviewer for this suggestion. To test the monotonicity assumption in Theorem 3, we conduct a length-matched, paired hop-level experiment. We fix the task, receiving agent, interaction history, and experimental settings. We only change whether the message contains new supporting reasoning, keeping the token length difference within ±5%. For each model, we select 150 paired samples and repeat each condition five times.

Experimental results:

ModelI Acceptance (%)A Acceptance (%)Δ (pp)Negative Cases
MiniMax-M331.141.4+10.34/150
GPT-5.6-Luna35.145.8+10.72/150
DeepSeek-V4-Pro29.435.3+5.99/150
Gemini-2.5-Flash17.421.3+3.96/150

New supporting reasoning increases error acceptance by 3.9–10.7 percentage points across all four models. These results provide direct hop-level empirical support for the monotonicity assumption in Theorem 3.

However, a small number of negative cases remain. In these cases, new arguments may expose factual errors or logical flaws, prompting agents to reject or correct the target error.

Thus, our results support monotonicity at the aggregate level, but do not imply that it holds for every interaction. Theorem 3 guarantees cascading gains under shared effective exposure and the hop-level condition αℓA≥αℓI\alpha_\ell^A \geq \alpha_\ell^I. We will include these results and clarify the theorem's scope in the revised paper.

Thank you for your response. While some of my concerns have been addressed, my primary concern regarding the paper's limited novelty remains unresolved. The additional analysis does not fundamentally change my assessment of the paper's original contribution. I will maintain my current score.

Reviewer_Pmx5OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes SEIAR-D, a hierarchical framework for modeling error propagation in LLM-based multi-agent systems. Instead of treating error propagation as a binary process, the framework introduces six fine-grained agent-level states to characterize error exposure, acceptance, semantic amplification, deviation, and recovery. It further connects these intermediate states to system-level outcomes. Experiments on BBH tasks using four LLMs investigate how normal agents relay or amplify injected errors and whether intermediate propagation states can help predict final system outcomes.

Critical strengths
  1. The paper focuses on the issue of error propagation in multi-agent systems, a topic of practical significance and provides a formal derivation for the modeling.
Critical weaknesses

1.The paper attempts to model the propagation of errors within the system; however, some of the conclusions are rather trivial, and certain theoretical derivations appear to lack rigor. For instance, given the discussion on how agents might amplify errors, it is natural to ask why the scenarios where agents attenuate or even correct errors were not addressed—factors that directly determine whether the propagation chain described in Theorem 1 remains unbroken.

2.The authors do not clearly describe the experimental details; an examination of the source code in the supplementary materials reveals that the classification of the six states relies on a combination of static rules and LLM, yet no sufficient evidence is provided to demonstrate the reliability of this classification. Furthermore, regarding RQ4, the improvement of the proposed method over the baseline is not significant.

Questions for authors

1.Why does the Clean setting in Table 2 exhibit a non-zero ASR, and how are naturally occurring errors distinguished from attack-induced errors? Could the authors also provide evidence supporting the reliability of the six-state labeling procedure?

2.What empirical evidence supports the "monotonic acceptance" assumption in Theorem 3? Have the authors examined whether newly generated supporting inferences reduce the rate of erroneous acceptance or facilitate error correction?

2 comments
AuthorsOpenReview ↗

We sincerely thank the reviewer for the careful evaluation and constructive feedback. Your comments have helped us identify important points that require further clarification and stronger empirical evidence. We address these concerns below and will incorporate the corresponding improvements into the revised paper.

1. Error Attenuation and Recovery Mechanisms

LLM agents have inherent abilities to detect and correct errors. These abilities are part of their reasoning and interaction processes. SEIAR-D captures these behaviors through E (exposure without acceptance), A→I (reduced semantic amplification), and I/A/D→R (error recovery). Our focus is on how errors can still propagate across agents despite their inherent error-correction abilities.

Theorem 1 characterizes the possibility of multi-hop propagation in the presence of error correction. It does not assume that errors always propagate. The success probability of each hop, qℓ=βℓαℓq_\ell=\beta_\ell\alpha_\ell, accounts for propagation failures caused by error rejection or correction. If the conditional success probability of any hop is zero, the path cannot succeed. Thus, Theorem 1 shows that even with inherent error-correction abilities, a local error can still reach downstream agents through a propagation path with a positive success probability. We will clarify this theoretical scope in the revised paper.

2. Annotation Reliability and RQ4 Performance

Regarding annotation reliability, we sampled 360 interaction records from four LLMs. Two human annotators labeled them independently and resolved disagreements through arbitration. Human agreement was 92.2%, with Cohen's κ=0.906\kappa=0.906. Against the arbitrated labels, the evaluator achieved 93.3% accuracy and Macro-F1. The F1 scores for S, E, I, A, D, and R were 98.3%, 96.7%, 89.2%, 90.0%, 88.3%, and 97.5%, respectively. For 120 I/A samples, three independent evaluations achieved 95.8% label stability. These results support the reliability of our annotation method.

Regarding RQ4, Static State and I/A Ratio both use SEIAR-D state representations. They serve as simplified internal baselines and therefore already provide strong predictive signals. IBMF is an existing external baseline. On MiniMax-M3, our method achieves an AUROC of 0.9914, compared with 0.8175 for IBMF. Compared with Static State, our method improves F1 from 90.00% to 93.62% and reduces the average warning round from 2.39 to 1.23. These results demonstrate the value of fine-grained SEIAR-D states and the additional benefits of modeling state transitions for accurate and timely risk prediction.

3. Non-zero ASR in Clean and Attack Attribution

First, we apply a Clean-start filtering mechanism. We exclude samples where agents have already accepted or supported the target error before injection. This ensures that the target error has not propagated at the initial stage and provides a clear starting point.

Second, Clean does not involve malicious error injection. However, LLMs may still naturally produce the same target error during later interactions. Thus, the non-zero ASR in Clean reflects the natural occurrence rate of the target error.

To distinguish natural errors from attack-induced errors, we use Clean as the baseline and measure the additional risk under attack conditions. Table 2 shows that Full increases ASR by 11.05, 13.54, 10.00, and 4.66 percentage points over Clean across the four models. These results indicate that attacks increase the risk of target-error consolidation. We will clarify the definition of Clean and the attack attribution method in the revised paper.

4. Empirical Evaluation of the Monotonicity Assumption

The monotonicity assumption in Theorem 3 is a conditional assumption. To examine its validity, we conduct a length-matched, paired hop-level experiment. We fix the task, receiving agent, interaction history, and experimental settings. We only change whether the message includes new supporting reasoning. The token length difference is kept within ±5%. For each model, we select 150 paired samples and repeat each condition five times.

Experimental results:

ModelI Acceptance (%)A Acceptance (%)Δ (pp)Negative Cases
MiniMax-M331.141.4+10.34/150
GPT-5.6-Luna35.145.8+10.72/150
DeepSeek-V4-Pro29.435.3+5.99/150
Gemini-2.5-Flash17.421.3+3.96/150

The results show that semantic amplification increases error acceptance by 3.9–10.7 percentage points across all four models. This provides empirical support for the monotonicity assumption in Theorem 3 at the aggregate level. However, a small number of negative cases are observed. In these cases, new supporting reasoning may expose factual errors or logical flaws, prompting agents to reject or correct the target error.

These findings are consistent with the positive amplification effect described in Theorem 3, while also highlighting its conditional nature. We will clarify this scope and include the additional analysis in the revised paper.

The author's response has not fully addressed my concerns. The premise of Theorem 1 assumes the existence of a reachable path; however, it is unclear how such a path can still exist when the agent is capable of correcting its own errors. Furthermore, the proposed method does not demonstrate a significant improvement over the baseline. Consequently, I will maintain my original rating.

Reviewer_sNAWOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This work investigates error propagation dynamics in LLM-based multi-agent systems and proposes a hierarchical SEIAR-D framework for systematic analysis. The framework defines six agent-level states to characterize agent error behaviors. The authors further bridge these micro-level agent states to macro system-level outcomes via rigorous analyses of relay error propagation, semantic amplification, population-level criticality, and error consolidation risks.

Critical strengths
  1. The research addresses a timely and practically significant problem. Exploring how local errors diffuse through multi-agent interactive processes is crucial for evaluating the reliability and robustness of LLM multi-agent systems, especially given that initially error-free agents may evolve into new sources of misleading information and amplify system-level biases.
  2. The proposed framework refines the characterization of agent error behaviors. By distinguishing error acceptance, semantic amplification, behavioral deviation, and state recovery, this work provides a finer-grained description of error dynamics than conventional binary correct/incorrect classifications, which lays a foundation for future diagnostic and analytical research on multi-agent system errors.
Critical weaknesses
  1. The main theorems lack substantive innovation for LLM multi-agent scenarios. Theorem 1 is a direct corollary of the chain rule (non-zero single-hop propagation probability guarantees non-zero full-path propagation probability). Theorem 2 conforms to the monotonicity property of event union probability. Theorem 4 adopts the standard spectral-radius criterion for linear branching processes, and Theorem 5 is a Bellman-style recursion built on explicit Markov property assumptions. Although mathematically sound, these results do not deliver new theoretical insights tailored to LLM multi-agent systems.
  2. The semantic amplification analysis relies on unproven assumptions of universal shared exposure and monotonic error acceptance, which are exactly the core hypotheses that require empirical verification. Moreover, the framework imposes a first-order Markov property on aggregated agent state counts. In practical multi-round LLM interactions, agent behaviors are contingent on message content, agent roles, interaction order, repeated exposure and full historical context. The paper provides no empirical tests to justify these critical assumptions.
  3. The Full and No-A comparison groups are constructed by removing newly generated supportive content, yet this manipulation inevitably alters message length, discourse structure, contextual information and agent interactive behaviors. As such, the observed performance differences cannot be solely attributed to semantic amplification. The experiment lacks rigorous control of confounding factors including exposure scope, interaction topology, message quantity and context length, without paired sample-level validation.
  4. All evaluations are restricted to BBH tasks, with a fixed interaction paradigm consisting of four discussion agents, one target agent and predefined malicious agents, as well as single-model system settings. The paper fails to explore the impacts of agent quantity, network topology, model heterogeneity, agent role settings, error types, error injection timing, interaction rounds and diverse task scenarios. Furthermore, the filtering of clean-start cases introduces selection bias, further restricting the generalization of the conclusions to realistic multi-agent systems.
Questions for authors
  1. What type of evaluator is adopted to generate SEIAR-D state labels (LLM-based evaluator, deterministic rule-based system, or human annotators)? Does the paper include human annotation validation, multi-evaluator cross-checking, inter-rater agreement quantification and label stability analysis?
  2. How does the work distinguish semantic amplification from simple repetition, paraphrasing, generic background knowledge supplementation and irrelevant text addition? Please provide concrete annotation examples and typical failure cases to clarify the definition.
  3. Do the Full and No-A conditions strictly maintain consistent interaction topology, round number, exposure content, message count, token length and other contextual variables? How do the authors eliminate confounding effects introduced by prompt design and intervention operations?
  4. Has the first-order Markov assumption been verified? How does the predictive performance of the proposed SEIAR-D compare against second-order history-dependent models, full-history models and binary propagation models?
1 comment
AuthorsOpenReview ↗

We sincerely thank the reviewer for the thoughtful comments and valuable suggestions. Your feedback has helped us identify important areas for improvement. We have conducted additional experiments, clarified key assumptions, and strengthened our theoretical analysis. We address each concern below.

1. Theoretical Novelty

To further clarify our theoretical contribution, we introduce a new proposition on how interactions between different error-carrier states affect sustained error propagation.

New Proposition: Cross-State Semantic Feedback Criticality

Consider the SEIAR-D next-generation matrix:

k_{II} & k_{IA}\\\\ k_{AI} & k_{AA} \end{pmatrix} \geq 0$$ where $k_{uv}$ denotes the expected number of new state-$u$ carriers generated by a state-$v$ carrier. When $k_{II}<1$ and $k_{AA}<1$, neither error acceptance (I) nor semantic amplification (A) can independently sustain propagation. However, the system becomes supercritical ($\rho(K)>1$) if and only if $$\boxed{ k_{IA}k_{AI}>(1-k_{II})(1-k_{AA}) }$$ **Theoretical Insight:** Even when I and A cannot independently sustain error propagation, the A→I→A cross-state feedback loop can drive the system into a supercritical regime. This reveals how error acceptance and newly generated supporting reasoning can reinforce each other and sustain error propagation in LLM-based multi-agent systems. We will include the complete proof in the revised paper. ### 2. Reliability of SEIAR-D State Annotation We use a Rule-first Hybrid Evaluator. Deterministic rules identify S, E, D, and R, while an LLM distinguishes I (error acceptance) from A (semantic amplification). System-level states are determined entirely by rules. We stratified 360 samples from four LLMs. Two human annotators labeled them independently and resolved disagreements through arbitration. Human agreement reached 92.2%, with Cohen's $\kappa=0.906$. Against the arbitrated labels, the evaluator achieved 93.3% accuracy and 93.3% Macro-F1. The F1 scores for S, E, I, A, D, and R were 98.3%, 96.7%, 89.2%, 90.0%, 88.3%, and 97.5%, respectively. We also evaluated 120 I/A samples three times independently. 115 samples received identical labels across all runs, yielding 95.8% stability. ### 3. Distinguishing Semantic Amplification from Other Behaviors We further clarify the definition of A (semantic amplification). An agent must accept the target error and generate new arguments that were absent from the previous context and provide substantive support for the error. Repetition, paraphrasing, general background information, and irrelevant additions are classified as I rather than A. Consider the false claim "The Sahara is the largest desert in the world": | Agent Output | Behavior | State | |---|---|---| | "The Sahara is the largest desert in the world." | Repetition | I | | "The world's largest desert is the Sahara." | Paraphrasing | I | | "The Sahara is the largest desert, and it is located in Africa." | Background information | I | | "Although Antarctica is larger, ice-covered regions should not count as deserts. Therefore, the Sahara is the largest." | New supporting reasoning | A | Typical misclassification cases include: (1) labeling paraphrased arguments as A; (2) treating general background knowledge as new supporting evidence; and (3) treating irrelevant additions as amplification. The correct label in all three cases is I. ### 4. Controlling Confounding Factors in Full and No-A Full and No-A follow the same interaction protocol. However, removing newly generated supporting reasoning may also change message length. To control for this factor, we introduce a Length-matched No-A setting. We keep the tasks, target errors, interaction topology, agent roles, discussion rounds, and message counts unchanged. We replace the removed reasoning with content that adds no new supporting arguments, keeping the token length difference within ±5%. We also conduct paired comparisons on the same samples. Compared with Length-matched No-A, Full achieves ASR improvements of 5.38, 4.55, 3.91, and 3.19 percentage points across the four models, respectively. These results show that newly generated supporting reasoning contributes to error propagation even after controlling for message length. ### 5. Markov Assumption and History Comparison The first-order Markov assumption is an approximation of the system's macro-state dynamics. It does not imply that LLM interactions are strictly memoryless. We further compare four methods: Binary Propagation, First-order SEIAR-D, Second-order, and Full-history. The average AUROC scores are 0.9109, 0.9895, 0.9936, and 0.9932, respectively. First-order SEIAR-D outperforms the binary model by 0.0786. Second-order and Full-history improve over First-order SEIAR-D by only 0.0041 and 0.0037, respectively. These results suggest that additional historical information provides only limited predictive gains. Our method achieves competitive predictive performance with lower complexity. We will include these additional comparative experiments in the revised paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.