GAMA: Graph-Augmented Multi-Source Attribution for LLM-Based Multi-Agent Systems
Abstract
Large language model (LLM)-based multi-agent systems solve complex tasks through collaboration, but errors can propagate across agents and jointly cause task failure. Existing failure-attribution methods typically identify a single responsible agent or error step, overlooking multiple error sources within one execution. We study multi-source failure attribution: identifying all agent-step pairs that introduce errors contributing to task failure. This is challenging because multiple errors and their effects can become intertwined, requiring both structural reasoning to trace error propagation and semantic reasoning to identify error sources. We propose GAMA, a Graph-Augmented Multi-Source Attribution framework that combines these two forms of reasoning. GAMA represents each execution as a temporal graph with dependency and same-agent relations. A graph encoder integrates upstream dependencies and downstream consequences to produce source-error scores, which guide an LLM to predict the final attribution set. We also introduce Whos & Whens, a benchmark with step-level multi-source annotations constructed through context-dependent error injection and failure validation. Experiments show that GAMA improves failure attribution across single- and multi-source benchmarks. On Whos & Whens, GAMA improves exact match by 7.58% over the best baseline across three LLM backbones. On CORRECT-Error, it improves exact-step accuracy over CORRECT by 18.23% on average across seven tasks. Code is available at https://anonymous.4open.science/r/GAMA-54FB.
Then back it, or bet against it.
Related papers
Open the market on this paper to see 7 more related papers.