Information Flow Is Not Execution Authority: A Controlled Evaluation of LLM Multi-Agent Systems
Abstract
Safety gains in action-taking LLM multi-agent systems can arise from better proposal formation, lower execution coverage, or more effective selection among existing proposals, yet end-to-end metrics can conflate these sources. We present a controlled evaluation design to distinguish these mechanisms: we vary information flow under fixed tasks, evidence, and call budgets for proposal generation, compare authorization mechanisms on the same proposals, and separately examine candidate availability and authorization-contract validity. To enable these comparisons under explicit deployment criteria, we construct GateSim-T, a synthetic testbed comprising 192 tasks across seven model checkpoints and five information-flow configurations, yielding 6,720 paired decisions and 40,320 model calls. Within this testbed, we implement a separate authorization layer using a fixed scoped authorizer, DecisionGate, which checks proposals against an explicit contract before execution. To distinguish its selection effect from simply executing fewer proposals, we conduct a post-hoc comparison with random and confidence-based abstention, exactly matching execution counts within every model–flow stratum. At 12.26% overall execution coverage for each policy, adverse deployment on risk-eligible tasks is 0.12% under DecisionGate, compared with 8.57% for random and 7.80% for confidence abstention. A direct-rule control demonstrates that structured evidence can support effective deployment decisions without MAS-generated proposals in this testbed. In a pooled analysis of external Criteo and AgentDojo evaluations, DecisionGate occupies a lower-coverage, lower-regret operating point than a one-call LLM verifier under the specified loss. Further interventions show that explicit candidates restore useful action under trusted candidate authorization, while tested contract shifts change deployment risk. Overall, these results support treating proposal formation and execution authorization as distinct mechanisms: information flow shapes what is proposed, whereas scoped authorization determines what may be executed. Their combined effectiveness depends on candidate availability and the validity of the authorization contract.
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.