REALMS-CG: Resilient Ensemble of Autonomous LLM Multi-agent with Spectral clustering for Code Generation

Large language model (LLM) multi-agent systems are often built on the assumption that decomposing a task across multiple agents improves reliability. However, when one of the intermediate agents produces an error, it can propagate across stages rather than being corrected. We study this as correlated failure: the propagation of error across successive agent stages. We introduce Resilient Ensemble Of Autonomous LLM Multi-agent with Spectral clustering for Code Generation (REALMS-CG), a heterogeneous multi-LLM code-generation framework that combines model diversity with structure-aware candidate selection. REALMS-CG is implemented as a fully local pipeline using five compact LLMs ranging from 8B to 20B parameters, requiring less than 16GB of memory each. REALMS-CG embeds each LLM output into a semantic similarity graph and applies spectral clustering to reduce redundant output before downstream processing. We evaluated five configurations; single agent, homogeneous multi-agent generation, a Mixture-of-Agents (MoA)-style approach, and two REALMS-CG variants on 378 MBPP+ and 342 LiveCodeBench tasks. Beyond Pass@1, we traced failures across multiple agent stages and analysed the scalability using peak stage-average token input and runtime duration. REALMS-CG with an LLM final aggregator achieved the highest Pass@1 scores of 91.50% on MBPP+ and 43.27% on LiveCodeBench.