Decentralized Multi-Agent Systems with Shared Context

Multi-agent systems (MAS) can scale large language model agents on long-horizon tasks by running agents in parallel, but existing designs waste much of this parallelism in bubbles: agent time spent waiting on others or redoing a peer's work. Independent agents rediscover the same dead ends, peer-communicating agents wait at synchronous rounds, and centralized orchestration leaves the main agent blocked on its sub-agents while progress is relayed, and sometimes lost, through its context. We propose Decentralized Language Models (DeLM), a MAS framework that squeezes out these bubbles by replacing the main agent with a shared context and a task queue. Agents asynchronously claim tasks, publish findings as they become available, and build on or correct one another's progress, with peer status visible to all. On long-horizon tasks from Terminal-Bench 4.0 and DeepSWE v1.1 and on SWE-bench Verified, DeLM is more accurate and faster than Codex, Claude Code, and AOrchestra in every setting, gaining up to 17.5 points accuracy over the strongest baseline while running up to 2.49x faster than the underlying harness. Ablations show these gains come from coordination: removing any single mechanism brings two agents back to roughly single-agent speed.