acceptodds
Under review as a conference paper at ICLR 2027

Decentralized Multi-Agent Systems with Shared Context

19 views

Abstract

Multi-agent systems (MAS) can scale large language model agents on long-horizon tasks by running agents in parallel, but existing designs waste much of this parallelism in bubbles: agent time spent waiting on others or redoing a peer's work. Independent agents rediscover the same dead ends, peer-communicating agents wait at synchronous rounds, and centralized orchestration leaves the main agent blocked on its sub-agents while progress is relayed, and sometimes lost, through its context. We propose Decentralized Language Models (DeLM), a MAS framework that squeezes out these bubbles by replacing the main agent with a shared context and a task queue. Agents asynchronously claim tasks, publish findings as they become available, and build on or correct one another's progress, with peer status visible to all. On long-horizon tasks from Terminal-Bench 4.0 and DeepSWE v1.1 and on SWE-bench Verified, DeLM is more accurate and faster than Codex, Claude Code, and AOrchestra in every setting, gaining up to 17.5 points accuracy over the strongest baseline while running up to 2.49x faster than the underlying harness. Ablations show these gains come from coordination: removing any single mechanism brings two agents back to roughly single-agent speed.

Primary area:
foundation or frontier models, including LLMs
Keywords:
multi-agent system, shared context, decentralized systems

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_cqYiOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

The paper proposes DeLM, a multi-agent framework in which parallel agents coordinate without a main orchestrating agent. Each agent has its own session and container. They share an append-only context and a task queue that enforces exclusive claims and shows each item's status.

The authors argue that existing multi-agent systems waste agent time in "bubbles": redundant work, waiting at synchronization barriers, and waiting on a main agent to relay results. They claim DeLM removes these bubbles.

Critical strengths

The problem of idle and redundant agent time in parallel agentic coding is practically relevant.

The paper reports latency and cost alongside accuracy. The execution timeline illustrations are informative.

Critical weaknesses

Main ones:

First of all, the major problem, I don't think the "decentralized" claim holds up. In line 91 the authors put blackboard-style systems under centralized orchestration, and in Appendix A the only difference they draw is that those systems have a control agent. But DeLM itself has one central task queue and one central shared context, which is basically a blackboard. Taking the controller out and having prompted agents do its job doesn't make the coordination decentralized. In practice it isn't leaderless either: the queue starts with "predefined, general preparation subtasks" (Fig. 4, Alg 1) with no explanation of where they come from, the prompt says "the first worker creates the initial queue" (line 1033), and when the queue runs out, "the agent that most recently completed a subtask" decides whether to add more (lines 180). That is a rotating central planner over central state, and I don't see how it really differs from the main-agent decomposition in Claude Code style orchestration.

Almost every experiment is DeLM against centralized systems, but there are basically no comparisons with other non-centralized approaches. The paper's own taxonomy (lines 80, Table 1) lists independent agents and multi-agent debate, yet neither shows up in the results. At the very least there should be a compute-matched baseline of n parallel ReAct agents with no communication and a clear rule for picking the final answer. Without that, the claim that DeLM is "both faster and more accurate … improving over the strongest baseline" (line 154) doesn't mean much, because the most obvious baseline is missing. The paper also brings up the straggler problem in debate-style systems (line 86) but never measures it statistically.

The comparison with the centralized baselines does not seem to be fair on resources. Each DeLM agent has "its own persistent session and workspace" (line 185) and "a separate container" (line 1031), while Claude Code/Codex main agents and their subagents all share one workspace. So DeLM gets n isolated environments, which removes file conflicts and resource contention and lets builds and tests run in parallel. The extra environment resources are never reported, and the cost numbers only count API spend. The authors should show this trade-off or give the baselines the same isolated setup.

DeLM seems to be "built on top of Codex and on top of Claude Code", so it isn't at the same abstraction level as Codex and Claude Code. Comparing it with a single copy of the harness mixes up the coordination mechanism with just running more copies, which contradicts the claim that "the only difference between them is how agents coordinate" (line 256). The fair comparison should be against other ways of coordinating multiple instances of the same harness.

The main results are questionable. Terminal-Bench 4.0 has 66 tasks, and the public leaderboards put GPT-6-Astra at about 58% and Claude Opus 5.5 at about 64%. On the authors' 10-task subset, the single-agent baselines already get 71.67% (Codex) and 81.67% (Claude Code). So the picked tasks look much easier than average even though they were chosen as "long-horizon." The tasks were also picked based on one baseline's latency (lines 234), DeepSWE is cut to 10 tasks too, and the std across three runs is 10-13 points. So cherry-picking can't be ruled out. The authors should run at least a much bigger pre-defined set and add significance tests. Also, Claude Code (subagent) dropping from 71.67% to 50.00% on DeepSWE (Table 3) is a huge drop and looks like a misconfigured baseline.

It looks like the acc gains mostly come from the prompt, not from coordination. In the Fig. 5 ablation, removing any single mechanism (even the whole shared context) bring speed back to about 1×, but acc barely moves. So coordination explains the speedup but not the roughly 10-point accuracy gain, which goes against the conclusion that "these gains come from coordination" (lines 479). The DeLM prompt is very long and full of engineering advice that has nothing to do with coordination (how to handle performance-constrained work, validation steps, "Derive necessary checks from the original task", measurement rules), while the baselines use default prompts. A single-agent baseline with the DeLM prompt minus the coordination parts is needed. Some of the prompt items, like "first-call path" and "timing measurements", look tailored to the performance and measurement tasks for Terminal Bench / Deep Swe. The paper doesn't say what tasks the prompt was tuned on and has no held-out set, so overfitting to the 10 test tasks is a real possibility. On top of that, the n=4 prompt is different from the n=2 one (lines 1204), which is also strange.

Also, the metric "Cost/Submission" makes the cost look lower than it actually is. The user only needs one solution in the end, and the n submissions all come from the same shared context, so they are highly correlated and not really n independent solutions. Dividing total cost by n (lines 262) artificially shrinks the cost. Cost/Task should be the number that matters, and by that measure DeLM usually costs more than the single-agent baseline. and when the text says DeLM is cheaper, it only compares against the subagent baselines.


And some other minor stuff:

Claude Code already has Agent Teams (https://code.claude.com/docs/en/agent-teams) a long time ago, where teammates share a task list with pending/in-progress/completed states and dependencies, claim unassigned tasks themselves (with file locking to avoid races), and message each other directly. That is very close to DeLM's idea: queue with exclusive claims, peer status, and shared findings, and it's available in the same Claude Code version the authors build on. The paper doesn't mention or compare with it, and Table 1 marks Claude Code as lacking peer-status awareness, which isn't accurate.

The name "Decentralized Language Models" is misleading. This is a multi-agent coordination framework built on top of existing agent harnesses, not a language model. Nothing is trained or changed at the model level.

Questions for authors
  1. It looks like the shared context can only grow by appending (Sec. 2.1, 4.4), so how does it stay under 15K tokens (Table 5)? Is that from the character limits in the prompt, from truncation, or just because the selected tasks are short? How does it grow with longer tasks and more agents, and what happens when it gets past what the model can handle well? Line 203 also says later reads "emphasize new publications". Does each agent actually read the whole context, and is the read cost in Table 5 calculated that way?

  2. How does DeLM perform if you keep increasing the number of agents to 8, 16, and beyond? At what point do queue contention, duplicated work, and the growing shared context start to hurt?

  3. Every agent submits its own solution, so how do you pick the one final answer for a task? If there is a selection step, how accurate is it?

  4. How exactly is accuracy computed? If each run scores 10 pass/fail tasks with one submission, per-run accuracy should move in 10% steps, and a three-run mean should be a multiple of 3.33%. Several reported numbers don't fit that. For example, Codex's 71.67 +/- 11.55 on TerminalBench only works with 5% steps (e.g., runs of 65/65/85), and some entries, such as Codex (subagent) at 71.67 +/- 11.69 and DeLM (n=2, Opus 5.5) at 93.33 +/- 3.33, can't be reproduced with any simple step size. Is there partial credit, and how is it assigned? For DeLM, is accuracy the average over the n submissions, or does a task count as solved if any submission passes? If it's the latter, the comparison is basically pass@n vs. pass@1? Please report the per-run scores for every method.

  5. For SWE-bench Verified, which subset did you use (all 500 or a sample)? Why switch to AOrchestra's harness here instead of Claude Code or Codex? And average latency there is only about 60–130 seconds, which doesn't fit the paper's focus on "long-horizon" tasks. What is this experiment meant to show?

13 comments
AuthorsOpenReview ↗

Author Response to Reviewer cqYi (1/10)

Thank you for your thorough review. We appreciate your recognition of the practical relevance of idle and redundant agent time in parallel agentic coding, and the value of reporting latency, cost, and accuracy alongside execution timelines.

We address your concerns below: (W1) whether DeLM is truly decentralized; (W2–W4) the missing non-centralized baselines, resource fairness, and the level at which DeLM is compared; (W5) task selection; (W6) the role of the prompt; and (W7) the cost metric, followed by your minor points and questions. To do so, we added seven new experiments and analyses: a centralized controller with the same workers (W1), compute-matched independent agents and a straggler analysis for debate (W2), Claude Code Agent Teams (W2), 42 additional tasks chosen by a fixed rule from public data (W5), baselines with a matched prompt (W6), and n=8n=8 and n=16n=16 agents (Q2).

Several of the new experiments (Agent Teams in W2, the 62 tasks in W5) run the baselines with a matched prompt: the DeLM prompt with its coordination instructions removed and all of its task-solving guidance kept verbatim. W6 describes how we built it.

We hope this addresses your concerns and would be grateful if you could consider updating your score in light of these improvements.

W1: The 'decentralized' claim

By decentralized, we mean decentralized control: no agent assigns the others' work or integrates their results, so no agent waits on a coordinator. We also respectfully argue that no system is purely decentralized, and that what matters is the form of coordination. DeLM's agents coordinate asynchronously. DeLM centralizes what agents store, not who decides: the board stores entries and the queue makes atomic claims, but neither decides which agent works on what or what enters a final answer.

Each agent reads the board directly, works from its own view of it, and decides for itself what to claim and when to submit. The board's only job is to keep these views in sync, and we show that using it as the synchronization primitive substantially lowers latency without hurting accuracy (see the centralized variant below). DeLM is also more accurate than the single-agent harness, and this holds when we control for the prompt (W6).

Our paper does not intend to discredit existing blackboard systems for using a centralized blackboard, but to emphasize an important point: “how” a blackboard is used for coordination and synchronization matters a lot more than what the community previously thought. This finding is one of our main novelties.

Existing systems synchronize more than they need to, which wastes agent time, and our ablations show how much each mechanism matters.

We call coordination centralized when one agent assigns the others' work and integrates their results, so that the others wait on it. DeLM has no such agent.

The three points you raise. None of them gives one agent authority over the others or makes the others wait:

  • Predefined preparation subtasks. These are the generic starting directions in the prompt, one per agent and identical for every task. No agent produces them.
  • "The first worker creates the initial queue." The first worker publishes an initial split while implementing its own part. The other agents start at once from their own starting directions. Published items are open: a peer claims them itself, can reserve a different scope instead, and can add items of its own. If the first worker ends before its handoff, another agent takes over.
  • Adding subtasks when the queue runs out. Any agent can add items at any time (§2.2); Fig. 4 shows the common case, in which the agent that has just finished a subtask does so. That agent blocks no one: the others keep implementing, integrating, or validating their own submissions, and there is no global plan-closure step. A main agent in Claude Code-style orchestration assigns the work and integrates the results. In DeLM, whoever adds items does neither: every agent claims for itself and finalizes its own submission.

(Continued in the next comment.)

AuthorsOpenReview ↗

Author Response to Reviewer cqYi (2/10)

(Continued from the previous comment) W1: The 'decentralized' claim

No agent waits on a coordinator. From the board's event log, we measured how long agents are blocked in read --wait, and split each team's agent-time into working, blocked, and finished (already submitted while a teammate is still working). Idle is the share of wall-clock time in which no agent is working:

Results are mean ± standard deviation over three repeats.

The first two columns are per agent; the next three split the team's total agent-time and sum to 100%. Results are mean ± standard deviation over three repeats.

MethodBlocked (% of own session)Blocked (min / agent)Working (% of team time)Blocked (% of team time)Finished (% of team time)Idle (% of wall-clock)
TerminalBench-4.0, GPT-6-Astra
DeLM (n=2n=2)1.57 ± 0.620.25 ± 0.1295.33 ± 3.431.55 ± 0.623.12 ± 2.970.00 ± 0.00
DeLM (n=4n=4)3.21 ± 0.770.45 ± 0.1693.19 ± 0.523.13 ± 0.753.68 ± 1.100.00 ± 0.00
TerminalBench-4.0, Opus 5.5
DeLM (n=2n=2)4.42 ± 2.162.61 ± 1.5784.30 ± 2.824.01 ± 1.8711.69 ± 0.940.33 ± 0.47
DeLM (n=4n=4)3.58 ± 2.342.00 ± 1.6678.17 ± 0.193.04 ± 2.1818.79 ± 2.070.00 ± 0.00
DeepSWE v1.1, GPT-6-Astra
DeLM (n=2n=2)1.43 ± 0.370.23 ± 0.0996.84 ± 0.961.42 ± 0.361.74 ± 0.650.00 ± 0.00
DeLM (n=4n=4)2.78 ± 0.300.36 ± 0.0293.33 ± 0.962.67 ± 0.274.00 ± 0.730.00 ± 0.00
DeepSWE v1.1, Opus 5.5
DeLM (n=2n=2)3.93 ± 0.851.18 ± 0.3087.28 ± 1.523.59 ± 0.689.13 ± 0.850.03 ± 0.06
DeLM (n=4n=4)3.20 ± 0.901.00 ± 0.2986.09 ± 1.742.89 ± 0.8011.02 ± 1.090.00 ± 0.00

DeLM agents spend 1.4–4.4% of their session blocked, and there is no working agent for at most 0.33% of the wall-clock time. For comparison, on the task in Fig. 3, the main agent of Claude Code's native subagents spends 73% of its time waiting on its subagents.

To further test how DeLM differs from main-agent orchestration, we built a centralized variant that keeps DeLM's worker prompt, shared context and expand/apply tools, and adds a controller that assigns the work, integrates the results, and returns one final submission. This is a shared board with a control agent. It uses the same number of agents as DeLM (n=4n=4): one controller and three workers. On TerminalBench-4.0 (Results are mean ± standard deviation over three repeats):

MethodAvg. Acc (%)Avg. Latency (min)Cost/TaskCost/Submission
GPT-6-Astra
Codex71.67 ± 11.5526.91 ± 0.91 (1.00×)\9.01±9.01 ± \\0.71\9.01±9.01 ± \\0.71
Centralized (n=4n=4)80.00 ± 10.0019.31 ± 1.62 (1.39×)\22.32±22.32 ± \\1.89\22.32±22.32 ± \\1.89
DeLM (n=4n=4)81.67 ± 12.3313.16 ± 1.43 (2.05×)\23.90±23.90 ± \\2.27\5.97±5.97 ± \\0.57
Claude Opus 5.5
Claude Code81.67 ± 2.89130.75 ± 6.27 (1.00×)\20.31±20.31 ± \\0.23\20.31±20.31 ± \\0.23
Centralized (n=4n=4)86.67 ± 5.7789.05 ± 7.73 (1.47×)\30.01±30.01 ± \\1.63\30.01±30.01 ± \\1.63
DeLM (n=4n=4)89.17 ± 5.2052.51 ± 4.43 (2.49×)\29.98±29.98 ± \\0.91\7.50±7.50 ± \\0.23

Accuracy differs by only 1.67 and 2.50 points, within the variation across repeats, but the centralized variant takes 46.7% and 69.6% longer at a similar Cost/Task. With everything else fixed, the controller alone accounts for the gap between a 1.39–1.47× and a 2.05–2.49× speedup. The centralized baselines we compare against therefore leave latency on the table by making agents wait on a controller.

AuthorsOpenReview ↗

Author Response to Reviewer cqYi (3/10)

W2: Comparisons with non-centralized approaches

We compare DeLM with independent agents and with Claude Code Agent Teams, and we measure the straggler cost of debate-style synchronous rounds.

Compute-matched independent agents. For every task and repeat, our baselines already launch two runs concurrently on the same server, and the two runs never communicate. Together they form a system of two independent agents, the same number of agents as DeLM (n=2n=2), and it spends more per task than DeLM in all four settings. We compare under three rules for returning one answer: avg@1 returns a random submission (the Avg. Acc of our main tables), Fastest returns the first submission and needs no selector, and pass@2 counts a task as solved if any submission passes, an oracle upper bound for any selection rule.

Latency under avg@1 is the mean agent latency; latency under Fastest is the time until the first agent finishes. Cost/Task sums the costs of all agents. Results are mean ± standard deviation over three repeats.

Methodavg@1 Acc (%)avg@1 Latency (min)Fastest Acc (%)Fastest Latency (min)pass@nn (%)Cost/Task
TerminalBench-4.0, GPT-6-Astra
Codex, 2 independent agents71.67 ± 11.5526.91 ± 0.9176.67 ± 20.8223.03 ± 0.6086.67 ± 11.55\18.02±18.02 ± \\1.41
DeLM (n=2n=2)85.00 ± 13.2317.47 ± 1.8286.67 ± 15.2816.83 ± 2.3186.67 ± 15.28\13.65±13.65 ± \\1.49
TerminalBench-4.0, Claude Opus 5.5
Claude Code, 2 independent agents81.67 ± 2.89130.75 ± 6.2780.00 ± 10.00111.65 ± 3.1490.00 ± 0.00\40.61±40.61 ± \\0.46
DeLM (n=2n=2)93.33 ± 5.7764.77 ± 5.0393.33 ± 5.7747.86 ± 3.6893.33 ± 5.77\17.91±17.91 ± \\1.17
DeepSWE v1.1, GPT-6-Astra
Codex, 2 independent agents88.33 ± 2.8916.57 ± 0.3990.00 ± 0.0014.38 ± 0.3890.00 ± 0.00\18.79±18.79 ± \\0.36
DeLM (n=2n=2)98.33 ± 2.8913.29 ± 0.9896.67 ± 5.7713.05 ± 0.89100.00 ± 0.00\16.83±16.83 ± \\1.65
DeepSWE v1.1, Claude Opus 5.5
Claude Code, 2 independent agents71.67 ± 2.8949.58 ± 3.1180.00 ± 10.0043.53 ± 3.1686.67 ± 5.77\24.01±24.01 ± \\1.20
DeLM (n=2n=2)88.33 ± 2.8932.98 ± 2.4790.00 ± 0.0029.22 ± 1.9890.00 ± 0.00\11.89±11.89 ± \\0.53

Under both avg@1 and Fastest, DeLM (n=2n=2) is more accurate than two independent agents in all four settings, returns its answer sooner, and costs less per task. Its avg@1 accuracy even exceeds the oracle pass@2 of independent agents in three of the four settings (93.33% vs. 90.00%, 98.33% vs. 90.00%, and 88.33% vs. 86.67%) and is within 1.67 points in the fourth (85.00% vs. 86.67%). The gain is therefore not explained by having more submissions: in three settings, even a perfect selector over independent agents would not reach DeLM's accuracy without any selection.

Straggler cost of synchronous rounds. We measured the straggler cost you mention on the same independent runs. In a synchronous round, an agent that finishes early waits for the slowest one. For agents finishing at times t1,…,tnt_1,\dots,t_n, a barrier leaves a fraction ∑i(max⁡jtj−ti)/(nmax⁡jtj)\sum_i (\max_j t_j - t_i)/(n \max_j t_j) of agent time idle, and the round takes max⁡jtj/tˉ\max_j t_j/\bar t times as long as the average agent:

Settingn=2 Idle (%)n=2 Debate speedup (upper bound)n=2 DeLM speedupn=4 Idle (%)n=4 Debate speedup (upper bound)n=4 DeLM speedup
TerminalBench-4.0, GPT-6-Astra11.440.88×1.54×18.920.81×2.05×
TerminalBench-4.0, Claude Opus 5.512.330.87×2.02×23.660.75×2.49×
DeepSWE v1.1, GPT-6-Astra11.140.88×1.25×19.110.80×1.34×
DeepSWE v1.1, Claude Opus 5.511.030.88×1.50×19.390.79×1.57×

Idle is the share of agent time spent waiting for the slowest agent in one synchronous round. The debate speedup bound is the average agent's latency divided by the round time, for one round alone; speedups are relative to a single agent.

A single synchronous round already leaves 11–12% of agent time idle with two agents and 19–24% with four. With Claude Opus 5.5 on TerminalBench-4.0, each of four agents waits 41.14 minutes on average. Debate pays this cost in every round.

Debate also targets a different goal. It was proposed to improve answer quality (Du et al., 2023; Liang et al., 2023), not to reduce latency: every agent produces a complete solution in every round, so a debate run can never finish sooner than its slowest agent's single attempt. The table above therefore bounds the speedup of debate over a single agent: at most 0.87–0.88× with two agents and 0.75–0.81× with four, for one round alone. DeLM (n=2) instead reaches 1.25–2.02×, because it has no rounds and therefore no barrier.

(Continued in the next comment.)

Reviewer_ByvrOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents DeLM. It's a decentralized multi-agent framework for long-horizon agentic tasks. Agents share a persistent context and task queue, reuse intermediate findings and files from peers. On selected long-running tasks, DeLM reports better accuracy and latency than the compared baselines. Ablations and trace analysis suggest that early sharing, peer-status awareness and persistent shared state are important for the coordination gains.

Critical strengths
  1. The ablation study demonstrates that merely having two parallel intelligent agents running together does not automatically lead to acceleration. Early sharing, peer-status awareness, and shared context are indeed all important. I think this is very persuasive.
  2. The paper also reports multiple complete indicators and conducts trajectory analysis, which makes the mechanism explanation a bit more credible.
Critical weaknesses
  1. The experiment did not cleanly isolate the contribution of decentralization itself. DeLM used significantly more explicit worker prompts and changed multiple coordination mechanisms at once; in contrast, the baseline of native subagents only added a brief instruction. The existing ablation studies can demonstrate the usefulness of these components, but cannot directly attribute the improvement to the controller topology itself.
  2. The conceptual novelty is also incremental. Shared memory/blackboard coordination is already an existing idea. The more obvious novelty of DeLM is combining asynchronous task claiming, peer-status visibility, and artifact reuse. The paper needs a clearer definition.
Questions for authors
  1. Could more experiments be conducted? For instance, keep the same worker prompt, shared artifacts/retrieval tools, concurrency budget, and validation rules, but let the central controller be responsible for allocating and integrating the work. This would make it easier to determine whether the gain comes from decentralization or from the entire DeLM design.

  2. How exactly are "Avg. Acc." in Tables 2 and 3 calculated? The paper states that there are 10 tasks and they are repeated 3 times, but a value like 88.33% doesn't seem to be directly obtained by averaging "a binary score for each task each time". Please explain how the score values used for each task (per-task score), the averaging order, and the standard deviation are calculated specifically.

3 comments
AuthorsOpenReview ↗

Author Response to Reviewer Byvr

Thank you for your thoughtful review and constructive feedback. We appreciate your recognition that our ablation and trajectory analysis show why each coordination mechanism matters.

Below, we first explain exactly how Avg. Acc is computed and report per-repeat scores. Then we address your concerns on: W1) isolating the effect of decentralization, with a new centralized-controller baseline that keeps DeLM’s worker prompt but gives one agent sole authority to allocate and integrate work; and W2) a precise definition of what is new in DeLM relative to existing shared-memory and blackboard systems. We hope this addresses your concerns and would be grateful if you could consider updating your score.

We first clarify the accuracy metric, since the other answers build on it.

Q2: “How exactly are Avg. Acc in Tables 2 and 3 calculated?”

Each submission is scored pass/fail according to the original benchmark’s evaluation criteria. For each of the 10 tasks in each repeat, we launch two independent runs of each baseline concurrently on the same server, matching the parallelism and server allocation of DeLM (n=2). Each baseline run produces one submission, and each DeLM worker likewise produces its own submission; all submissions are graded separately. Thus, each repeat contains 20 submissions for each baseline and DeLM (n=2), and 40 for DeLM (n=4). A repeat’s accuracy is the percentage of these submissions that pass. We report the mean accuracy and sample standard deviation across three repeats. For example, Codex on DeepSWE yields 17/20, 18/20, and 18/20 passing submissions (85%, 90%, and 90%), giving 88.33 ± 2.89.

We have added per-repeat results for every method in Tables 2 and 3 to Appendix E (Tables 7 and 8), together with the calculation details, and will clarify this computation in Section 3.

W1&Q1: The experiments do not isolate the contribution of decentralization itself

We ran the experiment you suggested on Terminal-Bench 4.0. The centralized variant uses the same four agent slots as DeLM (n=4): one controller and three workers. The workers keep DeLM’s worker prompt, shared context and artifact tools, and validation rules. We retain DeLM’s task-solving and validation instructions, shared-context access, and artifact tools, while replacing worker-side task claiming and individual finalization with controller-managed assignment and integration. Note that the centralized system returns a single final submission per run, whereas DeLM returns four separately graded worker submissions per run.

MethodAvg. Acc (%)Avg. LatencyCost/TaskCost/Submission
GPT-6-Astra
Codex71.67 ± 11.5526.91 ± 0.91 min (1.00×)\9.01±9.01 ± \\0.71\9.01±9.01 ± \\0.71
Centralized (n=4)80.00 ± 10.0019.31 ± 1.62 min (1.39×)\22.32±22.32 ± \\1.89\22.32±22.32 ± \\1.89
DeLM (n=4)81.67 ± 12.3313.16 ± 1.43 min (2.05×)\23.90±23.90 ± \\2.27\5.97±5.97 ± \\0.57
Claude Opus 5.5
Claude Code81.67 ± 2.89130.75 ± 6.27 min (1.00×)\20.31±20.31 ± \\0.23\20.31±20.31 ± \\0.23
Centralized (n=4)86.67 ± 5.7789.05 ± 7.73 min (1.47×)\30.01±30.01 ± \\1.63\30.01±30.01 ± \\1.63
DeLM (n=4)89.17 ± 5.2052.51 ± 4.43 min (2.49×)\29.98±29.98 ± \\0.91\7.50±7.50 ± \\0.23

The clearest observed difference is latency. Under the same four-agent budget, the centralized variant takes 46.7% longer with GPT-6-Astra (19.31 vs. 13.16 min) and 69.6% longer with Claude Opus 5.5 (89.05 vs. 52.51 min), at similar total cost per task. These results support a latency benefit of DeLM’s decentralized allocation relative to this centralized variant. Mean accuracy differs by only 1.67 and 2.50 percentage points; we do not interpret these results as establishing an accuracy advantage. Both variants retain opportunities for agents to check one another’s work and share findings from complementary exploration.

W2: The conceptual novelty is incremental; the paper needs a clearer definition

Blackboard architectures such as Hearsay-II and MetaGPT’s shared message pool still make agents wait. A blackboard’s control component decides which agent acts next, and recent LLM blackboards keep such a control agent. MetaGPT’s pool serves a fixed workflow in which each role acts on the output of the previous one.

Here is a definition: DeLM is a multi-agent system with decentralized control: no agent assigns the others’ work, relays it, or integrates their results, so no agent waits on a coordinator. Agents coordinate asynchronously: work is allocated by exclusive claims on a shared queue, any agent can add items, and progress is exchanged through an append-only shared context.

DeLM introduces three new mechanisms:

  1. A task queue that exposes claim owners and session liveness;
  2. Early, append-only publication;
  3. Compact entries kept separate from the files they reference.
Reviewer_ByvrOpenReview ↗

Response to Authors

Your response has addressed my concerns, and I like this paper very much. The current score remains unchanged. In addition, I suggest that you improve the visual presentation of all the figures; at present, the signs of AI generation in the figures are somewhat obvious, which may affect the reviewers’ impression.

AuthorsOpenReview ↗

Author Response to Reviewer Byvr

Thank you for reading our response and for the kind words about the paper. We are glad the new experiments addressed your concerns.

We also appreciate the suggestion on the figures. We will redraw them in the revision with a consistent style. Thank you again for your constructive feedback.

Reviewer_sU7dOpenReview ↗
Rating
2
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes DeLM, a multi-agent framework for agentic coding that designs waste parallelism in “bubbles” (redundant work, barrier waits, relay waits), and asynchronous coordination through shared state removes them. Experiments on 10 long-horizon tasks each from Terminal-Bench 4.0 and DeepSWE v1.1 (GPT-6-Astra, Claude Opus 5.5) and on SWE-bench Verified (Gemini 3 Flash) show gains in both accuracy and wall-clock latency over Codex, Claude Code (with and without native subagents), and AOrchestra.

Critical strengths
  • good contribution about bubble taxonomy (redundant / barrier / relay) borrowed from pipeline parallelism gives a clean lens on why three MAS families underperform
  • Simple design with pppend-only corrections rather than overwrites, demand-paging-style expand/apply, and exposing session liveness in the queue are each small, principled choices. The 80.3% re-import statistic (§4.4) is good evidence the append-only design actually gets used.
  • Ablation is the strongest part of the paper. Showing that removing any one mechanism drops two agents to ~1× speedup (Fig. 5).
  • Joint accuracy + latency + cost reporting, with the shared-context read cost quantified (Table 5), is more honest than most MAS papers. Candid limitations. Appendix E shows DeLM losing to RLM on OOLONG and explains why; the paper does not hide this.
Critical weaknesses
  • The main evaluation is 10 tasks per benchmark with 3 seeds, and the variance swamps many headline deltas. . No significance tests or paired comparisons are reported, yet the abstract claims DeLM wins “in every setting” and “up to 17.5 points.” The paper needs either the full benchmarks or a substantially larger long-horizon subset, with confidence intervals and per-task results.

  • Prompt asymmetry can be an issue. The subagent baselines receive a one-sentence instruction (“Use native subagents to help solve this task”). DeLM receives roughly four pages (Appendix G) that include substantial task-solving guidance unrelated to coordination. These could speed up and improve a single agent. A fair control is a single-agent baseline given the DeLM prompt minus board commands, and a subagent baseline given comparably engineered orchestration instructions.

  • The accuracy metric for n>1 is ambiguous, and the cost framing is favorable. Table captions say each agent’s submission is scored independently, but “Avg. Acc” is never defined as mean-over-submissions vs. any-correct. Cost/Submission divides by n, but a practitioner pays Cost/Task, which roughly doubles from n=2 to n=4. Please define the metric explicitly and report both pass@1-mean and pass@n, and add an independent-agents baseline at matched Cost/Task (it appears in Table 1 but in none of the result tables).

  • Scaling claims rest on n ∈ {2, 4}. Accuracy drops from n=2 to n=4 in two of four main-table settings, and shared-context read cost reaches 17.7% with GPT-6-Astra at n=4. The conclusion speaks of “scaling multi-agent systems,” but there is no scaling curve, no n=8, and no analysis of where coordination overhead overtakes gains.

  • Novelty positioning relative to blackboard and shared-pool systems is thin. Hearsay-II and Nii are cited, but tuple-space/work-stealing designs are not, and MetaGPT’s publish–subscribe shared message pool is arguably the closest LLM-era precedent yet appears in Table 1 only implicitly. The genuinely new pieces — queue with liveness, append-only corrections, expand/apply separation — should be stated as the delta rather than the whole architecture being presented as new.

  • SWE-bench setup needs clarification. The number of tasks is not stated; Claude Code is routed through OpenRouter to run Gemini 3 Flash, a nonstandard configuration (it scores below mini-SWE-agent); its $1.00 ± $0.00 cost is an estimate per the footnote, not a measurement; and DeLM is built on AOrchestra’s harness here rather than Codex/Claude Code, so it is unclear which DeLM components transfer.

Questions for authors

How is accuracy computed when n agents each submit — mean over n, or any-correct? What are both numbers? What is single-agent Codex/Claude Code accuracy and latency when given the DeLM prompt with board commands removed? Across all 240 runs, what fraction of agent time is spent in read --wait or equivalent for DeLM vs. baselines? Why does accuracy degrade from n=2 to n=4 on Terminal-Bench (both models) and DeepSWE (GPT-6-Astra)? Is it overlapping claims, integration conflicts, or context dilution? How many SWE-bench Verified instances were evaluated, and were they the full 500?

8 comments
AuthorsOpenReview ↗

Author Response to Reviewer sU7d (1/n)

Thank you for your thoughtful review and constructive feedback. We appreciate your recognition of the bubble taxonomy, the principled design choices, the ablation, and our candid reporting of cost and limitations.

Below, we address your concerns on: W1) ten tasks per benchmark; W2) prompt asymmetry; W3) the accuracy and cost metrics; W4) scaling beyond four agents; W5) novelty relative to blackboard systems; and W6) the SWE-bench Verified setup. We also answer each of your questions. We hope this fully addresses your concerns and would be grateful if you would consider updating your score based on these improvements.

We first clarify the accuracy metric, since the other answers build on it.

W3: Accuracy metric and cost framing

Avg. Acc is the mean over all submissions (avg@1): each agent's submission is graded pass/fail on its own. It is not any-correct.

In our original experiments, every baseline was already run as two concurrent, independent runs on the same server for each task and repeat, matching the parallelism and server allocation of DeLM (n=2n=2). These two runs form the independent-agents baseline at n=2n=2 that the reviewer asks for; its Cost/Task is twice that of a single run.

As requested, we also report pass@2 (any submission correct; an oracle upper bound). We additionally report Fastest (the first submission to finish), a rule that returns one answer without a selector.

Methodavg@1 Acc (%)avg@1 Latency (min)Fastest Acc (%)Fastest Latency (min)pass@2 (%)Cost/Task
TerminalBench-4.0, GPT-6-Astra
Codex, 2 independent agents71.67 ± 11.5526.91 ± 0.9176.67 ± 20.8223.03 ± 0.6086.67 ± 11.55\18.02±18.02 ± \\1.41
DeLM (n=2n=2)85.00 ± 13.2317.47 ± 1.8286.67 ± 15.2816.83 ± 2.3186.67 ± 15.28\13.65±13.65 ± \\1.49
TerminalBench-4.0, Claude Opus 5.5
Claude Code, 2 independent agents81.67 ± 2.89130.75 ± 6.2780.00 ± 10.00111.65 ± 3.1490.00 ± 0.00\40.61±40.61 ± \\0.46
DeLM (n=2n=2)93.33 ± 5.7764.77 ± 5.0393.33 ± 5.7747.86 ± 3.6893.33 ± 5.77\17.91±17.91 ± \\1.17
DeepSWE v1.1, GPT-6-Astra
Codex, 2 independent agents88.33 ± 2.8916.57 ± 0.3990.00 ± 0.0014.38 ± 0.3890.00 ± 0.00\18.79±18.79 ± \\0.36
DeLM (n=2n=2)98.33 ± 2.8913.29 ± 0.9896.67 ± 5.7713.05 ± 0.89100.00 ± 0.00\16.83±16.83 ± \\1.65
DeepSWE v1.1, Claude Opus 5.5
Claude Code, 2 independent agents71.67 ± 2.8949.58 ± 3.1180.00 ± 10.0043.53 ± 3.1686.67 ± 5.77\24.01±24.01 ± \\1.20
DeLM (n=2n=2)88.33 ± 2.8932.98 ± 2.4790.00 ± 0.0029.22 ± 1.9890.00 ± 0.00\11.89±11.89 ± \\0.53

From the table, we find:

  • Accuracy: DeLM is higher under avg@1 (+10.0 to +16.7 points) and under Fastest (+6.7 to +13.3 points) in all four settings, and at least as high under pass@2.
  • Latency: DeLM's Fastest answer arrives 1.10–2.33× sooner, e.g., 16.83 vs. 23.03 min with GPT-6-Astra and 47.86 vs. 111.65 min with Claude Opus 5.5 on TerminalBench-4.0.
  • Cost: DeLM (n=2n=2) costs 10–56% less per task than two independent runs in all four settings (\13.65vs.13.65 vs. \\18.02, \17.91vs.17.91 vs. \\40.61, \16.83vs.16.83 vs. \\18.79, \11.89vs.11.89 vs. \\24.01).
AuthorsOpenReview ↗

Author Response to Reviewer sU7d (2/n)

W2: Prompt asymmetry

The DeLM prompt contains two kinds of text: coordination instructions (board commands, reading and publishing on the board, queue claims) and general task-solving guidance. The matched prompt keeps every sentence of the task-solving guidance verbatim and in its original order, and removes only the coordination instructions. The baselines therefore receive exactly the task-solving guidance that DeLM agents receive.

We reran Codex (GPT-6-Astra) and Claude Code (Claude Opus 5.5), with and without subagents, on TerminalBench-4.0 with the matched prompt and all other settings unchanged. Parentheses give the change relative to the original prompt (percentage points for accuracy, percent for latency and Cost/Task). Results are mean ± standard deviation over three repeats.

MethodAvg. Acc (%)Avg. Latency (min)Cost/Task
GPT-6-Astra
Codex (original prompt)71.67 ± 11.5526.91 ± 0.91\9.01±9.01 ± \\0.71
Codex (matched prompt)71.67 ± 7.64 (+0.00 pp)24.33 ± 1.02 (−9.6%)\10.21±10.21 ± \\0.65 (+13.3%)
Codex (subagent, original prompt)71.67 ± 10.4120.04 ± 1.94\25.80±25.80 ± \\2.21
Codex (subagent, matched prompt)73.33 ± 7.64 (+1.67 pp)21.72 ± 2.56 (+8.4%)\26.35±26.35 ± \\2.08 (+2.1%)
DeLM (n=2)85.00 ± 13.2317.47 ± 1.82\13.65±13.65 ± \\1.49
Claude Opus 5.5
Claude Code (original prompt)81.67 ± 2.89130.75 ± 6.27\20.31±20.31 ± \\0.23
Claude Code (matched prompt)80.00 ± 5.00 (−1.67 pp)118.32 ± 7.31 (−9.5%)\18.04±18.04 ± \\1.93 (−11.2%)
Claude Code (subagent, original prompt)80.00 ± 5.00138.58 ± 10.17\30.88 ± \3.17
Claude Code (subagent, matched prompt)80.00 ± 0.00 (+0.00 pp)125.63 ± 8.16 (−9.3%)\29.01±29.01 ± \\2.34 (−6.1%)
DeLM (n=2)93.33 ± 5.7764.77 ± 5.03\17.91±17.91 ± \\1.17

The matched prompt changes baseline accuracy by at most 1.67 points, and DeLM (n=2) remains 11.7–13.3 points above every matched-prompt baseline with the same model. It changes baseline latency by −9.6% to +8.4%: three of the four baselines become about 9.5% faster and one becomes slower. DeLM (n=2) is still 28.2% faster than matched-prompt Codex and 45.3% faster than matched-prompt Claude Code. The task-solving guidance therefore does not explain DeLM's gains. Later, in our response to W1, all Codex baselines on the 62 tasks across both benchmarks also use the matched prompt.

In the experiment above, the subagent baselines' orchestration instruction remains the single sentence used in the original paper. For a subagent baseline with engineered orchestration, we built a centralized variant that keeps DeLM's worker prompt and shared context, and adds a controller that assigns the work and integrates one final submission. It uses the same number of agents as DeLM (n=4): one controller and three workers. Results on TerminalBench-4.0: (Parentheses give the speedup over the single-agent harness. Results are mean ± standard deviation over three repeats.)

MethodAvg. Acc (%)Avg. Latency (min)Cost/Task
GPT-6-Astra
Codex71.67 ± 11.5526.91 ± 0.91 (1.00×)\9.01±9.01 ± \\0.71
Centralized (n=4)80.00 ± 10.0019.31 ± 1.62 (1.39×)\22.32±22.32 ± \\1.89
DeLM (n=4)81.67 ± 12.3313.16 ± 1.43 (2.05×)\23.90±23.90 ± \\2.27
Claude Opus 5.5
Claude Code81.67 ± 2.89130.75 ± 6.27 (1.00×)\20.31±20.31 ± \\0.23
Centralized (n=4)86.67 ± 5.7789.05 ± 7.73 (1.47×)\30.01±30.01 ± \\1.63
DeLM (n=4)89.17 ± 5.2052.51 ± 4.43 (2.49×)\29.98±29.98 ± \\0.91

The two designs differ in who assigns the work and who produces the final submission. Accuracy differs by only 1.67 and 2.50 points, within the variation across repeats, but the centralized variant takes 46.7% and 69.6% longer at a similar Cost/Task. Removing the controller therefore accounts for the rest of the speedup: from 1.39× to 2.05× with GPT-6-Astra and from 1.47× to 2.49× with Claude Opus 5.5.

AuthorsOpenReview ↗

Author Response to Reviewer sU7d (3/n)

W1: Ten tasks per benchmark

We evaluated GPT-6-Astra on a larger task set chosen by a fixed rule. For each benchmark, we take the 30 longest-horizon tasks that do not require a GPU, ranked by the average latency of GPT-6-Astra in the public results (TerminalBench-4.0 leaderboard [1]; DeepSWE v1.1 trials [2]). The rule uses only public data and none of our own results, so it cannot favour DeLM. On TerminalBench-4.0, these 30 tasks include all 10 original ones; on DeepSWE v1.1, they include 8 of the 10 original tasks. This adds 20 and 22 new tasks, each run for three repeats.

all combines the new tasks with all original ones (30 and 32 tasks), and the revision lists every task. All Codex baselines here, including those on the original tasks, use the matched prompt from W2, so this experiment also extends the prompt control to 62 tasks. Parentheses give the speedup over Codex. Best mean values are bolded. Results are mean ± standard deviation over three repeats.

TerminalBench-4.0 (GPT-6-Astra):

MethodAvg. Acc (%)Avg. Latency (min)Cost/TaskCost/Submission
TerminalBench-4.0, all (30 tasks)
Codex50.00 ± 3.3321.46 ± 1.36 (1.00×)\7.36±7.36 ± \\0.32\7.36±7.36 ± \\0.32
Codex (subagent)50.00 ± 1.6719.32 ± 1.33 (1.11×)\16.79±16.79 ± \\0.98\16.79±16.79 ± \\0.98
DeLM (n=2n=2)55.00 ± 2.8914.58 ± 1.13 (1.47×)\11.59±11.59 ± \\0.83\5.80±5.80 ± \\0.42
DeLM (n=4n=4)54.17 ± 3.6312.37 ± 0.72 (1.73×)\21.20±21.20 ± \\1.09\5.30±5.30 ± \\0.27

DeepSWE v1.1 (GPT-6-Astra):

MethodAvg. Acc (%)Avg. Latency (min)Cost/TaskCost/Submission
DeepSWE v1.1, all (32 tasks)
Codex66.15 ± 3.9317.77 ± 1.19 (1.00×)\6.97±6.97 ± \\0.10\6.97±6.97 ± \\0.10
Codex (subagent)69.79 ± 3.6117.28 ± 1.21 (1.03×)\16.46±16.46 ± \\1.02\16.46±16.46 ± \\1.02
DeLM (n=2n=2)75.00 ± 1.5612.02 ± 0.99 (1.48×)\11.56±11.56 ± \\0.74\5.78±5.78 ± \\0.37
DeLM (n=4n=4)72.40 ± 1.9711.15 ± 0.81 (1.59×)\21.55±21.55 ± \\1.13\5.39±5.39 ± \\0.28

The speedup holds on every task set (1.47–1.73×), while native subagents are at most 1.11× faster at more than twice the Cost/Task. On accuracy, DeLM (n=2n=2) is 5.0 points higher than Codex on TerminalBench-4.0 (55.00% vs. 50.00%) and 8.9 points higher on DeepSWE v1.1 (75.00% vs. 66.15%). Codex and DeLM receive the same task-solving guidance here, so this gain does not come from the prompt.

W4: Scaling beyond n=4

We ran n=8n=8 and n=16n=16 with GPT-6-Astra on TerminalBench-4.0 (Results are mean ± standard deviation over three repeats):

MethodAvg. Acc (%)Avg. Latency (min)SpeedupCost/TaskShared context (K tokens)
Codex (n=1n=1)71.67 ± 11.5526.91 ± 0.911.00×\9.01±9.01 ± \\0.71–
DeLM (n=2n=2)85.00 ± 13.2317.47 ± 1.821.54×\13.65±13.65 ± \\1.497.31 ± 0.42
DeLM (n=4n=4)81.67 ± 12.3313.16 ± 1.432.05×\23.90±23.90 ± \\2.2712.88 ± 0.97
DeLM (n=8n=8)81.25 ± 8.7512.03 ± 1.292.24×\41.81±41.81 ± \\4.3315.93 ± 0.75
DeLM (n=16n=16)81.46 ± 7.1911.55 ± 1.072.33×\78.93±78.93 ± \\6.2118.01 ± 0.62

Where the gains stop. Latency gains diminish beyond n=4n=4 while Cost/Task grows roughly linearly: going from n=4n=4 to n=8n=8 lowers latency by 8.6% at 1.75× the cost, and from n=8n=8 to n=16n=16 by 4.0% at 1.89× the cost. n=2n=2–44 is therefore the practical range for these tasks. Accuracy stays at about 81% from n=4n=4 to n=16n=16.

Why they stop. Coordination overhead does not cause the plateau: at n=4n=4, agents spend under 4% of their session blocked on the board, and the shared context grows by only 2.1K tokens from n=8n=8 to n=16n=16. The limit is the dependency structure of the task. With total work WW and critical path LL, a run takes at least max⁡(W/n,L)\max(W/n, L). Once W/n<LW/n < L, the queue holds no independent subtask for the extra agents. They rarely wait; instead, each continues with the next step of its own solution and integrates its peers' results into it as they become available. This is also why the context grows so slowly. By the same bound, tasks with more independent subtasks (larger WW relative to LL) leave more room for additional agents.

[1] https://hub.harborframework.com/datasets/terminal-bench/terminal-bench/4?tab=leaderboard&leaderboard=4-0-0
[2] https://deepswe.datacurve.ai/data/v1.1/trials

Comments on the paper

3 comments
AuthorsOpenReview ↗

Author Response to Reviewer cqYi (Follow-up)

Thank you for the reply. The reviewer has mentioned:

Thank you for your response. A few of my concerns regarding experiment details have been resolved, but other concerns about unfair comparisons & incomplete experiments of existing baseline methods, limited novelty and insights, and reports about cost, still remain. I might rate the current paper somewhere between 1 and 1.5.

Our rebuttal addressed the concerns about unfair comparisons and incomplete baseline experiments with new experiments and analyses. We would appreciate it if you could engage with those specific points, which we summarize below.

1. Comparisons and baselines. The review asked for four things, and all four are now in our rebuttal:

  • compute-matched independent agents with a rule for picking the answer (W2): avg@1, Fastest, and pass@2, in all four settings;
  • other ways of coordinating multiple instances of the same harness (W4): Claude Code Agent Teams and a centralized controller;
  • the same isolated setup for baselines (W3): the centralized variant gives every worker its own container, and is still 46.7% and 69.6% slower;
  • a single agent with the DeLM prompt minus coordination (W6): accuracy changes by at most 1.67 points.

2. Novelty and insights.

As we already mentioned in W1:

"Our paper does not intend to discredit existing blackboard systems for using a centralized blackboard, but to emphasize an important point: “how” a blackboard is used for coordination and synchronization matters a lot more than what the community previously thought. This finding is one of our main novelties."

Also, as Reviewer Byvr mentioned, the more obvious novelty of our system is combining asynchronous task claiming, peer-status visibility, and artifact reuse. Our paper and ablations investigate the effects of these mechanisms, showing that they are crucial and sufficient. Prior systems overlook this and oversynchronize, which hurts performance. Anthropic acknowledged this in a recent blog post: "Latency: Multi-agent architectural complexity adds latency. One customer that evaluated their multi-agent harness against a single agent loop saw a roughly 2x faster runtime with the latter." (October 22).

3. Cost.

We already reported the cost in W7 and showed that, in a fair setting where any rule that draws on two independent runs pays for both, their Cost/Task is twice that of a single run. DeLM (n=2) costs 10–56% less per task in all four settings.

AuthorsOpenReview ↗

Follow-up on our response to Reviewer cqYi (Follow-up)

Dear Reviewer cqYi,

We really hope to see your response. Thank you.

Best regards,

Authors of DeLM

AuthorsOpenReview ↗

Discussion and Rebuttal Summary

Dear Area Chair and Reviewers,

We thank the reviewers for their reviews and would like to briefly summarize the progress made during the rebuttal and discussion period.

Reviewer Byvr replied that our response addressed their concerns, expressed strong support for the paper, and kept the positive rating. Reviewers sU7d and cqYi have not yet responded to our rebuttal (posted Nov 14) or to our follow-ups. We briefly summarize the primary points addressed:

  1. Isolating decentralization (sU7d, Byvr, cqYi). We added the requested centralized variant: the same worker prompt, shared context, tools, plus a controller that assigns and integrates the work. We found the centralized variant takes 46.7% and 69.6% longer at a similar cost.

  2. Prompt asymmetry (sU7d, Byvr, cqYi). We reran all baselines with a matched prompt: the DeLM prompt with its task-solving guidance kept verbatim and only the coordination instructions removed. DeLM remains at least 11.7 points more accurate and 19.6% faster than every matched-prompt baseline.

  3. Accuracy metrics (sU7d, Byvr, cqYi). We clarified that Avg. Acc is the mean over all submissions (avg@1), not any-correct, and added per-repeat pass counts for every method. We now also report Fastest and pass@n.

  4. Novelty relative to blackboard systems (sU7d, Byvr, cqYi). We pointed out: our method does not intend to discredit existing blackboard systems for using a centralized blackboard, but to emphasize an important point: "how" a blackboard is used for coordination and synchronization matters a lot more than what the community previously thought.

  5. Non-centralized baselines (cqYi, sU7d). We compared with compute-matched independent agents, Claude Code Agent Teams, and debate-style synchronous rounds. In every comparison, DeLM is at least as accurate while being faster and cheaper per task.

  6. Task set size and selection (cqYi, sU7d). We extended the evaluation to 62 tasks (30 on TerminalBench-4.0 and 32 on DeepSWE v1.1), chosen by a fixed rule that uses only public leaderboard data. DeLM is 1.47–1.73× faster than Codex on both benchmarks.

  7. Experimental setup (cqYi, sU7d). We clarified that baselines and DeLM use the same number of containers on the same host at the same time.

We appreciate the reviewers' feedback, which has helped us strengthen both the evaluation and presentation of the work.

Sincerely,

Authors of DeLM

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.