acceptodds
Under review as a conference paper at ICLR 2027

Topological Coherence for Self-Evolving Multi-Agent Systems

Abstract

Complex tasks inherently couple workflow structure, agent responsibility, collaboration, and memory access: task regions delimit responsibility and tool scope, cross-region dependencies give rise to handoffs, and ownership boundaries delimit private and selectively shared memory. Existing methods can jointly optimize agent and communication structures, yet such optimization does not by itself require responsibility, handoff, and memory boundaries to remain consistent with task dependencies. We term this requirement topological coherence. We introduce TOCOMAS, a Topology-Coherent Multi-Agent System. TOCOMAS grounds a task graph in tool interfaces, organizes compatible task nodes into reusable responsibility domains, and derives dependency-induced and profile-conditioned collaboration together with boundary-regulated memory visibility. During online self-evolution, TOCOMAS proposes coupled changes to agent, collaboration, and memory policies, retaining for subsequent tasks only candidates that satisfy structural constraints and improve evaluated reward. Across BBEH, WorkBench, SWE-Bench-Verified, and CoMemBench, TOCOMAS improves task success over baselines across backbones. CoMemBench also shows gains over the self-evolving baseline in verified progress, handoffs, and memory isolation. Code is available at https://anonymous.4open.science/r/ToCoMAS_opensource-463F.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
LLM-based Multi-Agent Systems, Self-Evolving Agents, Topological Coherence

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_9Yn8OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes ToCoMAS, which uses a shared task-and-tool topology to organize agent responsibilities, communication, and memory permissions. Task nodes are grouped into responsibility regions, and cross-region dependencies determine required handoffs. An online evolution process proposes structural changes and retains candidates that satisfy coherence constraints and improve reward. Experiments cover reasoning, workplace tool use, software repair, and collaborative memory workflows.

Critical strengths

The paper identifies a useful organizational problem: changing agent responsibilities can invalidate communication paths and memory permissions. Treating these changes together is a sensible design principle for evolving multi-agent systems.

The evaluation considers more than final task success. Verified node progress, accepted handoffs, and resistance to polluted context provide useful views of where coordination succeeds or fails. The reported improvements across several benchmarks and backbones are also encouraging.

Critical weaknesses

The source of the task graphs is insufficiently explained. ToCoMAS begins with an upstream-provided task graph, which supplies much of the structure needed to assign responsibility and derive handoffs. CoMemBench provides such graphs, but the paper does not clearly explain how they are obtained for BBEH, WorkBench, and SWE-Bench-Verified, or whether competing methods receive equivalent information. This is important for interpreting the gains. Several central implementation choices are also underspecified, including the whole-graph tool-grounding procedure, compatibility scoring, proposal generation, and acceptance margin.

The ablations do not isolate the proposed coherence principle. Removing a module also removes useful functionality, such as compatible task grouping, consumer-aware handoffs, or restricted memory visibility. The resulting performance drops support those components, but do not establish that coupling their updates is the reason for improvement. A more direct control would retain the same components and capabilities while comparing independent updates with coherence-constrained updates.

The evaluation protocol needs more detail before the results can be assessed reliably. The paper does not report benchmark sample counts, repeated-run uncertainty, or sensitivity to task order. Task order is particularly relevant because feedback from one task changes the organization used on later tasks. The cost analysis also pairs post-recheck progress with first-pass tokens for WorkBench, while using a separate cohort for CoMemBench. These choices make the efficiency comparison difficult to interpret. Costs and outcomes should be reported for the same attempts, including candidate evaluation and evolution overhead.

Questions for authors

How are task graphs generated for the three benchmarks without native workflow graphs, and what graph information is available to each baseline?

Can you compare coupled and independent structural updates while keeping task grouping, handoff validation, memory filtering, and search budgets otherwise comparable?

Please provide evaluated sample counts, repeated-run results, task-order controls, and cost measurements that include all evolution-related calls.

4 comments
AuthorsOpenReview ↗

Response to Reviewer 9Yn8

We thank the reviewer for recognizing the importance of maintaining valid communication and memory access when agent responsibilities change. We clarify the graph inputs and implementation, compare full ToCoMAS with three ablations, and explain the effects of decoupling through execution evidence. We also report repeated outcomes and costs including evolutionary search.

Q1. Graph construction, baseline inputs, and implementation details

Graph access. On CoMemBench, all methods receive the same released task nodes, dependencies, artifact types, and acceptance requirements, and their artifacts are assessed by the same verifiers. ToCoMAS derives responsibility regions, agent assignments, handoff contracts, and memory permissions from this shared input. On BBEH, WorkBench, and SWE-Bench-Verified, ToCoMAS generates its task graph, while baselines follow their native protocols. These benchmarks compare complete systems; CoMemBench supplies the comparison under shared graph information.

The planner reads the request, public environment information, and fixed domain vocabularies and dependency rules, without accessing answers, tests, or evaluators. It outputs typed nodes with descriptions, roles, and upstream inputs. Validation checks vocabulary membership, acyclicity, and a single terminal node. For the controlled CoMemBench planner study, predicted dependencies are mapped onto released executable nodes; on the other three benchmarks, the predicted nodes themselves are executed. CoMemBench reference graphs incur no construction calls. All planning calls on the other benchmarks are charged to ToCoMAS.

Tool grounding and region formation. In the CoMemBench grounding pipeline, a node's declared execution contract specifies its required operations, resources, capabilities, effects, and receipts. Candidate tool bundles must cover these requirements; read-only nodes exclude side-effecting tools. We enumerate bundles in increasing cardinality and retain candidates at the smallest feasible size. A width-32 beam search then selects reference bundles in topological order, considering lexical mismatch, surplus capabilities, and provider continuity. Artifact-type incompatibility is rejected before search, and incompatible shared-resource provider assignments are pruned. Thus, “minimal” concerns each node's reference bundle, not the entire authorized tool set or a claim of exact global optimality. Execution may additionally authorize compatible alternatives. The other benchmarks bind nodes to tools through their benchmark adapters.

In the ToCoMAS experiments, node-pair affinity combines tool overlap, input-output compatibility, dependency adjacency, and lexical similarity with weights 0.35, 0.25, 0.25, and 0.15. Tool and lexical overlap use Jaccard similarity; input-output compatibility takes the larger directional Jaccard, and direct non-revision adjacency is binary. Eligibility gates first check adjacency, isolation, and role/verification compatibility. Eligible pairs are processed in descending score order, with an organization-affinity threshold of 0.45. Each proposed single-linkage region merge must satisfy capacities of four nodes and eight tools, convexity, and an acyclic quotient. They do not constitute a general cross-node side-effect conflict solver.

Proposal and acceptance. Execution diagnosis guides retries, configuration changes, memory-conditioned retries, upstream re-execution, and structural split, move, or merge. Candidate selection requires structural validity and either of two improvement criteria: its scalar reward exceeds the parent's by more than 0.01, or it completes without a crash and increases the number of verified nodes without losing any previously verified node. These are alternative acceptance criteria within the same setup, not requirements that must both hold. Unaccepted changes are rolled back.

Across all four CoMemBench domains, node recall is 85.7% and Edge F1 is 60.8%. Alignment is one-to-one, requires matching output types and similarity of at least 0.45, and considers roles, descriptions, and identifiers. Edge F1 measures aligned dependency overlap, excluding revision edges; neither metric measures execution success.

Case: v15_family_0707_1c6ff89bea6c (mathematics). Exact graph agreement is not necessary for successful execution. In this task, the supplied graph retained nine of thirteen released dependencies, omitted four, and added one; all eight executable nodes and the task still passed. This separates reference-graph fidelity from executable organization: a different edge set can still support the required computation. Topological coherence concerns compatibility among task dependencies, responsibility assignments, handoffs, and memory permissions, rather than reproducing a particular graph shape.

AuthorsOpenReview ↗

2nd Response to Reviewer 9Yn8

The noisy-graph evolution and no-evolution conditions use identical perturbation instances, task sets, initialization, and first-execution budgets.

Table N1. CoMemBench with predicted or perturbed dependencies (%).

Fixed denotes a single execution without evolution; Round 0 and Final denote first execution and post-adaptation outcomes with online evolution. Perturbation drops 25% of dependency edges and adds up to 10% type-compatible spurious edges, retaining executable nodes and acyclicity.

Setting              SR            VNCR          VHS
Released / fixed     10.35 ± 1.03  39.97 ± 1.25  47.29 ± 1.28
Predicted / fixed    9.24 ± 0.63   37.11 ± 0.74  39.26 ± 1.23
Predicted / Round 0  10.66 ± 1.01  37.50 ± 0.45  47.16 ± 0.97
Predicted / Final    12.59 ± 0.85  42.58 ± 0.71  46.79 ± 0.34
Perturbed / fixed    8.00          40.67         42.91
Perturbed / Round 0  10.65         46.97        49.47

ToCoMAS derives responsibility regions and interfaces from the supplied dependencies, while verified artifact delivery constrains error propagation. Evolution improves first-attempt performance under both predicted and perturbed dependencies; Final additionally captures current-task recovery. These results indicate useful adaptation under imperfect dependencies, rather than exact reconstruction of the reference graph.

Cases: local recovery under predicted dependencies. A targeted retry increased verified nodes from nine to eleven in code task v15_family_0335_0a440df98d99; configuration changes increased them from four to six in stateful task v15_family_0073_1e248b8fd711. These traces show partial progress recovery within the dependency-based organization, without reconstructing the reference graph.

Because dependencies alter eligible handoffs, VHS characterizes coordination within each graph setting; SR and canonical-node VNCR provide the primary cross-graph comparison.

Q2. Coupled versus independent structural updates

We supplement the component ablations with a decoupled structural-update control that retains structural operations, handoff validation, and memory filtering, but does not reconcile handoff interfaces and existing memory ownership after an edit. The full and decoupled configurations share the same initial organization, capabilities, validators, and search budget; only post-edit reconciliation differs. Two additional ablations remove structural operators or memory-conditioned retry. Table N2 compares their post-adaptation performance with full ToCoMAS; all rows report Final outcomes, separately from the first-attempt results in Q3.

Table N2. Full ToCoMAS and three ablations on CoMemBench (Final, %).

Configuration                     SR            VNCR          VHS
Full ToCoMAS                      15.79 ± 1.24  48.72 ± 0.47  55.32 ± 0.31
Decoupled structural updates      14.14         48.00         54.52
Without structural operators      13.50         47.48         53.98
Without memory-conditioned retry  12.00         47.57         53.26

Full-model values are means ± sample standard deviations; ablations are point estimates.

Full ToCoMAS exceeds all three ablations. Decoupling tests whether interfaces and memory ownership must track structural edits; removing structural operators tests adaptation beyond retries; removing memory-conditioned retry tests memory-assisted recovery. The latter produces the largest SR reduction despite a smaller VNCR gap, consistent with critical unresolved nodes blocking terminal completion.

Execution evidence makes the effect of decoupling concrete. In the 800-task evaluation, 408 tasks exercised at least one structural-edit round. The records contain 184 cross-region handoff-drop events caused by interfaces not being rebuilt and 28 accepted structural edits.

Case: django-11299. Both configurations apply the same split, making n01 → n02 a cross-agent dependency. ToCoMAS rebuilds the interface and delivers the artifact; the decoupled variant retains the old interface, leaving n02 without its required input and exhausting the budget through subsequent missing-input failures. The dependency is unchanged, but its responsibility boundary changes: this illustrates why structural edits require corresponding handoff updates. The ablation also tests reconciliation of memory ownership.

AuthorsOpenReview ↗

3rd Response to Reviewer 9Yn8

Q3. Evaluation size, ordering, repeated results, and total cost

Evaluation protocol. The main evaluation covers 4,520 BBEH tasks, 690 WorkBench tasks, 500 SWE-Bench-Verified issues, and 800 CoMemBench tasks. Its four domains contain 200 tasks each; the matched structural-update evaluation covers the full set. The online experiments use a score-before-update protocol: each task's Round 0 score is fixed before its own candidate evaluations, and retained updates become available to later tasks. Final is reported separately to show additional within-task adaptation. Repeated runs use the same task set in different shuffled orders.

For each ToCoMAS task, all evolutionary candidates share one additional step budget equal to the domain-level full-graph budget; it is not renewed per candidate. Rejected candidates also consume this budget. Tasks read the available pool state at launch; with up to four concurrent tasks per domain-fold stream, they use updates committed before that snapshot. In the EvoMAS evolution comparison, evaluator pass/fail feedback replaces the default LLM judge, so terminal correctness feedback is not exclusive to ToCoMAS. EvoMAS follows the same score-before-update protocol.

Table N3. Repeated CoMemBench outcomes with Qwen3.8-27B (%).

Setting           SR            VNCR          VHS
No evolution      10.35 ± 1.03  39.97 ± 1.25  47.29 ± 1.28
Online / Round 0  13.75 ± 1.21  42.04 ± 1.31  49.39 ± 1.73
Online / Final    15.79 ± 1.24  48.72 ± 0.47  55.32 ± 0.31

Table N4. Repeated general-task outcomes with Qwen3.8-27B (%).

Setting           WorkBench SR  BBEH accuracy
No evolution      73.66         50.34
Online / Round 0  78.83 ± 1.26  54.89 ± 0.99
Online / Final    80.42 ± 0.77  57.31 ± 1.08

Values with uncertainty are means ± sample standard deviations: four runs for CoMemBench online evolution, three for its non-evolving reference, and three for WorkBench and BBEH online evolution. The main table uses single runs for all methods; repeats were added in response to the review. The online configuration has higher mean first-attempt performance than the non-evolving reference, and Final shows further within-task gains. The BBEH rows compare ToCoMAS configurations on the same item set.

Online total costs include all model calls: planning, execution, coordination, memory, meta-model calls, and candidate evaluation, including calls incurred before rejection or rollback.

Clarifying “post-recheck.” This denotes offline rescoring of the original WorkBench execution trace, without additional model calls. Figure 5 therefore pairs progress and tokens from the same first execution; Tables N5–N6 additionally account for online evolution, including rejected candidates.

Table N5. WorkBench tokens per task (k = thousand).

Method / configuration        Mean   Median
Direct LLM Call               2.6k   2.4k
ADAS                          20.7k  16.8k
OMAC                          47.1k  44.1k
Peer Review                   268k   209k
SMoA                          313k   246k
ToCoMAS, first execution      32.0k  29.6k
ToCoMAS, online evolution     86.2k  60.1k
EvoMAS, frozen configuration  30.5k  18.0k
EvoMAS, online evolution      290k   160k

ToCoMAS's first-execution mean is close to that of frozen EvoMAS. Including evolution, its mean cost is 70.3% lower than online EvoMAS; Direct and ADAS remain cheaper.

Table N6. CoMemBench tokens per task with Qwen3.8-27B (k = thousand; M = million).

Method / configuration     Mean   Median
Direct LLM Call            44k    16k
ADAS                       565k   436k
EvoMAS, first execution    1.31M  977k
SMoA                       1.42M  858k
Peer Review                3.61M  2.27M
OMAC                       8.01M  5.54M
EvoMAS, online evolution   10.7M  8M
ToCoMAS, no evolution      343k   208k
ToCoMAS, online evolution  937k   784k

The CoMemBench online totals likewise include candidate evaluation. ToCoMAS's recorded expenditure is allocated as 38.2% to Round 0, 6.1% to accepted candidates, and 55.7% to rejected or rolled-back candidates. Thus, unsuccessful search is included rather than discarded from the cost.

Targeted recovery reuses verified artifacts and re-executes affected nodes. In code task v15_family_0335_0a440df98d99, re-executing five nodes increased verified progress from nine to eleven. This limits repeated work and provides a mechanism consistent with the observed cost pattern.

Manuscript revisions. We will clarify tool grounding, region formation, and candidate acceptance; add the matched coherence control; and report evaluation sizes, scoring denominators, repeated results, and total costs.

Dear Reviewer 9Yn8,

Could you please review the authors’ rebuttals and follow up with any further comments or questions?

Many thanks for your time and efforts.

Best regards, AC

Reviewer_WfyQOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces ToCoMAS, a topology-coherent framework for self-evolving multi-agent systems. The central idea is to derive agent responsibilities, collaboration interfaces, and memory-access boundaries from a shared task-and-tool topology rather than optimizing these structures independently. The framework grounds task nodes in tool capabilities, groups compatible nodes into responsibility regions, assigns these regions to capable agents, and derives dependency-aware handoffs and boundary-preserving memory policies. An online evolution mechanism proposes structural modifications and retains candidates that satisfy coherence constraints and improve evaluated reward.

The authors evaluate ToCoMAS on BBEH, WorkBench, SWE-Bench-Verified, and CoMemBench, using multiple LLM backbones and comparing against fixed, automatically designed, and self-evolving multi-agent systems. The reported results show consistent improvements in task performance. On Qwen3.8-27B, ToCoMAS achieves 54.78% BBEH accuracy, 77.68% WorkBench success, and 48.00% SWE-Bench-Verified resolution. CoMemBench further reports improvements in verified node completion, handoff success, and isolation from polluted context. Component ablations and parameter sensitivity experiments provide additional evidence for the proposed design.

Critical strengths

1. A coherent and practically motivated framework for multi-agent organization.

The strongest conceptual contribution is the explicit coupling of task responsibilities, collaboration interfaces, and memory-access boundaries through a common task topology. Unlike approaches that evolve agent configurations and communication structures without enforcing consistency among their relationships, ToCoMAS introduces structural constraints intended to preserve compatible ownership, handoffs, and information visibility during evolution. This provides a well-motivated organizational principle and addresses a practically relevant problem in increasingly complex multi-agent workflows.

2. Substantial empirical improvements supported by informative ablations.

ToCoMAS achieves strong results across reasoning, tool-use, and software-engineering benchmarks. The Qwen3.8-27B results improve over the strongest reported baselines by 8.26, 15.51, and 8.00 percentage points on BBEH, WorkBench, and SWE-Bench-Verified, respectively. The CoMemBench results are particularly notable, including improvements in verified handoffs and resistance to polluted context.

The ablations also provide useful evidence that the architectural components affect different aspects of performance. In particular, removing structure-adaptive orchestration substantially reduces verified handoff success, while removing boundary-preserving memory coordination reduces the isolation metric. These findings strengthen the case that the proposed organization has benefits beyond terminal task accuracy.

Critical weaknesses

1. The construction and availability of task topology are insufficiently specified, raising concerns about comparison fairness.

ToCoMAS assumes an upstream-provided task graph and grounds its nodes in a public tool graph. This representation is central to the method because it provides the dependencies and operational requirements from which agent responsibilities and handoff boundaries are derived. However, the paper does not sufficiently explain how these graphs are obtained for general benchmarks such as BBEH, WorkBench, and SWE-Bench-Verified. Although CoMemBench explicitly supplies canonical workflow graphs, the corresponding preprocessing pipeline for the other benchmarks is unclear. It is also insufficiently established whether competing methods receive equivalent structural information or whether the proposed method benefits from additional task-specific representations or preprocessing.

This matters because access to a well-structured task graph may itself yield considerable performance improvements, independently of the proposed coherence mechanism. Without clarifying and controlling this factor, it is difficult to determine how much of the reported advantage should be attributed specifically to topological coherence.

2. The experimental evidence does not fully isolate the effect of coherence constraints, and some evaluation details limit interpretability.

The component ablations establish that agent organization, adaptive handoffs, memory coordination, and evolution each contribute to performance. However, they do not directly establish whether enforcing coherence among otherwise comparable components outperforms an unconstrained organization with equivalent capabilities and computational budgets. Consequently, the central causal claim remains only partially supported.

The online evolution protocol also requires clarification. Candidate configurations are evaluated on the same task that generated their feedback, and accepted configurations influence subsequent tasks. The relationship between this adaptive selection process, benchmark evaluation order, and the optimization opportunities afforded to baselines is not sufficiently characterized.

Additionally, the cost analysis uses progress-conditioned cohorts and, for WorkBench, associates post-recheck progress with first-pass token usage. This makes it difficult to assess total end-to-end efficiency, including candidate evaluation overhead. The CoMemBench isolation metric is conditioned on clean-target success, meaning different methods may be evaluated on different eligible subsets. Without eligibility counts and uncertainty estimates, the very large isolation differences are difficult to interpret quantitatively. These limitations do not invalidate the reported improvements, but they reduce confidence that the experiments conclusively establish the claimed benefits of coherence-constrained self-evolution.

Questions for authors

1. Clarification — Task graph construction and baseline comparability.

How are the task graphs and tool-grounded execution requirements constructed for BBEH, WorkBench, and SWE-Bench-Verified? Are they generated automatically, supplied externally, or derived using benchmark-specific knowledge? Do all baselines receive equivalent structural information, and are any graph-construction costs included in the reported comparisons?

2. Clarification and additional evidence — Isolating topological coherence.

Could the authors clarify whether the reported gains can be separated from the benefits of topology-aware task organization itself? In particular, would an otherwise comparable variant that retains the same agent capabilities, task decomposition, and evolution budget but relaxes the cross-component coherence constraints achieve similar performance?

3. Clarification — Evolution protocol and evaluation reliability.

How are benchmark task order, random seeds, and candidate evaluation budgets controlled during online evolution? Does structural selection have access to benchmark-specific reward information unavailable to competing methods? Could the authors report variability across runs, the eligible denominators for CoMemBench's VHS and ICS metrics, and total token costs including evolutionary candidate evaluations?

5 comments
AuthorsOpenReview ↗

Response to Reviewer WfyQ

We thank the reviewer for recognizing topology-coherent organization and evaluation beyond task success. We address the concerns with protocol clarifications and supplementary evidence.

Q1. Task graph construction and baseline comparability

Graph inputs. On CoMemBench, all methods receive the same released nodes, dependencies, artifact types, and acceptance requirements for each task, and the same verifiers assess their node artifacts. ToCoMAS derives its responsibility regions, agent assignments, tool permissions, handoff contracts, and memory ownership from this shared graph. Baselines retain their own role assignment and execution policies.

On BBEH, WorkBench, and SWE-Bench-Verified, ToCoMAS uses planner-generated graphs; baselines follow their native protocols. The planner uses requests, public environment information, and fixed domain vocabularies and dependency rules, without answers, tests, or evaluators. It predicts node descriptions, artifact types, roles, and inputs, checking vocabulary membership, acyclicity, and a single terminal node. Benchmark adapters bind nodes to tools; CoMemBench grounding additionally checks capability coverage, artifact compatibility, and provider continuity. Thus, the three benchmarks compare complete systems, while CoMemBench compares shared-graph organization. Released graphs require no construction calls; planning calls elsewhere are included in ToCoMAS's cost.

Table W1 reports planner fidelity across all four CoMemBench domains.

Table W1. Planner fidelity on CoMemBench (%).

Evaluation scope  Node recall  Edge F1
All four domains  85.7         60.8

Node recall uses one-to-one maximum-similarity alignment with matching output types and a similarity threshold of 0.45; similarity incorporates roles, descriptions, and identifiers. Edge F1 measures aligned dependency overlap, excluding revision edges. These are structural-fidelity metrics, not execution-success measures.

Case: v15_family_0707_1c6ff89bea6c (mathematics). Exact graph agreement is not necessary for successful execution. In this task, the supplied graph retained nine of thirteen released dependencies, omitted four, and added one; all eight executable nodes and the task still passed. This separates reference-graph fidelity from executable organization: a different edge set can still support the required computation. Topological coherence concerns compatibility among task dependencies, responsibility assignments, handoffs, and memory permissions, rather than reproducing a particular graph shape.

For the controlled CoMemBench execution study, we retain canonical executable nodes and replace their dependencies with planner predictions. A separate perturbation removes 25% of dependency edges and adds up to 10% type-compatible spurious edges while preserving acyclicity. Both evolutionary settings in Table W2 use cross-task carry-over. Round 0 follows the paper's scoring protocol; Final reports additional within-task adaptation.

The noisy-graph evolution and no-evolution conditions use identical perturbation instances, task sets, initialization, and first-execution budgets.

Table W2. Execution with predicted and perturbed dependencies (%, mean ± sample standard deviation where available).

Fixed denotes a single execution without evolution; Round 0 and Final denote first execution and post-adaptation outcomes with online evolution. Perturbation drops 25% of dependency edges and adds up to 10% type-compatible spurious edges, retaining executable nodes and acyclicity.

Setting              SR            VNCR          VHS
Released / fixed     10.35 ± 1.03  39.97 ± 1.25  47.29 ± 1.28
Predicted / fixed    9.24 ± 0.63   37.11 ± 0.74  39.26 ± 1.23
Predicted / Round 0  10.66 ± 1.01  37.50 ± 0.45  47.16 ± 0.97
Predicted / Final    12.59 ± 0.85  42.58 ± 0.71  46.79 ± 0.34
Perturbed / fixed    8.00          40.67         42.91
Perturbed / Round 0  10.65        46.97         49.47

ToCoMAS derives responsibility regions and interfaces from the supplied dependencies, while verified artifact delivery constrains error propagation. Evolution improves first-attempt performance under both predicted and perturbed dependencies; Final additionally captures current-task recovery. These results indicate useful adaptation under imperfect dependencies, rather than exact reconstruction of the reference graph.

Cases: local recovery under predicted dependencies. A targeted retry increased verified nodes from nine to eleven in code task v15_family_0335_0a440df98d99; configuration changes increased them from four to six in stateful task v15_family_0073_1e248b8fd711. These traces show partial progress recovery within the dependency-based organization, without reconstructing the reference graph.

Because dependencies alter eligible handoffs, VHS characterizes coordination within each graph setting; SR and canonical-node VNCR provide the primary cross-graph comparison.

AuthorsOpenReview ↗

2nd Response to Reviewer WfyQ

Q2. Isolating topological coherence

We supplement the component ablations with a decoupled structural-update control that retains structural operations, handoff validation, and memory filtering, but does not reconcile handoff interfaces and existing memory ownership after an edit. The full and decoupled configurations share the same initial organization, capabilities, validators, and search budget; only post-edit reconciliation differs. Two additional ablations remove structural operators or memory-conditioned retry. Table W3 compares their post-adaptation performance with full ToCoMAS; all rows report Final outcomes, separately from the first-attempt results in Q3.

Table W3. Full ToCoMAS and three ablations on CoMemBench (Final, %).

Configuration                     SR            VNCR          VHS
Full ToCoMAS                      15.79 ± 1.24  48.72 ± 0.47  55.32 ± 0.31
Decoupled structural updates      14.14         48.00         54.52
Without structural operators      13.50         47.48         53.98
Without memory-conditioned retry  12.00         47.57         53.26

Full-model values are means ± sample standard deviations; ablations are point estimates.

Full ToCoMAS exceeds all three ablations. Decoupling tests whether interfaces and memory ownership must track structural edits; removing structural operators tests adaptation beyond retries; removing memory-conditioned retry tests memory-assisted recovery. The latter produces the largest SR reduction despite a smaller VNCR gap, consistent with critical unresolved nodes blocking terminal completion.

Execution evidence makes the effect of decoupling concrete. In the 800-task evaluation, 408 tasks exercised at least one structural-edit round. The records contain 184 cross-region handoff-drop events caused by interfaces not being rebuilt and 28 accepted structural edits.

Case: django-11299. Both configurations apply the same split, making n01 → n02 a cross-agent dependency. ToCoMAS rebuilds the interface and delivers the artifact; the decoupled variant retains the old interface, leaving n02 without its required input and exhausting the budget through subsequent missing-input failures. The dependency is unchanged, but its responsibility boundary changes: this illustrates why structural edits require corresponding handoff updates. The ablation also tests reconciliation of memory ownership.

Q3. Evolution protocol, uncertainty, denominators, and cost

Online scoring and candidate evaluation. The paper reports each ToCoMAS task's first execution (Round 0), using the system state available at launch. Its own candidate evaluations cannot revise this score; retained updates become available to subsequently launched tasks. Round 0 therefore tests the benefit of accumulated updates on later tasks, while supplementary Final outcomes measure adaptation on the current task. Repeated runs use the same task set with different shuffled orders.

For each ToCoMAS task, all evolutionary candidates share one additional step budget equal to the domain-level full-graph budget; it is not renewed per candidate. Rejected candidates also consume this budget. Unaccepted changes are rolled back. Each task reads an agent-pool snapshot before execution; completed updates are merged under a lock. With up to four concurrent tasks per domain–fold stream, tasks use updates committed before their snapshots.

Feedback and selection. EvoMAS and ToCoMAS use the same score-before-update protocol and official binary pass/fail feedback, which replaces EvoMAS's default LLM judge. ToCoMAS generates rule-based proposals from failure categories; EvoMAS's meta-model also uses execution trajectories, errors, and cost information to revise architectures.

Candidate selection requires structural validity and either of two improvement criteria: its scalar reward exceeds the parent's by more than 0.01, or it completes without a crash and increases the number of verified nodes without losing any previously verified node. These are alternative acceptance criteria within the same setup, not requirements that must both hold. The ToCoMAS execution model does not receive reference answers.

AuthorsOpenReview ↗

3rd Response to Reviewer WfyQ

Repeated evaluations. The main evaluation covers 4,520 BBEH tasks, 690 WorkBench tasks, 500 SWE-Bench-Verified issues, and 800 CoMemBench tasks. Tables W4a and W4b report the CoMemBench and planner-based WorkBench/BBEH results, respectively. These CoMemBench results characterize online adaptation with cross-task carry-over and its non-evolving reference. The main table uses single runs for all methods; repeats were added in response to the review.

Table W4a. CoMemBench repeated evaluations with Qwen3.8-27B (%, mean ± sample standard deviation).

Setting           SR            VNCR          VHS
Online / Round 0  13.75 ± 1.21  42.04 ± 1.31  49.39 ± 1.73
Online / Final    15.79 ± 1.24  48.72 ± 0.47  55.32 ± 0.31
No evolution      10.35 ± 1.03  39.97 ± 1.25  47.29 ± 1.28

Table W4b. General-task results with Qwen3.8-27B (%, mean ± sample standard deviation for repeated settings).

Setting           WorkBench SR  BBEH accuracy
No evolution      73.66         50.34
Online / Round 0  78.83 ± 1.26  54.89 ± 0.99
Online / Final    80.42 ± 0.77  57.31 ± 1.08

Across these benchmarks, first-attempt performance improves over the non-evolving reference, while Final captures additional within-task progress. The two stages make the role of accumulated state and current-task adaptation explicit.

Handoff performance and isolation targets. Tables W5a–W5b report the paper's Qwen3.8-27B VHS/ICS scores and their aggregate passed/eligible counts. VHS retains the paper's domain-averaged score. ICS is computed over paired targets that succeed in the clean condition, as defined in Appendix A.2. A polluted target succeeds when it still passes verification under the injected artifact; an explicit rejection message is not required.

Table W5a. Paper VHS scores and aggregate handoff counts.

Method           Passed/eligible  Paper VHS (%)
Direct LLM Call    778 / 2,255    34.50
Peer Review      1,826 / 5,472    33.37
SMoA             2,958 / 6,829    43.32
ADAS               766 / 2,762    27.73
OMAC             2,410 / 5,949    40.51
EvoMAS           1,705 / 5,549    30.73
ToCoMAS          2,603 / 5,104    51.00

Table W5b. ICS scores and eligible polluted-target counts.

Method           Passed/eligible  ICS (%)
Direct LLM Call  0 / 139          0.0
Peer Review      35 / 183         19.1
SMoA             28 / 277         10.1
ADAS             7 / 95           7.4
OMAC             35 / 202         17.3
EvoMAS           59 / 220         26.8
ToCoMAS          185 / 218        84.9

These counts make the conditioning explicit: ToCoMAS preserves success in 185 of 218 eligible polluted targets, compared with 59 of 220 for EvoMAS. Each rate describes its method's clean-success population. We complement this outcome measure with direct inspection of boundary enforcement using the same agents and tasks under boundary-preserving versus global memory visibility.

Table W6. Memory-boundary behavior under real artifact injection.

Memory               Decoys in context   Foreign private records
Boundary-preserving  0 / 2,586           0 / 13,523 contexts
Global visibility    202 / 2,575 (7.8%)  —

The exposure results explain the memory side of coherence: responsibility and sharing boundaries determine which records may enter an agent's context, rather than leaving the model to ignore contamination after retrieval. Boundary-preserving admission blocks inapplicable records, whereas global visibility admits some of them. Table W5b measures target success under pollution; Table W6 checks the corresponding access mechanism.

Online total costs include all model calls: planning, execution, coordination, memory, meta-model calls, and candidate evaluation, including calls incurred before rejection or rollback.

Reviewer_QUyCOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces ToCoMAS, a topology-coherent framework for self-evolving multi-agent systems. Its central idea is that task dependencies, agent responsibilities, collaboration interfaces, and memory visibility should be derived from a shared task-and-tool topology rather than optimized as independent components.

ToCoMAS consists of four main components: (1) topological agent organization, which groups compatible task nodes into reusable responsibility regions and assigns them to capable agents; (2) structure-adaptive collaborative orchestration, which derives mandatory handoffs from cross-region dependencies and permits additional profile-conditioned communication; (3) boundary-preserving memory coordination, which regulates memory retrieval and sharing according to responsibility ownership; and (4) coherence-constrained structural evolution, which proposes organizational changes and retains candidates that satisfy structural constraints and improve evaluated reward.

The authors evaluate ToCoMAS on BBEH, WorkBench, SWE-Bench-Verified, and CoMemBench, using multiple model backbones and comparing against fixed, automatically designed, and self-evolving multi-agent baselines. They report improvements in general task performance, verified workflow progress, handoff reliability, and resistance to irrelevant or polluted context. Ablation studies further examine the contributions of individual components.

Critical strengths
  1. A coherent architectural formulation of multi-agent organization, collaboration, and memory.

The paper identifies an important systems-level issue: jointly optimizing agent configurations does not necessarily ensure that task ownership, inter-agent handoffs, and memory permissions remain mutually compatible. Deriving these structures from a shared task topology provides a clear organizing principle, and the explicit treatment of ownership boundaries and dependency-induced handoffs makes the proposal more concrete than a general recommendation to jointly optimize multi-agent systems.

  1. Broad empirical evaluation with component-level ablations.

The evaluation covers reasoning, workplace tool use, software engineering, and collaborative memory. The reported improvements are accompanied by ablations that associate agent organization, collaboration orchestration, and memory coordination with distinct outcomes. In particular, CoMemBench evaluates verified handoffs and resistance to polluted context rather than relying exclusively on terminal task success. This helps assess the proposed framework's intended structural benefits.

Critical weaknesses
  1. The reported gains are not accompanied by sufficient evidence of statistical reliability.

The main results are presented primarily as point estimates, without clearly reporting repeated-run variability, confidence intervals, or statistical significance. This matters because multi-agent execution and online structural evolution can be sensitive to model stochasticity, task ordering, and candidate-selection outcomes. The substantial reported improvements are encouraging, but the current presentation does not establish how consistently they would be reproduced across independent runs.

  1. The generalization of the proposed organization mechanism beyond supplied task graphs remains insufficiently established.

ToCoMAS explicitly assumes an upstream-provided task graph and maps its existing nodes to executable tool interfaces without redecomposing the user query. Consequently, the evaluation primarily tests organization and evolution given an available task structure, rather than establishing the reliability of the full pipeline when task decomposition and dependency identification must also be performed automatically. Since the quality of the task graph directly determines the responsibility regions, required handoffs, and memory boundaries, this assumption materially affects the scope of the paper's claims about general-purpose multi-agent systems.

Questions for authors

Q1. Statistical reliability. How many independent runs were conducted for each method and backbone? Could the authors report standard deviations or confidence intervals for the main benchmark results, particularly WorkBench, SWE-Bench-Verified, and CoMemBench?

Q2. Evaluation protocol for online evolution. The paper states that a structural candidate is proposed from execution feedback and tested on the same task before acceptance, while accepted changes affect subsequently launched tasks. How are task ordering and candidate evaluation controlled? Are the reported results averaged across different task orders, and is there evidence that improvements transfer to unseen tasks rather than overfitting the task used to evaluate a candidate?

Q3. Dependence on task-graph quality. How sensitive is ToCoMAS to errors or missing dependencies in the upstream task graph? Can the authors provide an evaluation in which task graphs are automatically generated or deliberately perturbed, to establish whether the proposed coherence constraints remain effective under imperfect task decomposition?

4 comments
AuthorsOpenReview ↗

Response to Reviewer QUyC

We thank the reviewer for recognizing the coherent formulation of agent organization, collaboration, and memory, and the value of evaluating verified progress and handoffs beyond terminal success. We agree that the paper should make the statistical evidence, online evaluation protocol, and dependence on task-graph quality more explicit. We address these points below with repeated-run results, implementation details, and additional graph-quality experiments.

Q1. Statistical reliability. We supplement the main results with repeated evaluations using Qwen3.8-27B. Table R1 reports means and sample standard deviations under the paper's first-attempt scoring protocol, excluding improvements from each task's own adaptation. Q2 explains the distinction between Round 0 and Final.

Table R1. Repeated-run results with Qwen3.8-27B (%, mean ± deviation).

Benchmark   Configuration               Accuracy / SR  VNCR          VHS
WorkBench   ToCoMAS, online evolution   78.83 ± 1.26               
BBEH        ToCoMAS, online evolution   54.89 ± 0.99               
CoMemBench  ToCoMAS, online evolution   13.75 ± 1.21   42.04 ± 1.31  49.39 ± 1.73
CoMemBench  ToCoMAS, without evolution  10.35 ± 1.03   39.97 ± 1.25  47.29 ± 1.28

Table R1 summarizes three runs per setting, except for CoMemBench online evolution, which has four.

Online evolution improves first-attempt SR, VNCR, and VHS by 3.40, 2.07, and 2.10 percentage points over the non-evolving reference. Repeats use the same tasks in different shuffled orders; reported variability reflects ordering and stochastic execution. The main evaluation covers 4,520 BBEH tasks, 690 WorkBench tasks, 500 SWE-Bench-Verified issues, and 800 CoMemBench tasks. The main table reports single runs for all methods and backbones. In response to the review, we added repeated evaluations with Qwen3.8-27B on WorkBench, BBEH, and CoMemBench; SWE-Bench-Verified retains the original single-run result.

We prioritized repeated evaluations of ToCoMAS to assess the stability of the proposed method. Two additional full runs of the six baselines on WorkBench and CoMemBench alone would require approximately 40.3 billion tokens at the observed mean costs, using EvoMAS's online total without adding first-execution cost again. This is an estimate of additional computation, not incurred expenditure.

Q2. Online evolution, task ordering, and cross-task transfer. The paper reports Round 0: each task is scored using the system state available at launch, before its own evolutionary updates. Candidate evaluations cannot revise this score; retained updates become available to subsequently launched tasks. Round 0 therefore evaluates cross-task adaptation. We additionally report Final, which measures current-task recovery after candidate evaluation and includes additional execution.

Evolution proposes changes from execution diagnosis and selects candidates using verification outcomes, without supplying reference answers to the execution model. In the workflow experiments, proposals are generated by rules driven by failure categories. The terminal node contributes to verified progress like other nodes; whole-task pass/fail is not a separate condition in the node-progress selector.

Candidate selection requires structural validity and either of two improvement criteria: its scalar reward exceeds the parent's by more than 0.01, or it completes without a crash and increases the number of verified nodes without losing any previously verified node. These are alternative acceptance criteria within the same setup, not requirements that must both hold. For each ToCoMAS task, all evolutionary candidates share one additional step budget equal to the domain-level full-graph budget; it is not renewed per candidate. Rejected candidates also consume this budget. Unaccepted changes are rolled back. A non-evolving configuration executes once and is reported in Table R1.

Table R2. Supplementary paired outcomes: first attempt (Round 0) → post-evolution (Final), %, Qwen3.8-27B. Only Round 0 follows the paper's scoring protocol.

WorkBench

Run  SR
1    78.65 → 80.27
2    77.68 → 79.74
3    80.17 → 81.26

BBEH

Run  Accuracy
1    54.02 → 56.69
2    54.68 → 56.68
3    55.97 → 58.56

CoMemBench

Run  SR             VNCR           VHS
1    13.50 → 17.00  43.94 → 49.07  51.45 → 55.71
2    12.18 → 14.18  40.97 → 49.14  47.21 → 55.11
3    14.30 → 15.50  41.44 → 48.16  49.44 → 55.42
4    15.00 → 16.50  41.80 → 48.51  49.47 → 55.03

Each stream starts with an empty agent pool. Tasks read pool snapshots and merge completed updates under a lock; with up to four concurrent tasks per domain–fold stream, only updates committed before a task's snapshot are available to it. The folds share a learner prior and are not an isolated train/test split. Ordinary owner-private pool experience is available at first execution; evolution-specific repair-memory injection begins afterward.

AuthorsOpenReview ↗

2nd Response to Reviewer QUyC

Q3. Dependence on task-graph quality. We first clarify the distinction between the upstream planner and ToCoMAS's organizational mechanism. For BBEH, WorkBench, and SWE-Bench-Verified, the evaluation setup uses planner-predicted task graphs, rather than assuming that the benchmarks supply a canonical workflow. ToCoMAS takes the resulting graph as input and derives agent responsibilities, handoffs, and memory boundaries from it. Thus, the statement in Section 4 that ToCoMAS does not redecompose the query concerns the downstream organization stage; it does not require the upstream task graph to be manually supplied. The supplementary WorkBench and BBEH results in Tables R1–R2 evaluate this planner-based pipeline using the benchmarks' terminal metrics.

CoMemBench additionally provides reference workflow graphs for measuring planner fidelity and the effects of imperfect dependencies. Table R3 summarizes the planner evaluation across all four domains.

Table R3. Planner fidelity against released CoMemBench workflow graphs (%).

Evaluation scope              Node recall  Edge F1
CoMemBench, all four domains  85.7         60.8

Node recall uses one-to-one maximum-similarity alignment with matching output types and a similarity threshold of 0.45; similarity incorporates roles, descriptions, and identifiers. Edge F1 measures aligned dependency overlap, excluding revision edges. These are structural-fidelity metrics, not execution-success measures.

Case: v15_family_0707_1c6ff89bea6c (mathematics). Exact graph agreement is not necessary for successful execution. In this task, the supplied graph retained nine of thirteen released dependencies, omitted four, and added one; all eight executable nodes and the task still passed. This separates reference-graph fidelity from executable organization: a different edge set can still support the required computation. Topological coherence concerns compatibility among task dependencies, responsibility assignments, handoffs, and memory permissions, rather than reproducing a particular graph shape.

For this controlled execution study, ToCoMAS retains CoMemBench's executable task nodes but replaces their dependencies with planner predictions. We also perturb the released graph by removing 25% of dependency edges, subject to rounding, and adding up to 10% type-compatible spurious edges while preserving acyclicity. The perturbation affects the method-visible graph rather than replacing the benchmark's reference task nodes. This isolates sensitivity to dependency quality from changes in the available task operations.

The noisy-graph evolution and no-evolution conditions use identical perturbation instances, task sets, initialization, and first-execution budgets.

Table R4. CoMemBench performance under different task dependencies. Round 0 follows the paper's first-attempt scoring protocol; Final is a supplementary post-evolution diagnostic. Both evolutionary settings enable cross-task carry-over.

Fixed denotes a single execution without evolution; Round 0 and Final denote first execution and post-adaptation outcomes with online evolution.

Setting              SR            VNCR          VHS
Released / fixed     10.35 ± 1.03  39.97 ± 1.25  47.29 ± 1.28
Predicted / fixed    9.24 ± 0.63   37.11 ± 0.74  39.26 ± 1.23
Predicted / Round 0  10.66 ± 1.01  37.50 ± 0.45  47.16 ± 0.97
Predicted / Final    12.59 ± 0.85  42.58 ± 0.71  46.79 ± 0.34
Perturbed / fixed    8.00          40.67         42.91
Perturbed / Round 0  10.65         46.97         49.47

Entries with standard deviations summarize three runs; each noisy-graph setting has one run.

ToCoMAS derives responsibility regions and interfaces from the supplied dependencies, while verified artifact delivery constrains error propagation. Evolution improves first-attempt performance under both predicted and perturbed dependencies; Final additionally captures current-task recovery. These results indicate useful adaptation under imperfect dependencies, rather than exact reconstruction of the reference graph.

Cases: local recovery under predicted dependencies. A targeted retry increased verified nodes from nine to eleven in code task v15_family_0335_0a440df98d99; configuration changes increased them from four to six in stateful task v15_family_0073_1e248b8fd711. These traces show partial progress recovery within the dependency-based organization, without reconstructing the reference graph.

Higher VNCR need not imply higher SR: local progress can coexist with an unresolved critical handoff or terminal node.

Changing dependencies changes eligible handoffs, so SR and canonical-node VNCR provide the primary cross-graph comparison; VHS measures coordination within each graph setting.

Manuscript revisions. We will add repeated-run results, clarify online scoring and task ordering, and document graph construction and robustness to predicted and perturbed dependencies.

I appreciate the authors' substantial efforts and additional experiments. However, repeated evaluations remain limited in model coverage, and the perturbed-graph experiments lack sufficient statistical validation. Moreover, the evidence for the general applicability of topological coherence remains limited. Overall, my main concerns are not fully resolved.

Thank you for the follow-up. We have expanded the evaluation to address repeatability, graph quality, and the contribution of coherence. We completed three ToCoMAS runs each on BBEH and WorkBench and four on CoMemBench, using the same task sets with shuffled orders. We also added three no-evolution runs on CoMemBench and three-run comparisons with and without evolution under planner-inferred dependencies. Further evidence includes controlled dependency perturbations, decoupled-update and other ablations, memory-isolation checks, execution case studies, and online cost accounting that includes rejected candidates.

These experiments provide complementary evidence. Repeated runs assess stability; predicted dependencies test operation without reference edges; and decoupled updates directly test whether responsibility changes require compatible handoff and memory updates. Execution traces illustrate the resulting failure mechanism: retaining an outdated interface after a responsibility change can prevent required inputs from reaching their consumers. The main experiments additionally provide coverage across multiple backbones.

Extending repetition across the full evaluation matrix entails substantial computation. Completing three runs of the six Qwen baselines across all four benchmarks would require approximately 41.4 billion additional tokens. Completing the remaining Qwen evaluations, repeating the existing DeepSeek and Luna comparisons, and adding perturbation repetitions yields an estimated 109.6 billion additional tokens: 43.5 billion for self-hosted Qwen and approximately $8,800 in API-equivalent costs for DeepSeek and Luna. These estimates use reported per-task costs, proxy token budgets where measurements are unavailable, and current standard API rates with an assumed 80:20 input/output split. At an assumed effective throughput of 1,000 total tokens/s per A100, the Qwen workload corresponds to approximately 12,100 A100 GPU-hours; these are budget estimates, not measured expenditure or runtime.

We therefore prioritized repeated evaluations across task types and predicted dependencies alongside mechanism-focused controls. We will consolidate this evidence in the revision, distinguishing repeated-run findings from single-run perturbation observations and explaining how the controls test topological coherence.

Comments on the paper

3 comments
AuthorsOpenReview ↗

General Response to the Reviewers

We thank the reviewers for their constructive feedback on maintaining consistency among agent responsibilities, collaboration, and memory boundaries. Our responses clarify task-graph inputs and online evaluation, and provide additional experiments and execution evidence examining the contribution of topological coherence.

Task-graph inputs and comparison fairness. WfyQ Q1 and 9Yn8 Q1 clarify that CoMemBench supplies all methods with the same task graphs and evaluation requirements. ToCoMAS derives responsibilities, handoffs, and memory permissions from this shared input; baselines retain their own execution policies. On BBEH, WorkBench, and SWE-Bench-Verified, ToCoMAS uses planner-generated graphs, while baselines follow their native protocols. CoMemBench therefore tests organization under shared graph information, whereas the other benchmarks compare complete systems.

Execution with imperfect dependencies. QUyC Q3 reports 85.7% node recall and 60.8% Edge F1. With executable nodes fixed, online evolution improves first-attempt SR from 9.24% to 10.66% under predicted dependencies and from 8.00% to 10.65% under matched perturbations. Execution cases further show that successful completion need not reproduce every reference edge. This distinction motivates our focus on compatibility among dependencies, responsibilities, handoffs, and memory permissions, rather than graph similarity alone.

Isolating topological coherence. WfyQ Q2 and 9Yn8 Q2 compare full ToCoMAS with a control that retains structural operations, validators, and memory filtering but does not reconcile interfaces and memory ownership after edits. Under matched initial organization and search budgets, Final SR is 15.79% versus 14.14%. In django-11299, the same split creates a cross-agent dependency: rebuilding the interface delivers the required artifact, whereas retaining the old interface leaves the consumer without its input. The task dependency is unchanged, but its responsibility boundary changes, illustrating why structural adaptation requires corresponding updates to execution relations.

Online evaluation and reliability. QUyC Q1–Q2 and 9Yn8 Q3 clarify that the paper reports Round 0, scored before each task’s own updates; retained updates benefit subsequently launched tasks. Supplementary Final results measure current-task recovery. EvoMAS receives the same official pass/fail feedback and follows the same scoring protocol. Supplementary repeats on WorkBench, BBEH, and CoMemBench use the same task sets in different shuffled orders. On CoMemBench, mean first-attempt SR is 13.75 ± 1.21%, versus 10.35 ± 1.03% without evolution (mean ± sample standard deviation); the main table retains single-run results.

Memory isolation and total cost. WfyQ Q3 reports no injected decoys entering model context with boundary-preserving memory (0/2,586), versus 202/2,575 under global visibility. This connects responsibility-based access boundaries to preventing contamination before model execution. Including all calls and rejected candidates, mean online costs for ToCoMAS versus EvoMAS are 86.2k versus 290k tokens on WorkBench and 937k versus 10.7M on CoMemBench. Local re-execution and reuse of verified artifacts provide a mechanism consistent with these costs (WfyQ Q3; 9Yn8 Q3).

Together, these results support maintaining compatible responsibility, handoff, and memory relations as the system evolves. We will incorporate the protocol clarifications and supporting evidence into the revised manuscript.

Dear Reviewers,

Could you please review the authors’ rebuttals and follow up with any further comments or questions?

Many thanks for your time and efforts.

Best regards, AC

Dear Reviewers,

As the AC–reviewer discussion phase is about to begin, please highlight any disagreements or inconsistencies across the reviews, as well as any concerns that remain unresolved after the author rebuttal.

Please share your perspectives on these points and engage with one another to help us reach a well-supported assessment of the paper.

Thank you for your time and contributions.

Best regards, AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.