- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes TCP-MCP, a framework for jointly optimizing agent prompts and communication topologies in multi-agent systems (MAS). It employs evolutionary search with structure-aware crossover, mutation, and landscape-guided adaptive control, while considering task accuracy, token cost, and structural complexity through Pareto-based optimization. Experiments on four benchmarks demonstrate competitive performance against existing MAS baselines.
Critical strengths
-
Relevant problem and interesting approach. Jointly optimizing prompts and communication topologies is a meaningful problem in MAS design. Applying evolutionary algorithms to search over agent roles, prompts, and graph structures is an interesting and reasonable direction.
-
Multi-objective optimization. Considering accuracy, token cost, and structural complexity together is a valuable design choice. The Pareto-based formulation provides a practical way to explore performance-efficiency trade-offs.
Critical weaknesses
-
Limited methodological novelty and questionable necessity of the complex design. The proposed framework appears to largely combine established techniques, including LHS initialization, evolutionary crossover and mutation, NSGA-II, and heuristic adaptive control. The extensive terminology makes the method appear more sophisticated than its underlying mechanisms suggest. The paper does not sufficiently distinguish its core algorithmic novelty or justify the necessity of these components.
In particular, the claimed landscape guidance mainly relies on a one-time FDC estimation and subsequent pareto-front statistics. Meanwhile, prompt evolution appears largely tied to role mutation and prompt regeneration, raising questions about the extent of actual prompt-topology co-evolution. Furthermore, restricting communication structures to DAGs excludes common iterative interactions, such as feedback loops between workers and verifiers, limiting the expressiveness of the searched MAS architectures. -
Experimental comparisons and evaluation limitations. First, the paper overlooks a highly relevant prior work, MASS (Multi-Agent Design: Optimizing Agents with Better Prompts and Topologies, ICLR 2026), which also studies joint prompt-topology optimization. Its absence from both related work and baseline comparisons makes it difficult to assess the effectiveness of TCP-MCP relative to existing approach. Second, Table 1 shows that the relatively simple LLM-Debate baseline already achieves higher accuracy on MMLU and MMLU-Pro, while remaining competitive on GSM8K and WorkBench. Although Debate incurs considerably higher inference costs, these results raise questions about whether the elaborate evolutionary framework offers sufficient practical benefits. Moreover, Table 1 excludes the substantial offline search cost. Appendix F reports nearly 1 billion on WorkBench alone, the overall computational efficiency therefore deserves more careful analysis. Finally, MMLU and MMLU-Pro share a substantial number of underlying questions, which limits the diversity of the evaluation. More independent benchmarks would better support the generality of the method.
Questions for authors
- What concrete advantages does the proposed evolutionary approach offer over existing multi-stage prompt-topology optimization?
- How much do the landscape-guided mechanisms improve final performance over a standard evolutionary search baseline with the same search space and budget?
- Can the authors provide a more comprehensive efficiency comparison that accounts for offline search costs?
- Can the authors evaluate on other tasks, like coding or general reasoning, to better support the generality of the method?
5 comments
W1: Restriction to DAGs
Furthermore, restricting communication structures to DAGs excludes common iterative interactions, such as feedback loops between workers and verifiers, limiting the expressiveness of the searched MAS architectures.
We agree that the search space excludes loops with dynamic termination.
-
Fixed-round exchanges are expressible. A worker–verifier exchange with a fixed number of rounds can be unrolled into round-specific nodes, which forms a DAG.
-
Reason for the restriction. A DAG gives every candidate a defined execution order and a bounded cost, which keeps the structure search and the cost comparison consistent. Dynamically terminated loops require iteration state, stopping rules, and cumulative budget control as additional search variables.
We have added this limitation to Section 6.
W2 and Q1: Relationship to MASS
First, the paper overlooks a highly relevant prior work, MASS (Multi-Agent Design: Optimizing Agents with Better Prompts and Topologies, ICLR 2026), which also studies joint prompt-topology optimization. Its absence from both related work and baseline comparisons makes it difficult to assess the effectiveness of TCP-MCP relative to existing approach.
What concrete advantages does the proposed evolutionary approach offer over existing multi-stage prompt-topology optimization?
To answer directly: on MMLU the two methods reach the same accuracy and MASS uses 2.7 times fewer deployment tokens, on MMLU-Pro TCP-MCP is 3.44 points more accurate at 2.6 times the deployment cost of MASS, and on GSM8K and WorkBench TCP-MCP is 2.17 and 1.73 points more accurate while using 28.0% and 12.8% fewer deployment tokens.
-
Setup. We reimplemented the three stages of MASS (block-level prompt optimization, workflow topology optimization, and workflow-level prompt refinement) and evaluated it on the same held-out sets as TCP-MCP. Following Zhou et al. (2025), our implementation uses MIPRO for block-level and workflow-level prompt optimization after single-agent CoT warm-up, and rejection sampling over the Aggregate, Reflect, Debate, Summarize, and Executor building blocks under an agent-count budget cap for topology search. Aside from replacing the underlying backbone with DeepSeek-V3.2 and using the same evaluation splits as TCP-MCP, all hyperparameters match the official MASS implementation. The total search tokens are in a similar range across the four tasks (378M, 490M, 174.64M, and 946.83M for MASS, and 351.05M, 486.05M, 128.02M, and 998.19M for TCP-MCP), but our MASS accounting does not give per-stage candidate counts, so this is not a strictly budget-matched comparison.
-
Results (Table R8). On MMLU, MASS and TCP-MCP reach 89.94% and 89.96% at 2,323 and 6,305 tokens/query. On MMLU-Pro they reach 79.22% and 82.66% at 3,081 and 8,085 tokens/query. On GSM8K they reach 94.44% and 96.61% at 6,744 and 4,855 tokens/query, and on WorkBench they reach 75.58% and 77.31% at 17,543 and 15,289 tokens/query.
-
Difference in approach. MASS optimizes prompts, then topology, then prompts again in sequence. TCP-MCP updates topology and role–prompt configurations together within one population and selects on accuracy, token cost, and graph complexity. AB4 and AB5 (Table R4) test staged and alternating schedules inside our framework but do not replace the comparison with MASS.
We have added MASS to Section 2 and Table R8 to Appendix P, and revised the contribution statement in Section 1. We will release our MASS reproduction code.
Table R8: MASS and TCP-MCP
| Benchmark | Method | Optimization | Accuracy (%) | Deployment tokens/query | Search tokens |
|---|---|---|---|---|---|
| MMLU | TCP-MCP | Joint evolution of (G, P) | 89.96 | 6,305 | 351.05M |
| MMLU | MASS | Block prompts, topology, global prompts | 89.94 | 2,322.73 | 378M |
| MMLU-Pro | TCP-MCP | Joint evolution of (G, P) | 82.66 | 8,085 | 486.05M |
| MMLU-Pro | MASS | Block prompts, topology, global prompts | 79.22 | 3,081.19 | 490M |
| GSM8K | TCP-MCP | Joint evolution of (G, P) | 96.61 | 4,855 | 128.02M |
| GSM8K | MASS | Block prompts, topology, global prompts | 94.44 | 6,744 | 174.64M |
| WorkBench | TCP-MCP | Joint evolution of (G, P) | 77.31 | 15,289 | 998.19M |
| WorkBench | MASS | Block prompts, topology, global prompts | 75.58 | 17,543 | 946.83M |
TCP-MCP values are from Table 1 and Appendix F. MASS deployment tokens are 32,427,658 on MMLU and 37,072,826 on MMLU-Pro over 13,961 and 12,032 held-out queries.
W2: LLM-Debate
Second, Table 1 shows that the relatively simple LLM-Debate baseline already achieves higher accuracy on MMLU and MMLU-Pro, while remaining competitive on GSM8K and WorkBench. Although Debate incurs considerably higher inference costs, these results raise questions about whether the elaborate evolutionary framework offers sufficient practical benefits.
We agree that LLM-Debate is more accurate on MMLU and MMLU-Pro, by 0.88 and 1.01 points, and it uses about 4.6 and 5.7 times the deployment tokens of TCP-MCP on these tasks.
-
TCP-MCP is more accurate on GSM8K and WorkBench, where LLM-Debate uses about 4.1 and 5.7 times the deployment tokens.
-
The trade-off holds across Debate configurations (Appendix O, Table 18). Smaller Debate configurations cost less than TCP-MCP and are less accurate, and a six-agent, three-round configuration is more accurate at 5.6 times the cost.
We have stated this trade-off in Section 5.2.
W2 and Q3: Offline search cost
Moreover, Table 1 excludes the substantial offline search cost. Appendix F reports nearly 1 billion on WorkBench alone, the overall computational efficiency therefore deserves more careful analysis.
Can the authors provide a more comprehensive efficiency comparison that accounts for offline search costs?
To answer directly: including the one-time search, TCP-MCP becomes cheaper than LLM-Debate after 8,539–15,507 queries, and on WorkBench it breaks even against G-Designer and DyLAN after 23,519 and 28,598 tasks. On the three reasoning tasks, the Table 1 operating point uses more tokens per query than Self-Consistency, DyLAN, and G-Designer, whereas selecting lighter candidates from the same Pareto archive (Table R2) recovers the search cost whenever the selected candidate uses fewer tokens per query than the target baseline.
-
Search cost per task. The searches use 351M, 486M, 128M, and 998M tokens on MMLU, MMLU-Pro, GSM8K, and WorkBench (Appendix F, Table 7).
-
Break-even workloads (Table R3, see our response to Reviewer wSTn, W1 and Q1). On WorkBench, TCP-MCP breaks even against G-Designer and DyLAN after about 23,519 and 28,598 tasks while keeping a success-rate advantage of 12.50 and 14.04 points.
We have moved the per-task search costs and the break-even workloads into Section 5.2.
W2 and Q4: Benchmark coverage
Finally, MMLU and MMLU-Pro share a substantial number of underlying questions, which limits the diversity of the evaluation. More independent benchmarks would better support the generality of the method.
Can the authors evaluate on other tasks, like coding or general reasoning, to better support the generality of the method?
We agree on both points. We have not evaluated coding tasks, and MMLU and MMLU-Pro are not independent benchmarks.
-
WorkBench covers tool use, not code generation. Success is judged from the final environment state on 520 tasks. TCP-MCP reaches 77.31% against 64.81% for G-Designer and 63.27% for DyLAN, and EvoMAS reaches 57.24% with 16.8 times the deployment tokens (Appendix N).
-
MMLU and MMLU-Pro overlap. The shared questions also limit the transfer result between them in Table 2.
-
One search backbone. All searches use DeepSeek-V3.2. The GPT-4 results in Table 3 test the reuse of searched structures, not independent search on another backbone.
We have added these limits to Section 6.
We thank the reviewer for finding joint prompt–topology optimization a meaningful problem and the accuracy–cost–complexity formulation valuable. We follow the reviewer's numbering below.
W1: Novelty
We agree that LHS, crossover and mutation, NSGA-II, and heuristic adaptive control are established techniques. The contribution lies in the object being optimized and in how its parts are updated together.
One executable candidate. M = (G, P) encodes the communication graph and the node-level role–prompt configurations as one system that is executed and evaluated as a whole.
Coupled updates. Structural operators change what each node receives, and role mutation regenerates the prompt of the mutated node. Both kinds of change are selected in the same population.
System-level objectives. Accuracy, token cost, and graph complexity |V| + |E| are evaluated on the complete system.
The ablations show that the components are needed (Table 4). Removing crossover lowers MMLU-Pro accuracy by 2.51 points, removing mutation lowers WorkBench success by 8.27 points, fixing the operator rates lowers it by 4.23 points, and staged optimization (AB4) is 2.48 and 5.96 points lower on MMLU and WorkBench.
We have simplified the terminology in Sections 3 and 4 and now describe each operation directly.
W1 and Q2: Landscape guidance
We would like to clarify that landscape guidance also acts at runtime, not only through the one-time FDC estimate. Removing all of it (AB7) lowers performance across all four tasks, by 1.76, 6.45, 3.95, and 5.96 points on MMLU, MMLU-Pro, GSM8K, and WorkBench.
Runtime control uses landscape features. In addition to Pareto-front diagnostics, the controller adjusts topology and role mutation rates from diversity, neutrality, multimodality, and ruggedness features computed from recent populations (Appendix C.6).
AB7 removes both parts within the same framework. AB7 uses neutral initialization and fixed operator rates, with both the large-mutation trigger and targeted injection disabled. It keeps the rest of the TCP-MCP search, including the cost–complexity bins, the structural quota, and the elite archive. It therefore measures landscape guidance in the same search space and budget, but it is not a plain NSGA-II baseline.
Results (Table R4, see our response to Reviewer wSTn, W2 and Q2). Relative to Full, AB7 is 1.76, 6.45, 3.95, and 5.96 points lower on MMLU, MMLU-Pro, GSM8K, and WorkBench. FDC alone is evaluated over three complete runs in Table R6.
We have added AB7 to Section 5.4 and Appendix K. We have revised the abstract to describe landscape guidance as FDC-guided initialization together with runtime adaptive control and to report the effect of FDC only where it is significant.
W1: Extent of prompt evolution
We agree with this description. Prompt bodies change only through role mutation, which regenerates the prompt for the new role, and through inheritance when crossover moves a node into another graph.
No free-form prompt editing. The method does not rewrite the prompt text of every node in each generation. Crossover transfers subgraphs, nodes kept from the recipient keep their templates, and imported nodes bring the templates of the donor.
The effective prompt still depends on the topology. BuildPrompt adds the messages of each node's upstream neighbors to its stored template (Section 3.1), so a structural change alters what downstream nodes receive even when their templates are unchanged.
We have stated this scope in Section 4.2.