acceptodds
Under review as a conference paper at ICLR 2027

TacoMAS: Test-Time Co-Evolution for Dynamic Multi-Agent Systems

Abstract

Multi-agent systems (MAS) have become a promising paradigm for tackling complex tasks. Recent studies have investigated self-evolving MAS that automatically improve agent capabilities and communication structures. However, existing approaches either learn a topology that remains fixed during inference or adapt only one of topology and capability at test time. We investigate jointly adapting both dimensions at different time scales: agent capabilities should evolve rapidly to respond to emerging subtasks, whereas topology should change more gradually to maintain coordination stability. Inspired by a stylized replicator-mutator model, we propose TacoMas, a test-time co-evolution framework for dynamic MAS. TacoMas casts MAS inference as online graph adaptation, in which nodes correspond to agents with role-specific capabilities and edges represent their communication topology. During inference, a fast capability loop updates agent expertise based on trajectory-level feedback, while a slower meta-LLM-driven topology loop evolves the MAS through agent birth-death operations, including edge editing, agent addition, and agent removal. Experiments across four benchmarks show that TacoMas consistently outperforms nearly 20 multi-agent baselines, with an average improvement of 13.3 percentage points over the strongest baseline.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Multi-Agent System, Self-Evolution

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_FHwmOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes TacoMAS, a test-time co-evolution framework for multi-agent LLM systems. The key idea is to adapt two things jointly but at different time scales: a fast capability loop that refines agent memory/prompts each round based on trajectory feedback, and a slow topology loop driven by a meta-LLM that every K rounds performs bounded birth-death and edge edits. The design is motivated by a replicator–mutator surrogate model, with an explicit convergence bound. Experiments on four benchmarks against baselines show consistent gains.

Critical strengths
  1. The idea is well motivated and timely. Jointly evolving capability and topology at different time scales is a clean, principled contribution that fills a real gap between topology-only and capability-only test-time methods.
  2. The evaluation is honest and thorough. Four benchmarks, about 20 baselines, multiple backbones, and the authors are unusually upfront about their assumptions, confounds like unmatched compute, and what the theory does not actually show.
Critical weaknesses

1.The novelty feels a bit incremental. It basically combines two existing ingredients, CORAL-style memory refinement and SelfOrg-style rewiring, with a scheduling heuristic, and each piece on its own is fairly standard.

2.Compute is not controlled. The gains might largely come from extra LLM calls and judge rounds, and the equal-compute comparisons are weak or missing.

3.The theory is somewhat decorative. The surrogate theorem relies on unverifiable assumptions, applies to a made-up weight space instead of real text agents, and the authors repeatedly admit it does not establish anything useful.

4.There are reproducibility and rigor concerns. The paper leans on archived summaries, mixes protocols, uses unpaired comparisons and non-factorial sweeps, and lacks error bars, so several numbers cannot be fairly compared.

5.Scalability and cost are questionable. Wall time is about 86 minutes per instance on finance, the controller can eat up to 93% of the cost, and the single meta-controller is a bottleneck, which makes it impractical for real deployment.

Questions for authors

Please see the Weaknesses.

Reviewer_NgpKOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes TacoMAS, a test-time co-evolution framework that dynamically adapts both agent capabilities and communication topology in LLM-based multi-agent systems (MAS). TacoMAS introduces a two-time-scale adaptation mechanism: a fast loop that refines agent capabilities through trajectory-level feedback and a slower loop that restructures the agent network through agent addition, removal, and communication edge modifications. Inspired by evolutionary game theory, the framework aims to balance rapid capability improvement with stable multi-agent coordination. Experiments across four benchmarks covering financial reasoning, web browsing, planning, and workplace tasks demonstrate that TacoMAS outperforms 20 baselines, achieving an average improvement of 13.3% over the strongest baseline. Ablation studies further support the effectiveness of jointly adapting capabilities and topology at different frequencies, although comparisons are not compute-matched.

Critical strengths
  1. Propose jointly adapting agent capabilities and communication topology during inference
  2. Strong Empirical Performance. TacoMAS consistently outperforms 20 baselines across four diverse benchmarks
  3. Robustness Across Different Model Backbones. The framework demonstrates consistent improvements across multiple proprietary and open-weight models
Critical weaknesses
  1. Limited novelty and incremental compared to prior work. The novelty of jointly evolving agent capabilities and topology at test time is not sufficiently distinguished from prior work. MetaGen [1] already performs within-instance role prompt rewriting and communication topology adaptation based on execution feedback.
  2. The experimental configuration specifies a maximum of R=10 execution rounds and a topology update interval of K=2, which should permit at most five topology updates according to Algorithm 1. However, Table 2 reports a median of 10 slow updates for difficult Finance instances, exceeding this limit.
  3. A substantial fraction of tasks appear to require little or no topology evolution. Table 2 reports a median of zero slow topology updates for both feasible and impossible PlanCraft tasks, as well as most WorkBench task categories. Nevertheless, these benchmarks are among those where TacoMAS achieves substantial improvements over existing methods. This raises a concern about whether the improvements actually originate from the proposed topology co-evolution or primarily from iterative agent execution and capability refinement.
  4. The reported accuracy is sensitive to the choice of LLM evaluator. For example, Table 8 shows that TacoMAS's WorkBench accuracy drops from 0.853 with GPT-4o-mini to 0.496 with Claude Haiku 4.5, despite using the same evaluation rubric and stored predictions. The cross-judg analysis onlt compares TacoMAS against SelfOrg on Finance, without re-evaluating the strongest baselines across all four benchmarks. I am not sure whether the reported performance rankings and improvements are robust to the choice of evaluation model.

[1] Wang, Yimeng, et al. "MetaGen: Self-evolving roles and topologies for multi-agent LLM reasoning." arXiv preprint arXiv:2601.19290 (2026).

Questions for authors
  1. How generalizable are the evolved topologies across benchmarks? For instance, evolve both agent capabilities and topologies in one benchmark, and transfer only the topologies to the other benchmark and fix them while evolving only the agent capabilities.
  2. Can you report performance separately for instances with zero versus multiple topology updates and provide controlled comparisons under matched execution budgets?
  3. Can you provide a compute consumption comparison with baseline and prior methods? If the comparison is not compute-free (similar token), Can you provide a compute-fair comparison?
4 comments

Dear Reviewer,

Please reply to the authors' rebuttal.

Thanks!

AC.

Thank you for the rebuttal; while some of my concerns have been addressed, questions remain regarding the contribution of topology evolution to the gains on PlanCraft and WorkBench, the robustness of method rankings across evaluators, cross-benchmark topology transfer, and performance under matched compute budgets. Therefore I decide to maintain my current rating.

Thank you for your response and for taking the time to consider our clarifications. However, could you please specify which particular concerns remain unresolved? We would greatly appreciate more concrete feedback so that we can better understand and update our paper accordingly.

My remaining concerns are:

Source of improvements: The cases are informative, but results grouped by zero versus multiple topology updates and matched-budget fixed-graph comparisons on PlanCraft and WorkBench are still missing.

Evaluator robustness: The response does not establish whether TacoMAS retains its advantage over the strongest baselines across all four benchmarks under alternative judges.

Topology transfer: The discussion motivates task-specific adaptatio. However, I am still curious whether a topology evolved on other instances serve as an effective fixed structure for new tasks while retaining capability updates, or are instance-specific topology changes essential to the gains?

Compute fairness: The BrowseComp comparison is helpful, but similar average total token usage does not fully control for model choices, input/output token composition, or per-instance budgets.

Reviewer_CZeAOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes TacoMAS, a test-time dual-timescale co-evolution framework for multi-agent systems (MAS). The framework decouples the system's dynamic inference into two independent loops: the fast capability loop uses trajectory feedback in each execution round to update agents' specialized capabilities (i.e., prompts and contextual memory), while the slow topology loop is periodically driven by a Meta-LLM to reconfigure the communication network among agents through constrained node birth-death and edge editing operations. The theoretical motivation for this dual-loop architecture is inspired by a constrained replicator-mutator surrogate model. In the experimental section, the method is evaluated on four benchmark datasets covering financial analysis, web browsing, planning tasks, and workflow execution.

Critical strengths
  1. This paper proposes TacoMAS, a framework that co-evolves a multi-agent system (MAS) during inference through dual timescales (fast capability updates and slow topology updates).
  2. Experiments on four datasets spanning different dimensions, including Finance and BrowseComp, verify that the dynamic topology and capability adjustment strategy has a certain degree of environmental adaptability in task responses.
  3. The paper's motivation figure and the logical expression of the dual-loop architecture are relatively clear, and the mapping process from the theoretical surrogate model to the system implementation is reasonably readable.
Critical weaknesses
  1. The performance gains and computational costs in this paper are severely decoupled, making the comparative experiments unfair. The authors acknowledge in the paper that "tokens and dollar costs are not matched." TacoMAS actually introduces more rounds of inference, a stronger Meta-controller, and additional Judge calls; for example, it generates tens of thousands of API calls in the Finance task. This raises doubts as to whether the performance improvements in Table 1 come from the co-evolution mechanism or merely from brute-force compute stacking.
  2. The so-called dual-timescale co-evolution is essentially just an engineering combination of "modifying prompts based on feedback" plus "using an LLM to re-output the connection graph," which shows weak innovation.
  3. Ablation and sensitivity analyses are insufficient. For example, for the frequency parameter KK of slow topology updates and the structural editing budget BB (e.g., K=2K=2, B=(2,8)B=(2, 8)), the authors only provide a rough and incomplete grid search. For the key step-size parameter η\eta and its tolerance threshold ϵ\epsilon in the capability update formula wv,t+1∝wv,texp⁡(η(cv,t−cˉt))w_{v,t+1} \propto w_{v,t} \exp(\eta(c_{v,t} - \bar{c}_t)), there is no formula-level sensitivity analysis at all.
  4. The paper claims that its method can achieve dynamic "fast adaptation" and maintain stability during task inference, but in reality, the LLM-based Meta-controller introduces enormous inference latency and API overhead when performing topology updates. There is a contradiction between the motivation of "efficient collaboration" claimed in the paper and its extremely inefficient inference time.
Questions for authors
  1. The paper explicitly acknowledges in the experimental setup that "tokens and dollar costs are not matched," and the runtime statistics table shows that TacoMAS introduces a large number of high-latency Meta-controller and Judge API calls. How much of the reported performance improvement actually comes from the co-evolution mechanism itself, and how much merely comes from consuming far more inference compute than the baseline methods?
  2. For the key step-size parameter η\eta and its tolerance threshold ϵ\epsilon in the theoretical model of capability updates, why is there no formula-level sensitivity analysis?
  3. The framework claims to enable "fast response" to emerging subtasks, but the LLM-based Meta-controller introduces enormous API overhead and extremely high per-call latency when frequently performing global topology updates. In complex tasks with strict time constraints in the real world, how can this extremely inefficient inference time be reconciled with the research motivation of "efficient collaboration"?
1 comment

Dear Reviewer,

Please reply to the authors' rebuttal.

Thanks!

AC.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.