acceptodds
Under review as a conference paper at ICLR 2027

CoEvolve: Counterfactual-Guided Co-Evolution of Memory and Collaboration in Multi-Agent System

Abstract

Large language model (LLM)-based multi-agent systems have demonstrated remarkable capabilities across diverse tasks. Yet their memory and collaboration architectures often remain static, constraining continued adaptation and improvement over time. Recent work addresses this limitation by evolving memory or collaboration based on interaction experience. However, updates to these interdependent components are not necessarily aligned: an evolved workflow may require experience that memory does not provide, while evolved memory may encode guidance that the workflow cannot use. To address this problem, we propose CoEvolve, a skill-centered framework for aligned memory–collaboration evolution. CoEvolve represents reusable local collaboration patterns as skills that link evolving collaboration decisions to their supporting memories. It further compares original and counterfactual trajectories to identify local evolution signals, which coordinate updates to the skill library, skill memory, and executor memories across rounds. Experiments on the ALFWorld and WebShop benchmarks confirm that CoEvolve achieves the best overall performance among the evaluated baselines, while ablation studies further validate the effectiveness of aligned co-evolution and counterfactual-guided evolution. Our code is available at https://anonymous.4open.science/r/Co-Evolve-8239.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM-based multi-agent systems, self-evolving agents, agent skills, memory evolution, collaboration evolution, counterfactual analysis, cross-component mismatch

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_PoBrOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes CoEvolve, a framework that jointly evolves collaboration skills, skill memory, and executor memory in multi-agent systems. Skills provide a shared interface between collaboration patterns and their supporting experience. Counterfactual continuations, generated by removing an executor from a failed trajectory, supply signals for coordinated updates. Experiments evaluate the framework on ALFWorld and WebShop.

Critical strengths
  1. Jointly adapting collaboration and its supporting memory is a useful system-design contribution. The distinction between skill memory for stage allocation and executor memory for stage execution provides a concrete way to maintain compatibility as collaboration patterns evolve.

  2. The component ablations provide meaningful evidence for the complete framework. Table 2(a) evaluates all eight combinations of the three evolving components, and the full system outperforms every partial configuration. Table 1 reports improvements over the strongest listed baseline on both ALFWorld, from 0.7537 to 0.7985, and WebShop, from 0.6681 to 0.7072. Appendix E.1 also explicitly separates evolution tasks from validation and test tasks.

Critical weaknesses
  1. The experiments do not sufficiently distinguish the proposed counterfactual and alignment mechanisms from additional search and experience. Removing an executor changes the subsequent allocation and execution trajectory, while the counterfactual ablation does not replace these additional continuations with ordinary resampling under the same budget. This matters because Appendix F.2 reports an average of 3.912M tokens per round for counterfactual sampling on ALFWorld. Similarly, the factorial ablation establishes that evolving multiple components is beneficial, but does not isolate alignment itself: a configuration where all components evolve without conditioning memory updates on the updated skill space is missing. Consequently, the specific mechanism claims are less well supported than the overall system results.

  2. The advantage over strong baselines requires clearer evolution-stage resource accounting and uncertainty estimates. Evaluation backbones are aligned, but the paper does not adequately document each baseline’s evolution models, training-task exposure, additional rollouts, and optimization budget. The ALFWorld improvement over AutoGen-GMemory corresponds to approximately six additional successes among 134 tasks, yet no repeated evolution runs or paired uncertainty estimates are reported. The results therefore establish an advantage for the reported configurations, but do not yet demonstrate its robustness to sampling and resource allocation.

  3. CoEvolve should be positioned against Skill-MAS: Evolving Meta-Skill for Automatic Multi-Agent Systems, which evolves reusable orchestration knowledge through multi-trajectory feedback and contrastive reflection. Please clarify the additional contribution of jointly updating local collaboration skills, skill memory, and executor memory, and distinguish the proposed executor-removal interventions from trajectory-based contrastive reflection.

Questions for authors

See Weakness

Reviewer_ZiyhOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces CoEvolve, a framework that jointly evolves memory and collaboration in multi-agent systems. It represents reusable collaboration workflows as skills. It maintains a skill library, skill memory, and executor memory to support skill selection, stage assignment, and task execution. After a task fails, CoEvolve restarts from an intermediate state and removes one executor. It compares the original and counterfactual trajectories to identify important local segments. These signals are then used to generate or select new skills and update the related memories. Experiments on ALFWorld and WebShop show that CoEvolve outperforms the evaluated collaboration- and memory-based methods. Ablation studies also show that joint evolution, counterfactual trajectories, and delayed skill promotion improve task performance and execution efficiency.

Critical strengths
  1. Using skills to connect collaboration workflows and memory is a clear and practical design. It allows new collaboration patterns to receive relevant experience while avoiding mismatches caused by updating workflows and memory separately. This provides a complete framework for jointly updating persistent components in multi-agent systems.

  2. The experiments provide targeted support for the main design choices. CoEvolve achieves the best overall results on both ALFWorld and WebShop. The ablation studies also show the benefits of jointly updating skills and memory, using counterfactual sampling, and delaying the promotion of candidate skills.

Critical weaknesses
  1. The counterfactual process cannot reliably perform local credit assignment. Removing one agent changes task allocation, collaboration, and the behavior of other agents. Therefore, the performance difference may not accurately represent the local contribution of the removed agent or its memory.

  2. The alignment between collaboration and memory is not directly validated. The experiments mainly report improvements in final task success rates, but do not directly measure whether the collaboration structure becomes better aligned with individual agent memories. This leaves a central claim of the method insufficiently supported.

Questions for authors
  1. Removing one agent changes the execution of the entire system. How do the authors ensure that the counterfactual result reflects the local contribution of that agent, rather than changes in task allocation or the behavior of other agents?

  2. Can the authors provide direct metrics or analyses showing that joint evolution improves the alignment between the collaboration structure and individual agent memories, rather than only improving the final success rate?

  3. The related-work section discusses joint-evolution methods such as Mem2Evolve, Meta-Team, EVOCHAMBER, and COSPLAY. Why are these methods not included in the experiments? Could the authors adapt at least one closely related method or explain why a fair comparison is not possible?

  4. CoEvolve introduces additional computation through counterfactual re-execution, reflection, and multi-component updates. Could the authors provide a compute-matched comparison to separate the benefit of joint evolution from the benefit of additional model calls or environment interactions?

1 comment

Thank you for your response. I decide to raise my rating. Good luck!

Reviewer_zKGMOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies self-evolving LLM-based multi-agent systems and identifies cross-component mismatch as a central failure mode: when memory and collaboration structures evolve independently, an evolved workflow may require experience that memory does not provide, while evolved memory may encode guidance that the workflow cannot invoke. To align the two, the proposed framework CoEvolve represents reusable local collaboration patterns as skills and organizes three evolvable components around this shared skill space, namely a skill library, a skill memory that guides stage allocation, and per-executor success and failure memories. After each round, CoEvolve builds counterfactual trajectories by re-running failed tasks from a branch point with one executor removed, scores and filters local segments, identifies outcome-sensitive pivot segments, and uses these signals to jointly update the skill library, skill memory, and executor memories across rounds. Experiments on ALFWorld and WebShop report the best overall performance among seven baselines, and ablation studies are presented to validate aligned co-evolution and counterfactual-guided evolution.

Critical strengths
  1. Motivation: The problem is well motivated and clearly positioned. The cross-component mismatch argument in Section 1 and Figure 1 is concrete and intuitive: memory-only evolution can produce lessons that a locked workflow cannot invoke, while collaboration-only evolution can introduce stages such as Detect and Reflect that fixed memory cannot support. This is a genuine gap in the self-evolving multi-agent literature, and the related work section organizes prior art cleanly into memory-side evolution, collaboration-side evolution, and evolution-signal lines, situating the paper clearly against concurrent co-evolution work such as EvoChamber and Mem2Evolve.

  2. Method design: The skill-centered interface is a clean and sensible design: skills act as a shared unit that binds an evolving collaboration pattern to the memories that support it, with structural fields kept fixed and metadata fields evolvable, which directly implements the alignment the paper argues for. The counterfactual signal acquisition is a reasonable step beyond task-level scores and post-hoc trajectory analysis, the delayed promotion of candidate skills through a candidate pool is a pragmatic safeguard against harmful updates, and the pivot identification rule in Eq. 16 and Eq. 17 based on the first positive crossing of cumulative step-score differences is simple and interpretable.

  3. Experiments: The evaluation covers two standard interactive benchmarks, seven baselines from both the collaboration-based and memory-based families, a task-wise breakdown in Table 1, a component ablation in block a of Table 2, counterfactual and promotion ablations in block b of Table 2, skill usage and coverage analysis in Figure 3, robustness to orchestrator capacity in Figure 4, retrieval-weight sensitivity in Table 3, and evolution overhead reporting in Table 4. The anonymized code repository and the complete prompt templates in Appendix I are helpful for reproducibility.

Critical weaknesses
  1. Fairness of the comparison is not established. Appendix E.1 states that CoEvolve evolves for three rounds on 50 training tasks per round, using Qwen3.5-Plus as the orchestrator and evolution module and GPT-4.1-nano for executors during evolution, while evaluation uses Qwen3-235B for all methods. It is never stated whether memory-based baselines such as ExpeL, AWM, EvolveR, and MemEvolve were given the same training tasks, the same number of adaptation rounds, or a comparable token budget. If baselines received less adaptation exposure, the 4.48 point margin over AutoGen-GMemory in Table 1 cannot be attributed to the proposed mechanism rather than to unequal adaptation budgets.

  2. The counterfactual design is narrow relative to its claims. Eq. 3 constructs counterfactuals only by removing one executor, which tests executor necessity but never alternative skill selections, alternative stage allocations, or alternative memory contents, even though the framework presentation suggests decision-level counterfactual credit assignment more broadly. Moreover, the pivot rule in Eq. 16 and Eq. 17 depends entirely on an LLM-based step scorer whose scoring accuracy is never validated, so errors in the scorer propagate directly into the evolution signals.

Questions for authors

Please refer to the weaknesses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.