- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes MACE, a memory-agent co-evolution framework for improving historical experience reuse in multi-agent systems. MACE introduces a graph-based memory representation to capture functional dependencies across experiences, along with adaptive memory composition, agent-specific memory coupling, and feedback-driven updates.
Critical strengths
-
The paper addresses an important problem of effectively reusing historical collaboration experiences in multi-agent systems
-
The proposed framework has a clear modular design, with separate components for memory construction, composition, agent-specific coupling, and feedback-based updates
Critical weaknesses
-
Limited Validation of Memory-Agent Co-Evolution. Although memory-agent co-evolution is the main contribution of MACE, the experiments in Section 6 use frozen memory graphs and agent-side policies, with execution feedback recorded but not used to update subsequent evaluations. Therefore, the reported performance improvements primarily demonstrate the effectiveness of memory composition and agent-specific coupling, rather than continuous co-evolution.
-
Misalignment between Benchmarks and Research problems. Most of the selected benchmarks are reasoning-oriented tasks, including question answering, mathematical reasoning, which can typically be solved without multi-agent coordination or persistent agent memory. Therefore, these benchmarks seemingly provide limited evidence of the effectiveness of MACE in scenarios that genuinely require collaborative experience reuse and memory-agent co-evolution.
-
High Cost but Limited Performance Gains. As shown in Table 3, MACE requires substantially more computational resources than the baselines. Its computational cost is approximately 11 times that of Direct, but achieving only a 2.14% improvement over the strongest baseline SAGE. The high computational cost and latency raise concerns about the practical effectiveness and efficiency of MACE.
-
Insufficient Clarity of the Empirical Study. The empirical study in Section 4 lacks sufficient details. In particular, the two memory composition strategies (C1 and C2) are only briefly described as task-matching and supporting strategies. However, the paper does not clearly explain how supporting memory units are identified and selected, or how the two strategies operate. Similarly, the Guide and Checklist formats are not concretely defined or illustrated.
Questions for authors
- How are the functional memory units and typed relations in MemGoG constructed in practice? How is the correctness of the constructed relations ensured?
- How is the credit vector in Eq.12 determined when multiple memory units contribute to a task? In particular, how does MACE distinguish genuinely useful memory units from those that are retrieved but have little influence on the final outcome?
Some other concerns listed in weakness part