- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
The paper proposes Knowledge-Centric Self-Improvement (KSI), a framework that improves agent performance by aggregating task experiences through forum-based discussions and distilling them into a shared knowledge base.
Critical strengths
The paper investigates an important research question about agent self-improvement and evaluates its effectiveness across multiple benchmarks, including coding and terminal tasks.
Critical weaknesses
-
Very low novelty. The work is largely identical to FoT [1], CORAL [2], FederatedSkill [3], SAGE [4], etc. These works are not even discussed.
-
The experiments do not support the claims. First, removing the cross-task forum only changes performance from 76% to 72%, corresponding to only two tasks on Polyglot. This ablation is also conducted using a single seed. In the main experiments, agents repeatedly attempt the same pool of tasks over ten generations while retaining task-specific knowledge from previous attempts. Consequently, improvements may largely result from accumulating information about previously encountered tasks rather than acquiring transferable knowledge. ARC-AGI-1 and ARC-AGI-2 contain tasks from the same domain, while SWE-bench Pro and Polyglot are both coding benchmarks. They cannot support the cross-domain claim. Terminal-Bench 2 can reflect cross-domain capabilities.
[1] Yao, Dixi, Tahseen Rabbani, Manzil Zaheer, and Tian Li. "Federation over Text: Insight Sharing for Multi-Agent Reasoning." arXiv preprint arXiv:2604.16778 (2026).
[2] Qu, Ao, Han Zheng, Zijian Zhou, Yihao Yan, Yihong Tang, Shao Yong Ong, Fenglu Hong et al. "CORAL: Towards Autonomous Multi-Agent Evolution for Open-Ended Discovery." arXiv preprint arXiv:2604.01658 (2026).
[3] Yang, Jingbo, Guanyu Yao, Yang Zhang, Ramana Rao Kompella, Gaowen Liu, and Shiyu Chang. "FederatedSkill: Federated Learning for Agentic Skill Evolution." arXiv preprint arXiv:2606.03143 (2026).
[4] Pan, Linyue, Yaoming Zhu, Lin Qiu, Xuezhi Cao, and Xunliang Cai. "SAGE: A Quantitative Evaluation of Socialized Evolution in Agent Ecosystems." arXiv preprint arXiv:2606.03544 (2026).
Questions for authors
No particular question.