- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper presents a systematic diagnostic analysis of language agents that self-evolve through textual experience. It distinguishes three core roles—trajectory builder, memory editor, and experience reader—and conducts experiments across four representative tasks, three memory-management strategies, and three model scales. The results show that new gains coexist with regressions on previously solved questions, stronger experience sources do not necessarily help weaker readers, and experience utility depends on the reader and surrounding context. These findings provide valuable evidence for understanding the limitations of experience-based self-evolution. Guided by this diagnosis, the authors explore Scope Protection and report preliminary performance improvements on the 4B model.
Critical strengths
- Beyond aggregate performance gains, the paper tracks gains and regressions at the question level. It shows that net gains can conceal failures on previously solved questions alongside successes on new ones, highlighting that more experience does not necessarily lead to sustained improvement. This is a meaningful and practically relevant problem.
- The authors develop a clear diagnostic framework that distinguishes memory construction and updating from memory reading and use, and examines reader-side differences through matched-input interventions. The appendices also document several attempts that did not yield consistent gains, providing useful lessons for future research.
Critical weaknesses
- Evaluation is limited to one model family, leaving generalization across model families unclear. All experiments use the 4B, 35B, and 122B models from the Qwen-3.5 family. However, model families may differ in long-context processing, instruction following, and robustness to interference. It remains unclear whether the observed degree of reader misalignment and scale-related trends generalize to other families.
- Differences in experimental protocols and run cohorts limit the interpretability of some cross-condition comparisons. Early HotpotQA construction curves use token F1, later interventions use binary correctness, and Table 4 uses EM. The main 4B/35B SuperGPQA construction curves use an answer-only protocol, whereas the 122B reference and some later analyses use a brief-explanation protocol. In the core refinement experiment in Table 4, Scope Protection uses an earlier complete cohort, native MemRL and Top-1 share another replay cohort, and Compact Examples rebuilds experience. Although these differences are disclosed, they make it difficult to disentangle method gains from protocol and run differences, while also making the main text and appendices harder to follow together.
- Scope Protection remains preliminary and does not establish reliable harm reduction. The main text provides limited detail on a reproducible implementation, and evaluation is restricted to the 4B model with MemRL. Average accuracy increases from 44.2% to 45.6% in Table 4, but this difference is subject to the limitations of cross-cohort comparison. More importantly, Appendix F explicitly states that, in its within-cohort comparison, additional corrections outweigh new errors while harm itself increases. The evaluation sets were also reused during exploration, making independent held-out evaluation necessary to establish the robustness of the improvement.
Questions for authors
- Could the paper provide its complete procedure or prompt template more clearly? Is the strategy also effective on the 35B model and with the ExpeL and TF-GRPO memory mechanisms?
- In regards of Table 4, could the authors conduct repeated paired replays for variants that can share a memory bank, using matched questions, banks, prompts, scoring rules, and budgets, and report construction variability for variants that require rebuilding experience? Given that Appendix F notes reuse of evaluation sets during exploration, could Scope Protection’s performance and help/harm outcomes be validated on an independent held-out set?
- Table 9 shows that, even with the same sample order, the final HotpotQA F1 gain of 4B TF-GRPO changes from -0.99 pp to -36.51 pp. However, Appendix C.2 also states that the API seed was unset and some historical code and content were not fully fixed. Could the authors fix the implementation and scoring protocol, vary generation randomness and the ordering of the 150 construction examples separately, and report multiple runs to distinguish sampling variability, ordering effects, and cumulative degradation from memory updates?