Learning Not to Forget How to Improve: Meta-Forgetting in Self-Evolving Agents

Self-evolving language-model agents promise sustained adaptation, yet rising task performance can conceal the loss of previously acquired abilities to generate effective agent updates. We investigate this failure, termed meta-forgetting, and seek to preserve improvement ability as agents continue to evolve. We introduce Improvement Function Replay (IFR), which revisits a small set of frozen historical agent states and evaluates how effectively candidate improvement strategies modify them. IFR selects strategy updates that maximize gains on current problems while limiting declines from fixed historical reference gains, preserving the effectiveness of past improvement strategies without requiring identical edits. Across five benchmarks and three frozen language-model backbones, IFR attains higher reported mean task performance. With Qwen3-32B, its paired mean improvements in unseen ALFWorld success, OfficeQA exact accuracy, and Seal-Hard accuracy are 2.3, 2.5, and 2.6 percentage points, respectively, over Current-only + search, the strongest tested controlled alternative. IFR makes retention of improvement ability an explicit objective, complementing task-level evaluation and supporting continued adaptation with a limited replay budget.