Not All Experience Transfers: Toward Boundary-Aware Self-Evolution in Task Streams

Large language model agents can improve themselves by converting accumulated experience into persistent parametric capabilities. However, existing self-evolving methods are predominantly developed and evaluated in isolated or homogeneous settings, whereas realistic deployment exposes agents to heterogeneous task streams in which tasks from different domains arrive continually without explicit boundaries or reliable feedback. This creates new challenges for recursive self-improvement, including how to accumulate experience, determine when past experience remains transferable, and suppress interference during continual parameter updates. In this work, we study parameter-level recursive self-improvement under heterogeneous task streams and propose BORDER, a framework that combines reinforcement learning with boundary-aware hierarchical experience distillation. BORDER estimates transfer boundaries directly from accumulated experience and organizes experience hierarchically according to its applicability to incoming tasks. Guided by these boundaries, transferable experience is selectively internalized into the policy through on-policy self-distillation, while reinforcement learning continues to optimize the agent using outcome-based rewards. An interference suppression mechanism further stabilizes continual parameter evolution and mitigates harmful interactions across experiences. Experiments across diverse benchmarks, task-stream settings, and model families demonstrate that BORDER consistently improves agent performance over strong self-evolving baselines, highlighting the importance of selective experience transfer for recursive self-improvement in heterogeneous task streams.