From Re-expression to Retention: A Framework for Understanding Forgetting

Why can on-policy learning reduce forgetting while a model adapts to new tasks? Prior studies identify on-policy data as a key factor and propose mode-seeking or KL-based explanations, but the mechanism linking new-task updates to the preservation of old capabilities remains insufficiently understood. We propose **implicit rehearsal** as a mechanism for this retention benefit: the current policy carries previously learned behaviors into newly generated responses and actions, allowing them to participate in new-task training. When these behaviors help solve the new task, feedback can reinforce their use, and the updated policy can reuse them in subsequent trajectories. This recurring process provides a route to preserving skills shared by old and new tasks without replaying old-task examples. We introduce the **Selection, Function, and Persistence** (SFP) framework to formalize this mechanism. Selection analyzes which historical contributions new-task updates reinforce; Function connects their preservation to old-task capability; Persistence tracks how their effects evolve as the policy and its training data change together. We derive conditions under which reinforcing historical behavior improves retention after further training. Complementary experiments in language models and control reinforcement learning provide evidence for the framework's components.