When Prompt Internalization Breaks: Continuous Experience Internalization in Large Language Models

Deployed large language models (LLMs) continually accumulate new experiences and rules, yet standard self-distillation fails to robustly internalize these updates over time. We identify and formalize Continuous Experience Internalization (CEI) collapse, a previously uncharacterized failure mode where recursive same-parameter self-distillation systematically forgets earlier patches, absorbs new ones poorly, and amplifies high-confidence errors. To diagnose this, we introduce CEI-Bench, a stage-wise benchmark quantifying task performance, old-patch retention, new-patch absorption, and error propagation. To mitigate CEI collapse, we propose Recycled Internalization Training (RIT), a framework separating stable consensus from unstable conflicts in teacher supervision. RIT combines multi-view variance isolation (MUSE) and targeted error recycling with cross-stage anchors (RACE), explicitly addressing the structural bottlenecks driving collapse. Across four backbones and three task families, RIT recovers 39%–72% of performance lost to vanilla recursive internalization and maintains stable trajectories. Our work reframes continuous self-distillation as a diagnosable phenomenon, providing a systematic analysis, mechanistic explanation, and principled mitigation for CEI. We position CEI collapse as a fundamental challenge for adaptive LLMs, and RIT as a framework for robust recursive internalization.