When Should the Teacher Move? Temporal Coupling and Stability in Self On-Policy Distillation

Self on-policy distillation trains a student policy against a teacher derived from its own parameter history, yet the teacher's update schedule—which governs the temporal coupling between teacher and student—has not been systematically studied as a stability variable. Through a controlled schedule sweep on Qwen3-8B, we establish that isolation periods—complete teacher freezing between updates—are the key structural property enabling stable learning, not teacher age, and we introduce a diagnostic framework of temporal KL structure, refresh shock, and length-tail risk to characterize training dynamics. We further identify state-oblivious collapse: the best short-horizon fixed schedule ($M=50$) catastrophically fails under long-horizon training because a clock-driven refresh can copy a transiently drifting student into the teacher in a single, irreversible step—a failure mode invisible under short-horizon evaluation and mechanistically distinct from EMA's chronic contamination. To address this, we propose Consolidation-Gated Teacher Refresh (CGTR), which preserves isolation periods while gating each refresh on joint evidence of reward improvement and length-tail safety, ensuring every teacher movement responds to genuine student consolidation rather than a clock signal. With a single shared parameter set and no per-dataset retuning, CGTR achieves zero collapse and the best final score on all four tasks (Chemistry, Biology, Physics, ToolUse), self-regulating its refresh frequency to each task's learning dynamics.