Matching While Alive: A Survival View of On-Policy Trajectory Distillation

On-policy distillation (OPD) brings teacher supervision to student-generated prefixes, but its continued availability does not ensure that it remains equally relevant along each trajectory. Our analysis reveals differences in teacher recoverability across trajectories, while vanilla OPD treats teacher matching as an uninterrupted process along every rollout. We therefore formulate on-policy trajectory distillation as a survival problem with a latent failure time for teacher matching. Marginalizing over the latent alive state converts the truncated matching objective into a survival-weighted one. This connects trajectory-level credit assignment to hazard modeling, with vanilla OPD recovered as the zero-hazard case. This framework admits a simple realization (Cox-OPD) based on a Cox proportional hazards model that combines a baseline hazard over prefix depth with observable student–teacher deviation. The accumulated hazard defines a trajectory-specific survival profile, allowing prefix history to shape how matching credit evolves along the rollout. Empirically, these weights reflect variation in independently measured teacher recoverability, and experiments on mathematical reasoning and code generation show gains across student scales in both single- and multi-teacher settings. Across all six single-teacher pairs, this survival-based extension improves average performance over vanilla OPD, with mathematical reasoning scores for 4B students increasing by 2.51 points on average across four benchmarks and both teacher settings. Ablations further highlight how the survival formulation structures the trade-off between retained credit and trajectory differentiation.