BCOD: Relaxing Suppressive Token Feedback in On-Policy Distillation

On-policy distillation (OPD) provides dense teacher supervision on student-generated reasoning trajectories, but its sampled-token reverse-KL update can suppress student-sampled alternatives that the teacher assigns lower probability. We propose Balanced-Clip On-policy Distillation (BCOD), which preserves positive teacher–behavior gaps and scales non-positive gaps by $\lambda$. At the on-policy point, its expected token update descends a convex asymmetric $f$-divergence with the same teacher-matching optimum as reverse KL. The operator needs only one rollout. In a matched ablation isolating the operator, attenuation improves Avg@32/Pass@32 by $2.1/2.1$ over OPD; in the full evaluated system, which inherits SCOPE's correctness routing, BCOD reaches $54.6$ Avg@32 and $74.5$ Pass@32, versus $49.3$ and $69.1$ for standard OPD (five datasets; SkyWork-OR1-7B as teacher, R1-Distill-Qwen-1.5B as student). On fixed sets of 32 rollouts per prompt, BCOD also retains more observed base-model solutions. These results support attenuating suppressive token feedback in the evaluated OPD setting.