- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
The paper proposes On-Policy Mechanism Distillation (OPMD), a two-stage framework for transferring teacher-derived intermediate information into a student language model. First, it trains layer-wise projectors to inject teacher representations into a frozen student, optimizing the resulting output behavior to construct a “student-compatible teacher.” Second, the native student learns to reproduce this composite model’s hidden states on student-generated prefixes. The main on-policy experiments train student LoRA parameters using only latent-state matching during this second stage; the teacher and projectors are discarded at inference. Evaluation covers three teacher–student settings on mathematical reasoning benchmarks, alongside multi-teacher math/code distillation and cross-family supervised fine-tuning. OPMD achieves the highest average accuracy among the evaluated baselines in each main setting, although gains vary across tasks.
Critical strengths
(1) The proposed OPMD provides a coherent approach to constructing usable intermediate supervision. Training projectors through downstream output behavior gives a concrete operational meaning to compatibility. This is consequential because it addresses a limitation of geometric alignment and provides a practical alternative for constructing distillation targets.
(2) OPMD achieves the highest average mathematical accuracy in each of the three main teacher–student settings. Including OPRD-Bridge strengthens the comparison beyond output-only methods. These results support the usefulness of the proposed intermediate supervision through task performance rather than representation similarity alone.
(3) The training-time and FLOPs comparisons include both stages and indicate a favorable accuracy–cost trade-off in the examined configuration.
Critical weaknesses
(1) Weight patching establishes that learned parameter updates affect student performance, but does not identify which teacher computations were transferred or distinguish these effects from ordinary distillation. The solution-route analysis examines only two problems selected using observed performance differences, with 16 responses per condition. Its comparisons primarily concern patched and fully distilled students, rather than teacher–student computational correspondence. These analyses are informative exploratory evidence, but do not substantiate the broader claim of transferring how the teacher reasons.
(2) The presented tables and curves lack accompanying uncertainty estimates and clearly specified seed counts, while several gains are small or task-dependent. For example, the 0.61-point HumanEval+ improvement corresponds to approximately one additional solved problem; cross-family supervision also decreases AIME 2024 accuracy.
Questions for authors
(1) For OPRD-Bridge and the projected-MSE controls, which projector architectures, capacities, layer correspondences, training contexts, and optimization budgets differ from OPMD? In particular, does the MSE control use the same residual injection interface as Equation (3), or state replacement? These details determine whether the observed differences isolate functional versus geometric target construction.
(2) How many independent training seeds contribute to each main result and each “available-seed mean,” and are those counts consistent across methods and checkpoints? Can the authors provide uncertainty estimates for the principal comparisons that account for both training variability and shared evaluation problems? This is particularly important for the smaller gains and mixed cross-family results.