- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
RMT-OPD uses optimal transport to compare teacher and student reasoning modes during on-policy distillation (OPD), prioritizing teacher steps from modes the student rarely generates at states along unsuccessful student trajectories.
Critical strengths
1.The method requires only teacher-generated text, without access to teacher logits or internal representations, making it suitable for teachers available only through text-generation interfaces. 2.With the same number of samples per reasoning state, RMT-OPD achieves 60.3% average mathematical reasoning accuracy, 1.7 percentage points above uniform weighting (Uniform), showing the benefit of selectively allocating supervision weights.
Critical weaknesses
1.RMT-OPD uses 2.82 times the teacher generation volume of SOPD, but comparisons with SOPD and MOPD do not match total compute budgets, leaving unclear whether the accuracy gains justify the additional cost. 2.The code experiments use the same 131 problems for training and testing, varying only which unit tests are used, so the results do not establish improved performance on unseen code problems. 3.Only the newly generated step is passed to the semantic encoder, and the paper does not test whether reasoning-mode deficits and training outcomes remain stable with a different encoder or with the problem and preceding context included. 4.Table 4 checks whether a teacher continuation eventually reaches the correct answer rather than whether the individual step used for training is correct, leaving unclear what proportion of high-weight teacher steps are incorrect.
Questions for authors
1.Does RMT-OPD still outperform the baselines when evaluated on code problems that were not used for training? 2.Do the teacher steps selected for high weights and the final accuracy change substantially when the semantic encoder is replaced or the problem and preceding context are included? 3.According to an independent check, what proportion of high-weight teacher steps are both correct and based on reasoning modes that the student rarely generates?