- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies how to allocate direct teacher supervision during on-policy distillation. SAKI builds on TRB and uses maximal-coupling acceptance and correction events to select the training objective. Accepted positions retain reverse-KL updates, while correction positions use cross-entropy on the teacher's highest-probability token. The experiments report improvements in mathematical reasoning.
Critical strengths
-
The research question is worthwhile, and the idea makes sense. Using information from the rollout process to decide where to apply direct teacher supervision is a reasonable approach (Sections 1 and 3).
-
The paper includes relevant placement and target ablations. It compares correction-based placement with count-matched random and TV-weighted placement, and compares teacher Top-1 targets with teacher-sampled targets (Appendices D and F).
Critical weaknesses
-
[Medium]The experimental coverage is narrow for an incremental contribution. The behavior distribution comes from TRB, and both training losses are standard; the main algorithmic addition is the rule that switches objectives at correction positions (Section 3). All seven benchmarks concern mathematical reasoning. Both students, Qwen3-0.6B and Qwen3-1.7B, use the same Qwen3-4B teacher. There are no other model families or teacher configurations, and the placement controls are tested only on the 1.7B student (Section 4.1; Table 6). -
[Medium]The small gains over the closest control lack evidence of training stability. Relative to TV-Weighted-TM, SAKI improves macro Mean@8 and Pass@8 by 0.76 and 0.73 percentage points. The main results do not report means and standard deviations across independent training seeds. The prompt-bootstrap intervals in the alignment analysis do not address this uncertainty (Table 6; Appendix E).
The problem is valid and the idea is reasonable, but these limitations leave me unconvinced that this incremental design offers enough reliable value to justify acceptance.
Questions for authors
-
How stable are the gains across independent training seeds? Please report paired repetitions for SAKI, TRB, and TV-Weighted-TM, including the variability of their final benchmark differences.
-
Does the benefit extend beyond the current model and task setup? An additional teacher-student configuration or a non-mathematical task would help assess how broadly the routing rule applies.
-
What does correction-based routing add over simpler supervision schedules? There is no direct budget-matched SFT-to-OPD comparison. Moreover, moving teacher-mode supervision also changes which positions lose their reverse-KL updates, so the placement controls do not fully separate these two effects (Section 4.1; Equation 17; Appendix D). A budget-matched SFT-to-OPD comparison and a control that separates teacher-label placement from reverse-KL removal would clarify the source of the gains.
Thank you again for your time and consideration. We have provided detailed responses and additional experimental results addressing the concerns raised in the reviews. We would greatly appreciate it if you could take another look at our responses and let us know if any concerns remain or if further clarification is needed. Thank you for your consideration.