- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
Traditional on-policy self-distillation relies on distributional matching that does not directly assess a student's actual learning gains. To address this, MetaOPSD adapts the privileged teacher by backpropagating verified student outcomes from a pilot distillation step. Consequently, MetaOPSD consistently outperforms evaluated baselines on mathematical reasoning benchmarks and demonstrates superior teaching utility across student states.
Critical strengths
-
The paper distinguishes predictive power from teaching utility by framing teacher adaptation around post-update student performance, which better aligns with the true goal of OPSD than static distribution or likelihood matching.
-
It provides comprehensive specifications—from teacher adapters and differentiation through AdamW/clipping to RLOO surrogates and seed schedules—which are essential for reproducible optimizer unrolling.
Critical weaknesses
-
Fixed-step comparisons ignore compute fairness: While MetaOPSD leads at 200 updates, Table 3 reveals its RLOO variant consumes ~7.2× the GPU-hours of OPSD and ~5.9× that of VISTA. Under a shared 24 GPU-hour budget, MetaOPSD-RLOO (38.04%) is actually outperformed by both OPSD (38.24%) and VISTA (39.66%), suggesting gains stem from heavy compute consumption rather than algorithmic efficiency.
-
Teaching utility evidence is narrow and marginal: The cross-evaluation is limited to Qwen3-1.7B on just 128 MATH-500 questions over 20 updates, yielding an average gain of only ~0.48 pp over the initial teacher. Such tiny margins on a small probe set are insufficient to prove the teacher has acquired a meaningfully superior teaching strategy.
Questions for authors
-
Can you provide accuracy-vs-GPU-hour Pareto curves for OPSD, VISTA, Reference-NLL, and MetaOPSD-RLOO, rather than only fixed 200-update endpoints? My recommendation would change if MetaOPSD is competitive on this curve.
-
Under the same 24 GPU-hour budget, why does MetaOPSD-RLOO underperform VISTA so clearly? Is the bottleneck feedback generation, verifier cost, second-order graph construction, or teacher update frequency?
-
Does the conclusion hold beyond Qwen3 and three short math benchmarks? Results on a larger held-out math set, non-math verifiable tasks, or at least more than 90 final benchmark questions would materially strengthen the paper.
We thank the reviewer for recognizing the importance of the research problem and the novelty of our proposed method. In response to your concerns, we provide the following clarifications and conduct additional experiments to further validate the effectiveness of our method.
1. Experimental Reliability and Random Seeds
First, we would like to clarify that our main experimental results are not based on a single training run, but are averaged over three independent runs with different random seeds (42, 43, and 44), as detailed in Table A.1 and Appendix C.3. MetaOPSD consistently outperforms VISTA across all three benchmarks at both model scales, achieving average improvements of 1.98 and 1.36 percentage points, respectively. To further address concerns regarding randomness and experimental variability, we additionally report the results for each individual seed, along with the corresponding standard deviations.
2. Regarding the avg@12 Evaluation Metric
We adopt 12 samples to maintain consistency with the evaluation protocols used in related work. Both OPSD (https://arxiv.org/abs/2601.18734) and VISTA (https://arxiv.org/abs/2608.28306) report mathematical reasoning performance using avg@12, which measures the average proportion of correct responses across 12 independent samples for each problem. All methods in our paper are evaluated using identical sampling and scoring settings.
“Macro Avg.” refers solely to the arithmetic mean of the accuracies on the three benchmarks: AIME24, AIME25, and HMMT25. We will replace this term with “Average Accuracy” and revise “macro gains” to “improvements in average accuracy” to avoid ambiguity.
3. Regarding Test Set Size and Result Stability
We acknowledge that a test set of 90 problems may not be sufficient to fully characterize the generalization performance of our method. To address this concern, we conduct additional evaluations on Omni-MATH using the existing training checkpoints, without further training.
The supplementary evaluation results are summarized below (%):
4. Regarding Model Scale and Computational Cost
We agree that validation on larger models is important. MetaOPSD currently demonstrates consistent performance improvements at both the 1.7B and 4B scales. Furthermore, cross-evaluation of teacher checkpoints provides additional evidence that teacher adaptation improves the effectiveness of the supervision provided to the student. Nevertheless, these results do not establish that the same benefits will extend to larger models or other task domains.
Regarding training cost, the primary objective of this work is to investigate how feedback from student learning can improve teacher supervision, rather than to minimize training compute. Table 3 explicitly reports the actual training costs of different methods and provides comparisons under a fixed computational budget. Our current results support the conclusion that MetaOPSD achieves higher accuracy with the same number of student update steps, but do not imply superior computational efficiency.
5. Regarding Paper Clarity and Presentation
We thank the reviewer for the suggestions on improving the readability of our paper. We will simplify the abstract and method description and refine the overall presentation to highlight the core idea more clearly: Rather than merely training the student to imitate the teacher, MetaOPSD optimizes the teacher based on the student's actual learning outcomes after receiving supervision, enabling the teacher to provide supervisory signals that better support the student's ability to solve problems independently.