- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper systematically studies when OPSD helps mathematical reasoning and when it causes degradation. During training, the student generates its own reasoning trajectories. A frozen copy of the initial model receives privileged information, such as a reference solution or answer, and provides token-level supervision on those same trajectories. The authors primarily study Qwen3 and OLMo-3, examining reasoning modes, rollout length, training duration, teacher context, and initialization, alongside different losses and choices of supervision positions. The evaluation reports Avg@8, Pass@8, and output length, and uses formatting and training dynamics to distinguish ineffective length growth, stable degradation, and behavioral collapse. The authors find that OPSD gains depend on the model and configuration: neither richer privileged information nor a stronger teacher signal alone determines training utility. Some training metrics are more useful as warnings of collapse than as predictors of final performance.
Critical strengths
- The experimental coverage is systematic. The reasoning-mode and no-privilege controls help identify mode-transfer effects, while the rollout-length, context, and initialization experiments distinguish several failure modes. These findings provide useful guidance for applying OPSD.
- The appendix includes repetitions across independent training and decoding seeds, cross-dataset validation, LoRA controls, and replications of selected OPSD variants. Together with the analyses of accuracy, output length, and formatting, these experiments provide substantial evidence for degradation and collapse.
Critical weaknesses
My main concern is the mechanistic attribution to teacher-prefix mismatch. Full solution, Answer only, and Unrelated solution use different teacher templates and apply length filtering under their respective templates. These conditions may therefore vary instruction wording, context length, target-problem position, and the retained training examples simultaneously. The observation that unrelated context also affects the supervision signal supports a contribution beyond task-relevant semantics, but does not establish how much each factor contributes to the training outcome. Section 5.3 acknowledges that these are complete-prefix interventions. The mechanistic claims in the abstract and conclusion should respect this scope or be supported by more tightly controlled comparisons.
Questions for authors
Do Full, Answer, and Unrelated use exactly the same training problems in the same order? Can the existing results separate semantic relevance from the effects of template, length, and target-problem position? If these effects cannot be separated, please clarify the scope of attribution supported by the current controls and adjust the mechanistic claims accordingly.
Response to reviewer
Thanks for your rigorous reading and recognition. Our work is based on large and broad training and evalution. We provide a revised version and hope to solve your concerns and questions.
To questions:
A1: Thanks for pointing out. In the revised version, we retrain on the same dataset in exactly the same order. The conclusion stays the same: Full solution > Unrelated > Answer only (Section 5.3, Page 5, Table 3). The old results are kept in the appendix as a robustness check.
A2: They can be separated sometimes. We additional do some similar experiments you've mentioned in Appendix E.1(Page 32, Table 30), which keeps Answer only and changes only the privileged answer, so the template is fixed and the length is fixed or only slightly different. A wrong answer yields better results than the correct answer. This shows that prefix distribution difference contributes more than semantic information in OPSD. In our main text, more detailed priviledged information always brings longer length. To minimize the impact of length, we keep the template of Answer only, Full solution and CoT+solution same. The conclusion supported by these controls is that what the privileged information brings is a context-distribution difference rather than semantic information, and a proper distribution difference can bring more gains.
To weakness:
A1: We agree that these are complete-prefix interventions, as Section 5.3 states(Page 5). The mechanistic claims stay within the scope in the answers above.
We sincerely hope this reply and the revised paper resolve your concerns and questions. We are willing to answer if anything remains unclear. If they do resolve your concerns and questions, we would be grateful if you could raise the score.