- Rating
- 2(+1)
- Confidence
- 3
- Informativeness
- 2(+1)
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The authors propose a variant of on policy self distillation (on policy distillation where the teacher is shown privileged context) wherein partial student rollouts are supervised from several branching points across a single student rollout.
Critical strengths
- At face value, the numerical results are impressive.
- The idea of forking at several key positions is intuitive and well motivated (although not totally novel).
- The paper is generally clean and well presented.
Critical weaknesses
- The leakage numbers aren't significant, and why does the base model have any leakage?
- The quoted OPSD numbers differ from the paper, making the comparison more favorable for the authors.
- The method costs more (it generates more tokens during training) but this was never equalized it seems?
- Confidence intervals should really be added for your numbers. Especially with such small benchmarks/effects.
Questions for authors
- How does OPSD do when you give it the same strategy?
- What happens when you equalize training steps?
3 comments
Reply
Thank you for the detailed response. I am still considering all the replies and re-reading the parts of the paper I had concerns with, but I will certainly raise my score as several of my concerns have been addressed. I will potentially raise it further in the future after considering the responses more.
Thank you for your thoughtful response and for taking the time to reconsider our paper. We appreciate your careful evaluation of the additional experiments. We would also like to ask whether you have any remaining concerns or questions regarding our method or experimental results. We would be happy to provide further clarification and address any outstanding issues.
We thank the reviewer for the constructive feedback. We address the concerns about leakage, OPSD comparability, training cost, and statistical uncertainty below.
[W1] Leakage analysis and the base model's non-zero leakage. We agree that the leakage-rate differences in the original analysis are small and should not be treated as conclusive on their own. The analysis uses the relatively short training schedule in our main experiments, with responses truncated at 1,024 tokens. We also consider the response-style statistics reported in the paper: answer density, solution density, boxed frequency, and final-answer frequency. OPSD shows higher answer/solution densities and boxed frequency than LBSD, indicating a stronger shift toward answer-oriented generation. These statistics complement, rather than replace, direct leakage measurement. We further conducted an AIME26 leakage experiment with four OPSD rollouts per prompt and unchanged training steps. Under the same manual inspection protocol, the OPSD leakage rate rises to 3.75%, compared with 1.25% in the original AIME26 analysis. The base model's non-zero rate may reflect exposure to public benchmark problems during pretraining.
[W2] Difference between our OPSD results and those in the original paper. The OPSD results in our main table come from our reproduction rather than the original paper. The protocols differ in three respects: we train on DeepMath rather than OpenThought; we evaluate with Avg@36 rather than Avg@12; and we report a fixed 100-step checkpoint rather than the best checkpoint selected from evaluations every 20 steps. Only AIME25 and HMMT25 overlap between the evaluation sets. In particular, a fixed-checkpoint score can be lower than the best-checkpoint score.
[W3, Q2] Additional rollout cost and training-budget comparison. We added OPSD-4r to examine whether LBSD's gains can be explained by generating more tokens. OPSD-4r uses four rollouts per prompt to bring its training-token budget to the level of LBSD. It is trained for 100 steps, with other settings and checkpoint selection following the original OPSD protocol. Results are evaluated with Avg@12 (%):
LBSD outperforms OPSD-4r in 10 of the 12 model–benchmark pairs, including all four Qwen3-8B benchmarks. Thus, the gains are not explained by additional rollout count alone.
[W4] Confidence intervals. We conducted a paired bootstrap analysis with 10,000 resamples. For each resample, we sampled benchmark problems with replacement, retained all 12 responses per problem, and used identical problem indices for LBSD and OPSD-4r. The table reports LBSD minus OPSD-4r (percentage points) and 95% percentile confidence intervals:
The intervals exclude zero for AIME26 (1.7B) and HMMT25 (8B), while several others remain inconclusive. We therefore interpret the results as a favorable overall trend, not as statistically significant gains in every setting.
[Q1] OPSD with the same abstract strategy. We also trained OPSD using the same abstract strategies as LBSD while keeping the remaining settings unchanged from the paper. Results are Avg@12 (%):
LBSD outperforms strategy-conditioned OPSD in 11 of 12 settings (except Qwen3-4B on HMMT25). This suggests that the improvements are not due solely to the abstract strategy; local branching and selective supervision provide additional benefits.