- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper combines teacher intervention during student rollouts with token-level distillation on teacher-generated segments.
Critical strengths
An interesting approach to addressing sparse rewards on difficult reasoning tasks.
Critical weaknesses
See Questions For Authors
Questions for authors
Could the authors report teacher-only accuracy under comparable decoding conditions, and compare the success rates of teacher-assisted versus unassisted rollouts throughout training? A broader analysis of whether teacher segments mainly correct student errors or instead solve most of the problem would also help clarify the proposed mechanism.
2 comments
Thank you for the detailed additional experiments and analysis. They clarify the role of teacher assistance, but I remain uncertain about how much of the improvement depends on the teacher directly supplying solutions and how broadly the findings generalize. I will maintain my original score.
We thank the reviewer for recognizing our work. Your questions were all very interesting and helped us to strengthen our paper! Below, we address each of your points in detail.
Thank you for this suggestion. We evaluated Qwen3-1.7B, the Qwen3-4B-Thinking-2507 teacher, and students trained with PIVOT under the same decoding settings. Results are below.
Students trained with PIVOT outperform the teacher on all three logic games, strengthening our conclusions.
Thank you for this suggestion. We analyzed 47,616 rollouts across all 93 training steps of the Survo PIVOT reverse-KL run. At each step, we compared the success rates of the assistance-enabled and unassisted branches. Selected steps are shown below.
Teacher assistance provides a clear early advantage, while the gap narrows as the student increasingly succeeds on its own. In this run, late-training success rates are comparable, with no consistent advantage from assistance.
Thank you for this question. We examine the teacher's role in two ways. First, we analyzed 23,658 teacher-assisted Survo trajectories from all 93 steps of the forward- and reverse-KL runs, comparing verifier success at teacher handoff and after student continuation. Results are below.
In both configurations, substantially more trajectories pass the verifier after student continuation.
Second, we randomly sampled 300 trajectories. For each KL direction, we selected 50 trajectories per training stage (early, middle, and late). In Survo, the model must fill empty grid cells with numbers that satisfy the puzzle constraints. We used gpt-6-luna to identify which student mistakes the teacher corrected and which previously unsolved cells the teacher supplied correct numbers for. We checked these numbers against the puzzle constraints. Results are below.
These automated annotations indicate that the teacher contributes both error correction and substantive solution progress.
We hope that our analysis can address your questions. We will include them in the revision. If you have any questions about our paper, please feel free to point out and we will try to address it as soon as possible. Thanks and looking forward to your reply!