- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies why a self-distillation teacher can become unreliable even when given a correct reference solution: erroneous student prefixes can weaken its corrective guidance, while providing the entire solution upfront can create excessive teacher–student disagreement. To address this issue, this paper proposed CIPI to selectively insert reference-solution fragments into the teacher’s context during a student rollout, which balances increased correct-answer likelihood against teacher–student KL divergence. Experiments on two small Qwen models demonstrated the effectiveness of the proposed method.
Critical strengths
- The proposed CIPI method outperforms OPSD consistently in all settings with statistically significant advantages.
- The proposed method is clearly motivated by the failure-mode analysis.
- Detailed ablation study demonstrates the advantage of controlled intervention over simply adding reminders.
Critical weaknesses
- CIPI selects fragments using the teacher’s likelihood of the known answer, but increasing answer confidence does not necessarily mean the teacher provides better intermediate corrections. Independent evaluation of intermediate correction quality is missing.
- The theoretical analysis is useful, but does not establish improvement guarantees for CIPI. For example, Theorem 1 considers a single gradient step with fixed prefix and reference distributions, whereas actual training changes the student and its rollout distribution.
- Experiments only cover two small Qwen3 models.
Questions for authors
Please refer to my comments in the weakness part.
Question. Figure 2 does not show what it is said to show. What is measured is the teacher's perplexity on the gold answer tokens, not on the correct solution the caption refers to, and it is taken by appending that answer directly to a truncated and wrong prefix, which is a probe of answer confidence rather than of the supervision the teacher would actually provide. With no curve on correct trajectories and no control for prefix length or content, a rising perplexity cannot be separated from the ordinary degradation that comes with a longer context, and CIPI's lower value is expected by construction, since it places the relevant fragments next to the prediction. The figure is therefore consistent with the mechanism the method assumes, but it does not test it, and it cannot show that CIPI provides a better supervision signal.
Response. Low gold-answer perplexity reflects the teacher’s confidence in the correct answer. We use this diagnostic to track how that confidence changes along student rollouts, providing the motivation for CIPI. We understand the reviewer's two concerns: whether NLL changes reflect student rollout quality or simply distance from PI, and whether the benefit extends to actual continuation accuracy. Our additional experiments indicate that both PI placement and rollout quality matter, and that CIPI improves teacher continuation accuracy, particularly on trajectories ending in incorrect answers. We would also clarify that the statement “it places the relevant fragments next to the prediction” does not match our measurement timing: PI is inserted before a student rollout segment (Section 3, lines 179–180). Thus, at least one student segment separates PI insertion from answer scoring; a low NLL at that point indicates that the teacher retains confidence in the correct answer after observing the student segment, which we interpret as indicating that the teacher can maintain effective supervision throughout that segment.
(1) Separating trajectory quality from PI distance. We pair natural correct-answer and incorrect-answer rollouts within the same question and truncate them to identical lengths.
Reading the table vertically, as the probe moves further away from PI, NLL exhibits a continuous increasing trend, particularly at 512 tokens and beyond. Reading horizontally, incorrect-answer trajectories have higher NLL at most positions. These patterns show contributions from both PI distance and student rollout quality to the observed NLL increases. Both observations motivate our in-rollout intervention: refreshing PI during the rollout can help sustain its influence and counteract misleading student context.
(2) Evaluating actual teacher continuations. We evaluate teacher continuations at 384 planned points across 128 questions. The 12 unassessable continuation points are counted as incorrect for both methods. CIPI with full PI insertion achieves 72.66% accuracy versus 68.23% for OPSD, a 4.43-point improvement. On the 78 planned points from trajectories whose final answers were incorrect, accuracy improves from 32.05% to 46.15% (+14.10 points). These results show that in-rollout intervention not only lowers gold-answer NLL but also improves the accuracy of the teacher's generated answers.