- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces a teachability-aware OPD based label-free and training-free prompt selection score. The score is a combination of (1) teacher-student disagreement in distribution, which quantifies the possible corrective signal, and (2) the teacher probability mass among the student’s top-k tokens, approximating the ease of the student to absorb that signal. Initial student rollouts and teacher-forced grading are used to score candidate prompts, without training separate models. The experiments on math reasoning benchmarks show PSV prompt improves over vanilla OPD by 1.92 AVG5 points at the best checkpoint on an average of 3 seeds. The combining prompt selection with teachability-based token selection is at 72.09 AVG5. Also the disagreement-based metrics predict useful prompts more reliably than initial model accuracy, and support weighting may improve ranking among closely matched candidates.
Critical strengths
(1). The paper presents interesting and practical motivation. It highlights an underexplored aspect of on-policy distillation: how the generation prompt affects the supervision available to the student. The proposed training-free, label-free selection approach is computationally efficient and can be useful in practice.
(2). The paper evaluates multiple prompts across full training runs and repeat the main comparisons over three seeds. The selected prompt consistently improves over vanilla OPD and also works well with token selection. These results support the importance of prompt choice, even if the advantage of the specific scoring function over simpler alternatives is less convincing.
Critical weaknesses
(1) The novelty of this paper is limited. The paper added a simple disagreement metric as KL divergence and select the same winning prompt as the proposed score. The additional benefit of support weighting is demonstrated is through a small, single-seed comparison with a tiny difference (0.51 points, see table 1 Canonical Freeform vs Tabular-CoT curve-arg score). this is within reported seed variability. Also, at the best checkpoint, Tabular-CoT performs better than Canonical Freeform (see table 1 AVG5). So the key methodological contribution is not convincing.
(2). The evidence is insufficient to support the mechanism. The same prompt is given to the student and the teacher, changing both the student's sampled trajectories and the teacher's supervision. it is unclear that wether the improvements come from more teachable student trajectories or just better teacher responses under the selected prompt, since there is no ablations separating these effects. This weakens the paper's explanation of why the proposed approach works.
Questions for authors
-
The experiments use one teacher–student pair and focus on math reasoning. Do you have evidence that the method works across other models or task domains?
-
please see weakness above.
Dear Reviewers,
The authors have posted responses. As the discussion period is ending soon, could you please take a look and reply shortly, or at least acknowledge that you have read it?
Thank you!
AC