- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
DuoOPD modifies sampled-token on-policy distillation using jointly verified teacher and student responses. Student correctness determines the feedback sign; a cached successful teacher response conditions teacher scoring when only the teacher succeeds, while a task-level shared positive weight reinforces responses when only the student succeeds.
Critical strengths
-
DuoOPD allows the student to learn from a teacher without treating the teacher as uniformly superior. In particular, it reinforces verified student successes even when the teacher fails, addressing a concrete problem in distillation across heterogeneous tasks.
-
A successful teacher response can guide scoring of a failed student response without being added to the student's input. This uses teacher solutions while preserving the student's inference setting; the overall method improves on the strongest compared baseline by 1.16 macro points for Qwen3 and 4.58 for Llama (Table 1).
Critical weaknesses
-
One cached teacher answer determines how the teacher is used for each question throughout training (Section 3). If that answer fails, the method uses a different rule than it would after a successful sample. Another teacher sample could therefore change the training signal for the same question. The three-run results do not establish whether performance is stable across independently generated teacher caches.
-
A correct final answer can still contain wrong reasoning, but DuoOPD reinforces all tokens of a passing response. Appendix D.2 gives such an example: the answer passes while some method names are explained incorrectly. The same problem can affect cached teacher references. The reported answer accuracy therefore does not show whether the method improves or worsens explanation quality.
Questions for authors
-
Do the three runs share one teacher cache? Do the gains hold when the cache is generated again with different sampling seeds?
-
Among answers with correct final options, does DuoOPD produce more or fewer incorrect explanations than OPD and OPDVR? Please also check the cached teacher references.
Dear Reviewers,
The authors have posted their responses to the reviews. Please go through their rebuttal at your earliest convenience, and engage in a discussion or leave any follow-up comments.
The discussion period ends on November 18.
Thanks,
Area Chair