- Rating
- 3
- Confidence
- 1
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies the statistical theory of reverse-KL distillation. It compares four settings: training on student-generated or teacher-generated data, with the teacher revealing either its full output distribution or only the probability of the sampled output. With full teacher distributions, the authors give matching upper and lower bounds in both online and offline settings.
Critical strengths
The paper sets up a clean framework with four settings and gives matching upper and lower bounds for three of them. The main messages are easy to follow.
Critical weaknesses
None
Questions for authors
Can the author provide some empirical validation?
2 comments
Thank you for the detail rebuttal. I would keep my positive score.
We thank the reviewer for the constructive feedback. You questions are elaborated below.
Question:
Answer.
We have added an offline full-feedback experiment using an MLP student with partial observations. In this experiment, trajectory clipping improves trajectory reverse KL over the unclipped and forward-KL baselines. Please see Section 4.3 and Appendix C.3 of the revised paper for the setup and results.