- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes Proximal Target Distillation, which aims to incorporate teacher guidance under a KL-based constraint tied to the student model. The method is evaluated on mathematical and scientific reasoning tasks, using DeepSeek-R1-Distill-Qwen-1.5B as the student and Skywork-OR1-Math-7B and Skywork-OR1-7B as the respective teachers. The paper also includes a synthetic Gaussian analysis to illustrate the proposed approach.
Critical strengths
The paper is reasonably organized, with an identifiable progression from the proposed method to the synthetic analysis and LLM experiments. This structure helps readers follow the intended contribution and locate the supporting results.
Critical weaknesses
- The mathematical formulation is insufficiently justified in the context of LLM post-training.
The assumptions underlying the mathematical development are not convincingly connected to the practical LLM post-training setting considered in the main experiments. Consequently, it is unclear what the analysis establishes about the implemented method. The authors should explicitly identify which assumptions hold in their training procedure, which require approximations, and how departures from these assumptions affect the conclusions. As presented, the mathematical development does not provide convincing support for the method’s practical use.
- The Gaussian synthetic experiment provides limited insight relative to the space it occupies.
The synthetic analysis considers a highly simplified Gaussian setting, but its relevance to the main LLM experiments is insufficiently established. Devoting approximately a full page to this analysis seems disproportionate to the insight it provides. The paper should explain what specific mechanism or prediction the experiment establishes and demonstrate its relevance in the LLM setting. Otherwise, the presentation should be shortened, with additional details moved to the appendix.
- The main experiments lack essential baseline comparisons.
The baseline coverage is insufficient to establish the benefit of the proposed distillation method. In particular, the paper should include straightforward alternatives such as supervised fine-tuning on a corpus of teacher-generated responses and rejection sampling. These comparisons are needed to determine whether the proposed formulation offers an advantage over simpler ways of using the same teachers. Data, teacher-generation budgets, and student-training budgets should be clearly reported and made comparable where feasible.
- The reported improvements are modest, even against the limited set of included baselines.
The empirical gains are small and do not yet establish a compelling advantage for the proposed method. This concern is particularly relevant given the missing baseline comparisons: the current results leave open whether similar or better performance could be achieved through simpler distillation strategies. The authors should substantiate the consistency and reliability of the improvements and explain the conditions under which the method provides a meaningful benefit. Appropriate uncertainty or variability estimates would help readers interpret these small differences.
- The presentation contains excessive numerical precision but insufficient analysis.
The numerical results are reported with an unnecessarily large number of digits, making the presentation harder to read without providing a correspondingly clearer understanding of the findings. The reported precision should be appropriate to the evaluation’s resolution and variability. More substantive analysis would be valuable, particularly regarding sensitivity to the strength of the KL constraint, differences across tasks, and the circumstances in which the method helps or fails to improve performance. Such analysis would better explain the contribution of the proposed formulation.
Questions for authors
-
Practical relevance to LLM post-training. What practical advantage does the proposed method offer in realistic LLM distillation scenarios? In particular, how do the assumptions underlying the mathematical analysis correspond to the actual LLM training procedure, and under what conditions should the method be preferred over simpler alternatives?
-
Comparison with stronger baselines. How does the method compare with supervised fine-tuning on teacher-generated responses and rejection sampling under comparable data and compute budgets? These comparisons would help establish whether the proposed formulation provides a meaningful benefit over simpler ways of using the same teachers.
-
Hyperparameter selection and justification. What is the rationale for the chosen hyperparameter values? Their selection currently lacks clear references or justification. Please clarify which settings follow prior work, which were tuned, and how the tuning was performed. How sensitive are the main results to these choices, particularly the strength of the KL constraint?