- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies the model-dependent effects of next-token prediction (NTP) during domain-specific continual pretraining. Motivated by opposite outcomes when applying the same mathematical corpus and NTP recipe to two base models, the authors propose On-Policy Mid-Training Distillation (OPMD). The method selects anchor positions using a frozen proxy model’s predictive entropy, turns preceding contexts into prompts, and trains the student on its own continuations using token-level teacher supervision. Experiments show improvements over the initial Qwen3-1.7B-Base student on MATH-500 and Minerva, avoiding the degradation observed under the evaluated NTP recipe. The paper also compares teacher sizes and anchor-selection strategies.
Critical strengths
- The paper examines the dependence of continual-pretraining outcomes on the starting model. The contrasting results obtained with the same mathematical corpus and NTP recipe provide a concrete motivation for investigating training signals better suited to the student’s current state.
- The method provides a practical pipeline for applying on-policy distillation to raw domain text without requiring annotated answers or external verifiers. The experiments demonstrate mathematical reasoning gains over both the evaluated NTP baseline and the initial student.
Critical weaknesses
-
The contribution beyond existing OPD and its connection to domain knowledge acquisition remain unclear. Student-generated trajectories, token-level teacher supervision, and reverse-KL distillation have been studied previously; the main addition is a pipeline for constructing prompts from raw corpora. The experiments do not sufficiently distinguish learning new knowledge from the corpus from transferring capabilities already present in the teacher, or establish the advantage over more direct domain-distillation approaches. Furthermore, the degradation observed under one NTP recipe does not establish that its underlying cause is specifically addressed by on-policy training.
-
The connection between entropy-based anchor selection and effective knowledge acquisition is insufficiently established. A proxy model’s local predictive entropy measures next-token uncertainty, but does not necessarily indicate knowledge importance, the current student’s learning needs, or the availability of useful teacher supervision. Domain knowledge often depends on broader context, whereas the method uses uncertainty at individual positions to select starting points for extended continuations. The selection-strategy comparisons provide some empirical support, but do not sufficiently explain why the selected anchors represent more valuable learning opportunities or how much they contribute to the overall gains.
Questions for authors
- The paper is motivated by opposite outcomes when applying the same corpus and NTP recipe to Llama and Qwen. How is OPMD intended to mitigate this model dependence? The current experiments primarily establish that it improves the Qwen regression case, leaving unclear whether the gains arise from on-policy training or capability transfer from the particular teacher. Can the authors provide results for both original student models using the same source corpus and comparable teacher-supervision conditions, to clarify how the method addresses the motivating observation beyond a particular teacher–student pairing?
- What evidence shows that high-entropy positions identified by the proxy correspond to knowledge-learning opportunities from which the current student can benefit through teacher supervision? Can the existing results distinguish the contribution of anchor selection from the benefits of teacher-guided continuation more generally?