- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2(+1)
Read review
Paper summary
This paper did an empirical study on Multi-teacher on-policy distillation. Specifically, Qwen3-1.7B/SmolLM3-3B with four domain teachers trained with RL from the same initialization as the student, and they investigated different loss averaging schemes, optimizer updates, and task learning curves. The authors claimed that they found token-level averaging favors longer responses, Adam first moment reduces differences, and BF16 rounding hides many small parameter changes visible in FP32 during training. The authors did experiments across four downstream benchmarks with five seeds.
Critical strengths
- The experiments are solid, as the authors did across different benchmarks and multiple seeds.
- The results provided several useful insights from loss aggregation, optimizer state, numerical precision, and vocabulary supervision.
Critical weaknesses
-
My main concern is that i don't find a clear and coherent research narrative. While the title says "From Gradients to Capabilities," the paper presents several largely disconnected empirical observations. The paper reads more like a collection of different experiments.
-
Although the authors show that shared Adam momentum substantially increases update similarity across teachers, they do not establish whether this alignment actually causes any corresponding effect. This is also applied to BF16/better gradient approximation.
-
The model scale is another concern.
Questions for authors
What is the central scientific hypothesis of this paper?