- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The setting is specialist distillation when you only have question-answer pairs and no reasoning traces. The authors fine-tune Qwen3-8B on answers only, using full fine-tuning, LoRA, or a layer-selective method. They then sample reasoning traces from that specialist and train a fresh student on them. Across chemistry, physics and translation, the student ends up with roughly the same strengths and weaknesses as its teacher. The authors argue that how far the specialist drifts from the base model is what controls this, and they back it up with a KL-anchored fine-tuning sweep.
Critical strengths
Good question, and the setup is controlled where it matters. Every student is trained the same way on the same amount of data, so differences between the students can be attributed fairly directly to the supervision generated by their corresponding specialists. The KL-anchoring sweep was the most convincing part for me.
Critical weaknesses
I think there is a simpler explanation for the results than the one emphasized in the paper. Full fine-tuning on answer-only targets produces very short and mostly structurally incomplete reasoning traces. Training a student on those traces could explain much of the degradation on harder reasoning tasks. LST moves the model less and preserves much more origin-like reasoning traces, so its student gets better supervision. This seems consistent with a more ordinary distribution-shift or forgetting explanation, without requiring the stronger interpretation of specialist optimization as "trajectory selection." I did not find an experiment that cleanly separates these explanations. The theory section does not resolve this for me either, since the actual training procedure does not marginalize over reasoning trajectories.
I am also not sure how much of some of the smaller differences in Table 2 is robust across training runs. The training appendix says experiments use a fixed seed unless multiple runs are otherwise required, and I could not find repeated training runs for the main specialist/student comparisons. Some reported gaps are under half a point. The 0.957 specialist-student correlation is interesting, and the blocked permutation test helps, but it still pools 27 measurements from only nine specialist-student pairs. Per-domain or profile-level correlations would make it easier to judge how broadly the relationship holds. The comparison to zero-shot Qwen3-32B is also less informative about the proposed distillation mechanism because the 8B specialists receive task-specific fine-tuning.
Questions for authors
-
What exactly does the specialist's answer-only training target look like? Is it just the final answer, or does it include an empty or otherwise formatted think block? If FFT is partly learning to omit reasoning because of the target format, that would change how I interpret the trajectory-collapse result.
-
Could you train a student on LST-generated traces truncated to approximately the same length distribution as the FFT traces? If the student still inherits the LST capability profile, that would make the trajectory-selection interpretation substantially more convincing.
Smaller point: an 8B model scoring around 90 on the physics in-task benchmark is striking, especially because roughly 85% of the physics sub-answers fall back to an LLM judge. I also had trouble reconciling the appendix's description of the PHYSICS benchmark with the cited Feng et al. dataset. Please clarify the exact dataset version, subset construction, and evaluation pipeline.
2 comments
Reply to Reviewer aCU8 [2/2]
Q1
We train with LLaMA-Factory’s qwen3_nothink template, which adds no <think> block or no thinking tokens, so the loss covers only the answer, as in the anonymous repository. Each training example has a system prompt asking for a direct answer (“Provide direct answers to the user’s question without showing your reasoning process. Do not use <think> and </think> tags.”), the task instruction and question, and the gold answer as the assistant turns. When sampling trajectories for distillation, we switch to the reasoning system prompt in Fig. 8 with Qwen3’s thinking mode on.
However, this format does push all three specialists toward answering without reasoning, and LoRA is clearly affected (complete traces fall from 87.9% for the origin model to 41.4%, Tab 3). The training data alone cannot tell the model whether to answer directly only when told not to think, or for every question whatever the prompt says. Both fit the training set equally well. They only diverge at sampling time, when we ask the model to think. The further a method moves the model from its origin (KL: FFT 3.05 > LoRA 2.00 > LST 1.43, Fig 2b), the more the specialist ignores that request. This is not a matter of trainable parameter count: LST updates roughly 20× more parameters than our LoRA setup, yet it keeps reasoning best.
Q2
This is a great idea for a controlled experiment. While we highly value your suggestion, we are unfortunately unable to carry it out due to limitations in computational resources and the short duration of the rebuttal period. Nevertheless we the authors thank you for your valuable advice.
Q3
Thank you for catching this. Our citation is indeed wrong: we used the PHYSICS dataset of [1]^ not Feng et al. (2025). In D.2.2, we believe we authentically describe the subset construction and evaluation pipeline for PHYSICS despite the citation error.
[1] Shenghe Zheng, Qianjia Cheng, Junchi Yao, Mengsong Wu, Haonan He, Ning Ding, Yu Cheng, Shuyue Hu, Lei Bai, Dongzhan Zhou, Ganqu Cui, Peng Ye: Scaling Physical Reasoning with the PHYSICS Dataset. NeurIPS 2025.
^ Actually we use the version from https://huggingface.co/datasets/desimfj/PHYSICS
Reply to Reviewer aCU8 [1/2]
We thank the reviewer for the careful review and the concrete suggestions. We are also glad the reviewer found the KL-anchoring sweep convincing. We address each of your point below.
W1: “A simpler explanation for FFT results”
We agree that, in terms of FFT, truncated or incomplete trajectories are the primary driver of student model degradation. However, we are afraid that this seemingly "simpler" explanation fails to address two issues:
W2: “Fixed Seed”
We did only perform the training once for each specialist/student, using a single fixed seed 42. As you appreciated in Critical Strengths, the experiments in our paper are extremely computationally intensive, requiring a vast number of GPU hours, which makes it unfeasible for us to rerun all experiments with different seeds. Maybe a fixed seed could explain the ~1pp performance fluctuation, however, we do not believe that using a fixed seed would bridge the substantial performance gaps between the various training methods presented in Table 2.
The mention of "multiple runs" primarily concerns the evaluation phase. We mainly use
OpenCompassfor evaluation, a framework that automatically performs multiple runs using different seeds for some predefined benchmarks. For instance, the results for AIME are calculated by taking the average across 32 independent runs (L1836).W3: “Per-domain or profile-level correlations”
That is an excellent suggestion. In the appendix of our next version, we will include a breakdown of the scores for each benchmark ( IT (14/1/16), ID (4/4/16), and OOD (5) ), which will provide more data for calculating correlation coefficients. Unfortunately, we have too much raw data to include in the rebuttal, but this addition will certainly not affect the conclusions in the main text, if anything, it will only strengthen them. Thank you again for your suggestion.
W4: “Zero-shot Qwen3-32B is less informative”
Our primary intention here is to demonstrate that the first stage, QA-only specialist-training on a smaller model yields better results than intuitively direct distillation from a larger, superior model. This is also precisely the approach taken by frontier models: processes like those used for DeepSeek-V3.2 involve training multiple domain-specific specialists from a single base model, and then distilling the data they generate back into a main model of the same size.