- Rating
- 4(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3(+1)
- Soundness
- 3
Read review
Paper summary
This paper studies if models can learn inference-time scaffolds from the execution traces of the model using the scaffold. Across tasks like Dyck, Sokoban and competition mathematics, student models that are the same size as teacher models generally underperform the teacher model using the scaffold. The paper argues that larger students, simple scaffolds and longer reasoning chains can help reduce the gap between the student and teacher model. The paper also provides theoretical contributions stating that some scaffold capabilities cannot be transferred into a student with a fixed depth and reasoning length through the graph-search example.
Critical strengths
- The paper showcases that a student model, that is equal in size to the teacher, is generally not able to match the performance of a teacher model + scaffold, and this is a quite important finding as this continues to establish why scaffolds are important.
- The paper does extensive ablation experiments on different configurations, such as bigger students, simpler scaffolds and longer chain-of-thoughts, that can reduce gap between student and teacher+scaffold.
Critical weaknesses
- The paper provides very little information on what specifically the student fails with. It is unclear if there are specific operations the student struggles with or if these errors accumulate over long traces.
- The paper provides a decent number of tasks (Dyck, Sokoban, Modular Arithmetic, DAPO-Math-17k, Math Proofs), but its main study with pre-trained model distillation focuses on mathematic tasks, so adding more variety of tasks could provide a more comprehensive picture.
Questions for authors
- Could you provide some analysis on specifically what the student fails on for a task? What intermediate steps is the student failing on or do errors accumulate over longer traces?
- Do you have distillation results with more pre-trained models and more variety in the tasks besides just math? It would be interesting to see whether a similar student and teacher gap appears in coding benchmarks like HumanEval [1] and more models like Qwen 2.5 or Llama series models.
I lean towards a weak accept because of variety of experiments and theoretical contribution towards understanding scaffolds. Intermediate steps error analysis that clarifies why the students fail would increase my confidence and raise my score.
References
[1] Chen M., Tworek J., (Open AI Team)..., "Evaluating Large Language Models Trained on Code". https://arxiv.org/pdf/2107.03374.
2 comments
Thank you to the authors for their detailed responses to my questions. I find the analysis on why the student fails quite interesting and it strengthen's the author's argument that their is a true representational limitation that is preventing the student to match the levels of the teacher. The additional experiments on HumanEval and MBPP generalize these results and reiterate that this gap occurs across various domains and models. Overall, the additions strengthen the author's claims and I will increase my score to a 4.
**Response to Reviewer WhrE**
We thank the reviewer for their time and for the positive comments on our main finding and on the range of our ablations. We address each point below.
W1 & Q1
Thank you for this suggestion. For the experiments in our paper, we analyzed which intermediate steps the student fails on and whether its errors accumulate.
Sokoban with A*. The student often fails at the step where, at each expansion, it has to work out in place of the base model which two moves are the best legal ones. Errors do accumulate. Before its first error, each expansion fails with a roughly constant probability of about 13%. The teacher needs 28 to 49 expansions, and the chance that the student makes 20 expansions in a row without an error is only about 5%. After the first error, its bookkeeping of the search becomes more and more inconsistent.
Sokoban with VGB. The student often gets the backtracking probability wrong. Errors accumulate here too: once it makes a wrong move, it is much more likely to reach a dead end later.
Modular arithmetic. The student fails on the step that has to evaluate several operators at once. In our setup this is usually the last step, which computes all the remaining operators in one go, so errors do not accumulate.
Math with Branching. Each part the student writes is reasonably good; the problem is that it does not finish. The whole tree is long, and the student fails to write out a tree of that length.
W2 & Q2
Thank you for the suggestion. To answer your question about other tasks and model families, we ran the following experiment.
We add an experiment showing that the gap also appears on a coding task and with two other model families.
Setup. We use Qwen2.5-1.5B-Instruct and Llama-3.2-1B-Instruct with the same tree as in Section 5 (13 calls). The leaves write code independently, and each node above them keeps a child's program that passes the examples given in the problem statement, asking the model for a new one only if none passes. The hidden tests are never shown to the teacher. The student is the same model with LoRA, trained as in Section 5 on correct teacher traces from MBPP problems 1–500 to write the whole tree in one pass, without an executor. We report the best of four training recipes and evaluate on HumanEval and on the held-out MBPP problems 501–974.
Results (accuracy in %):
The gap between teacher and student is 24 to 30 points for both model families, and HumanEval+ gives the same conclusion. Details are in Appendix F.4 (Table 7) of the revision. These results demonstrate that the gap is not specific to math or to Qwen3.