- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes MOTAB, which monitors student reasoning using teacher likelihood and an adaptive threshold, backtracks when a trajectory strays, and appends a teacher continuation to the student's failed exploration for distillation. It aims to address both the mismatch between student training and inference distributions and unreliable teacher supervision on erroneous student prefixes. Results across multiple models and datasets show improvements over the main offline baselines and performance comparable to top-k OPD.
Critical strengths
-
The method combines teacher continuations from backtracked prefixes with failed student exploration, preserving both exploration and correction in the distillation data.
-
In Table 1, the Avg. score improves by 7.27 and 2.50 percentage points over the best SKD configuration under the two teacher settings, respectively. Tables 6 and 7 in the main text also report gains with other student models and on tasks from other domains.
Critical weaknesses
-
Direct evidence of teacher supervision failure is missing. The low likelihood in Figure 1 could reflect the teacher correctly rejecting erroneous steps. The distributional differences in Figure 2 do not establish supervision failure either. Figure 4 evaluates the method using the same teacher likelihood used to select trajectories, without independent validation such as the correctness of teacher continuations.
-
The backtracking ablation does not control for the amount of teacher generation. Removing backtracking in Table 2 also changes where teacher continuations begin and how long they are. Without comparisons against fixed depth or random backtracking under matched teacher token and training budgets, the gains could come from more teacher generation.
-
The baseline comparisons remain insufficient. The main experiments modify SKD's acceptance criterion. In Table 4, the AIME24 and AIME25 pass@1 gains over top-k OPD are only 0.84 and 1.26 percentage points, respectively, while MATH500 performance is slightly lower. Uncertainty estimates for these differences are not reported.
Questions for authors
-
Could the authors compare the correctness of teacher continuations from the original and backtracked prefixes on the same set of student trajectories? Can likelihood and entropy distinguish valid negative feedback from ineffective supervision?
-
With teacher token counts and student training budgets matched, does backtracking based on the largest likelihood drop still outperform fixed depth or random backtracking?
-
Which sampling and training procedures were changed from the original SKD method? Can the small differences from top-k OPD in Table 4 be supported by independent runs or an uncertainty analysis paired by problem?
-
Do erroneous student prefixes contribute to the SFT loss? If so, how does training avoid reinforcing incorrect steps? If not, how does the implemented loss correspond to Equation (11)?