- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes DOSD, an adaptive on-policy self-distillation method that adjusts the amount of reference reasoning exposed to the teacher based on a per-problem FIFO queue of verified student outcomes. The student generates unassisted rollouts, while the teacher provides token-level supervision conditioned on a difficulty-dependent reference prefix. The paper presents theoretical analyses of exposure stability and gradient variability, along with experiments on mathematical reasoning benchmarks using Qwen3 models.
Critical strengths
DOSD introduces a lightweight, training-free controller for allocating privileged teacher context based on historical student performance, without additional online competence rollouts. The paper also provides a careful theoretical characterization of exposure changes and outcome-conditioned gradient variability.
Critical weaknesses
W1. Insufficient justification for prefix-based context selection.
DOSD controls teacher context by truncating the reference solution from the beginning according to a difficulty-dependent token ratio (Eq. 5). However, informative reasoning steps may appear anywhere in a solution, and shorter prefixes may omit the most critical information. The paper provides neither empirical evidence that prefix length is an effective proxy for supervision quality nor comparisons with alternative context selection strategies. This weakens the motivation for the proposed allocation mechanism.
W2. Insufficient experimental validation of the proposed mechanism.
The improvements over OPSD are marginal for the 1.7B and 4B models (+0.44 and +0.36 points), while performance decreases for 8B (−0.78 points). Although DOSD outperforms the reported budget-calibrated fixed-prefix baseline in the 8B-thinking setting, the actual context budgets are not provided. Moreover, the experiments lack multiple seeds, history-specific ablations, and direct comparisons with existing adaptive exposure methods. Consequently, the evidence is insufficient to establish the benefits of the proposed difficulty-based allocation.
Questions for authors
Q1. Have the authors considered alternative context selection strategies, such as selecting informative reasoning steps or using middle/random segments instead of fixed prefixes? How does the choice of prefix truncation affect performance under matched token budgets?
Q2. Can the authors provide controlled ablations isolating the contribution of historical outcome memory (e.g., current-only or shuffled-history allocation), together with actual context budgets and multi-seed results? These would help determine whether the reported improvements arise from the proposed difficulty-aware mechanism.