- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies on-policy self-distillation (OPSD), where the same model acts as student and teacher and only the teacher also sees a verified reference solution. It argues that conditioning the teacher on one complete reference concentrates its predictions on that single trajectory, and proposes MASS, which builds several teacher views from the same reference by masking different spans.
On AIME24, AIME25 and HMMT25 (Avg@12), MASS is the best method in all 12 model-benchmark cells, and its three-benchmark average is 1.5-2.5 points above the strongest baseline, AVSD or RLSD. It also reports ablations and OOD results on GPQA and BBH.
Critical strengths
-
MASS uses no more data or rollouts than OPSD and gives consistent gains at moderate cost. It is the best method in all 12 model-benchmark cells (two model families, 1.7B-8B), and its estimated per-step cost is 2.6x OPSD versus 6.4x for RLSD. On Qwen3-4B it keeps improving up to step 250, whereas OPSD falls below the base model.
-
The token weight is derived explicitly, and its limits are stated. Its population version minimizes the mean-squared error of the weighted correction plus a small ridge term and a linear anchor bonus. Remark A.8 states where this plug-in weight departs from the optimum.
Critical weaknesses
-
The margins are small, and no variance estimates are given. No seeds, confidence intervals or significance tests are reported, so all results appear to come from single training runs. On datasets such as AIME and HMMT, a difference of one or two problems can easily inflate the metrics. Each benchmark has 30 problems x 12 samples, so the per-benchmark margins over the strongest baseline in Table 1 (0.83-2.8 points) correspond to 3-10 of 360 responses, and many ablation gaps to 1-5 responses.
-
The checkpoint-selection rule and the baseline settings are not stated. Appendix C gives no hyperparameters, RLSD group size or tuning budget for the baselines.
-
The benefit of averaging several masks is not tested. Without the weight and adaptive masking, one random mask per step gives the same expected update as the average of K masks (Lemma A.1, Lemma A.4), so averaging helps only by lowering target variance. No K = 1 baseline is reported to check this (Fig. 5(a) covers K = 2, 4, 8).
-
Fig. 1 does not distinguish broader support for other correct solutions from a teacher that has simply become less strict. Because the averaged teacher keeps any answer that at least one view supports, its PI Bias Score [1] for any continuation, correct or not, is never below the single-view average (Prop. A.3). An incorrect-solution control is therefore needed to show that the rise for a different correct solution is specific to correct solutions. The in-context score also falls when several masks are used, which seems inconsistent with this bound, and the number of problems and the uncertainty are not reported.
[1] S. Harne et al. Privileged, but biased: How PI-conditioned teachers break self-distillation. arXiv:2608.04794, 2026.
Questions for authors
-
Checkpoints and baselines. Which checkpoint-selection rule was used for each method in Table 1, and were checkpoints selected based on AIME or HMMT results? Please report all methods under a single rule and give the baseline settings, including the RLSD group size and the AVSD configuration.
-
Variance. Please report mean +/- std over training seeds for Qwen3-4B in Tables 1 and 3, together with paired per-problem bootstrap intervals for the differences between MASS and AVSD and RLSD.
-
To support the paper's central claim that MASS supports diverse reasoning paths, please also report Pass@k in addition to Avg@k.
-
Multi-mask control. Please add an unweighted K = 1 random-mask baseline. If K = 8 averaging still clearly wins in this comparison, weakness 3 is largely resolved. Comparisons with suffix masking as in PAINT [1] under the same weight, or with a higher teacher temperature, would also help.
Minor comments:
- The fonts in the figures are generally too small, which hurts readability.
[1] Z. Tan and Y. Hong. PAINT: Partial-solution adaptive interpolated training for self-distilled reasoners. arXiv:2604.26573, 2026.
4 comments
Following up on Weakness 1 and Question 2: Table 1 now reports mean ± std over three training seeds for all four models, not only Qwen3-4B. The table below gives the margin of MASS over the strongest baseline in each cell. ✓ marks a margin larger than both methods' standard deviations; for ✗, the larger standard deviation is shown in parentheses.
| Model | AIME24 | AIME25 | HMMT25 | Avg. |
|---|---|---|---|---|
| Qwen3-1.7B | 1.67 ✓ | 0.84 ✗ (0.89) | 1.11 ✓ | 1.66 ✓ |
| Qwen3-4B | 2.59 ✓ | 0.39 ✗ (0.80) | 2.99 ✓ | 1.99 ✓ |
| Qwen3-8B | 1.67 ✓ | 1.20 ✓ | 1.67 ✓ | 1.72 ✓ |
| OLMo-7B | 1.48 ✓ | 3.15 ✓ | 1.76 ✓ | 2.19 ✓ |
MASS has the highest mean in all 12 model–benchmark cells, and the margin exceeds one standard deviation in 10 of them. The two exceptions, AIME25 on Qwen3-1.7B and Qwen3-4B, are consistent with the paired bootstrap in Table 2, and we do not claim them as resolved. On the three-benchmark average, the margin is more than three times the standard deviation for every model.
With the updated Table 1, the Avg@12 lead on Qwen3-1.7B used in our Pass@k comparison is 1.7 points (previously 1.5), against 7.2 points under Pass@12 (Table 11).
I thank the authors for their response and the revision. The three-seed results with paired bootstrap intervals, the same checkpoint-selection rule for all methods, the baseline settings, the K = 1 control, the incorrect-solution control and Pass@12 resolve most of the concerns raised in my original review, so I am raising my score to 3.
Thank you for the careful review and for raising your score. If any of your original concerns remain open, we would be glad to address them before the discussion closes.
We thank the reviewer for the detailed and constructive review. Each concern is addressed below, with pointers to the revised PDF (changes in blue).
Weakness 1 & Question 2: small margins, seed variance, and bootstrap intervals. To test whether the margins could come from a lucky run, we trained all methods on Qwen3-4B with three seeds (Table 7, Appendix B.1):
Standard deviations are smaller than the margin of MASS over the strongest baseline on AIME24 (2.6 points) and HMMT25 (3.0 points); on AIME25 the gap to AVSD is within one standard deviation, and we do not claim it as resolved. The component ablation is also reported over three seeds (Table 4, §3.3); each component's gain over the variant without either exceeds one standard deviation. To address the concern that one or two problems can drive the result, Table 2 (§3.2) reports paired bootstrap intervals that resample problems rather than responses:
Four of six intervals lie above zero, and the paper reports the remaining two as unresolved. We will extend Table 7 to all models in Table 1 before the end of the discussion period.
Weakness 2 & Question 1: checkpoint rule and baseline settings. All methods use the same rule (§3.1): checkpoints are evaluated every 50 steps and we report the best Avg@12 across them for each benchmark. We chose a common rule over a fixed final step because the methods peak at different times (Figure 3); a fixed step would penalize OPSD, which peaks early. The full AVSD and RLSD settings, including RLSD's group size G=8 and AVSD's three privileged views, are in Table 15 (Appendix C) and follow each method's published main results. We kept AVSD at three views because its views are heterogeneous rather than interchangeable masks, and its authors report no gain from adding views on mathematics.
Weakness 3 & Question 4: K=1 random-mask control. We added the unweighted K=1 baseline you suggested to Figure 5(a) (§3.3). K=8 clearly wins: K=1 improves only until about step 100 and ends below its starting accuracy, while K=8 keeps improving through step 250 and finishes more than 10 points higher. A single random mask with the same expected update therefore does not reproduce the benefit, consistent with the fact that averaging lowers the variance of the teacher target.
Weakness 4 & Question 3: Figure 1 control and Pass@k.
PS incorrect. If multi-mask averaging only made the teacher less strict, the incorrect-solution score would rise with the different-correct score. Instead, moving from a single mask to multiple masks raises it by only 0.002 to 0.003, against 0.040 to 0.063 for the different correct solution, and it stays below the full-reference teacher. The increase for correct continuations is thus more than an order of magnitude larger than for incorrect continuations.We hope these additions resolve the reviewer's concerns, particularly Weakness 3, and would be glad to clarify anything further.