- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper builds on on-policy self-distillation (OPSD; Zhao et al., 2026). A frozen copy of the initial model, given the reference solution, re-scores the prefixes of the student's own rollout, and the student matches this distribution with a clipped full-vocabulary forward KL. PAINT adds two components:
- Overlap-adaptive suffix masking. Anchors are extracted from the reference solution (the boxed answer, formula-like spans, numbers), and is the fraction of them that appear in the rollout. A character-level suffix of the reference is then hidden, with ratio .
- Sparse energy interpolation. At the top 3% of positions per sample by teacher/student entropy ratio , the teacher target is replaced by .
The theory consists of Prop 3.1 (a conditional mutual information view), Thm 3.2 (monotonicity of the mask ratio), Thm 3.3 (suffix placement), and Prop 3.4 (the geometric mixture as a KL barycenter).
Experiments train Qwen3-1.7B/4B/8B with LoRA for 100 steps on OpenThoughts math, using one rollout of up to 1024 tokens per problem, and report Avg@12 on AIME24, AIME25, and HMMT25. PAINT improves macro Avg@12 over OPSD by 0.8–2.1 points and over GRPO by 1.7–6.8 points. All ablations are on Qwen3-1.7B, AIME24 only.The paper builds on on-policy self-distillation (OPSD; Zhao et al., 2026). A frozen copy of the initial model, given the reference solution, re-scores the prefixes of the student's own rollout, and the student matches this distribution with a clipped full-vocabulary forward KL. PAINT adds two components:
- Overlap-adaptive suffix masking. Anchors are extracted from the reference solution (the boxed answer, formula-like spans, numbers), and is the fraction of them that appear in the rollout. A character-level suffix of the reference is then hidden, with ratio .
- Sparse energy interpolation. At the top 3% of positions per sample by teacher/student entropy ratio , the teacher target is replaced by .
The theory consists of Prop 3.1 (a conditional mutual information view), Thm 3.2 (monotonicity of the mask ratio), Thm 3.3 (suffix placement), and Prop 3.4 (the geometric mixture as a KL barycenter).
Experiments train Qwen3-1.7B/4B/8B with LoRA for 100 steps on OpenThoughts math, using one rollout of up to 1024 tokens per problem, and report Avg@12 on AIME24, AIME25, and HMMT25. PAINT improves macro Avg@12 over OPSD by 0.8–2.1 points and over GRPO by 1.7–6.8 points. All ablations are on Qwen3-1.7B, AIME24 only.The paper builds on on-policy self-distillation (OPSD; Zhao et al., 2026). A frozen copy of the initial model, given the reference solution, re-scores the prefixes of the student's own rollout, and the student matches this distribution with a clipped full-vocabulary forward KL. PAINT adds two components:
- Overlap-adaptive suffix masking. Anchors are extracted from the reference solution (the boxed answer, formula-like spans, numbers), and is the fraction of them that appear in the rollout. A character-level suffix of the reference is then hidden, with ratio .
- Sparse energy interpolation. At the top 3% of positions per sample by teacher/student entropy ratio , the teacher target is replaced by .
The theory consists of Prop 3.1 (a conditional mutual information view), Thm 3.2 (monotonicity of the mask ratio), Thm 3.3 (suffix placement), and Prop 3.4 (the geometric mixture as a KL barycenter).
Experiments train Qwen3-1.7B/4B/8B with LoRA for 100 steps on OpenThoughts math, using one rollout of up to 1024 tokens per problem, and report Avg@12 on AIME24, AIME25, and HMMT25. PAINT improves macro Avg@12 over OPSD by 0.8–2.1 points and over GRPO by 1.7–6.8 points. All ablations are on Qwen3-1.7B, AIME24 only.The paper builds on on-policy self-distillation (OPSD; Zhao et al., 2026). A frozen copy of the initial model, given the reference solution, re-scores the prefixes of the student's own rollout, and the student matches this distribution with a clipped full-vocabulary forward KL. PAINT adds two components:
- Overlap-adaptive suffix masking. Anchors are extracted from the reference solution (the boxed answer, formula-like spans, numbers), and is the fraction of them that appear in the rollout. A character-level suffix of the reference is then hidden, with ratio .
- Sparse energy interpolation. At the top 3% of positions per sample by teacher/student entropy ratio , the teacher target is replaced by .
The theory consists of Prop 3.1 (a conditional mutual information view), Thm 3.2 (monotonicity of the mask ratio), Thm 3.3 (suffix placement), and Prop 3.4 (the geometric mixture as a KL barycenter).
Experiments train Qwen3-1.7B/4B/8B with LoRA for 100 steps on OpenThoughts math, using one rollout of up to 1024 tokens per problem, and report Avg@12 on AIME24, AIME25, and HMMT25. PAINT improves macro Avg@12 over OPSD by 0.8–2.1 points and over GRPO by 1.7–6.8 points. All ablations are on Qwen3-1.7B, AIME24 only.
Critical strengths
1. A natural question and a nearly free method. PAINT needs no extra generation, only a masked teacher prompt and a target interpolation at a few positions. If the results hold, two findings are practically useful: moderate suffix masking beats the full solution while heavier masking hurts (Table 2), and suffix masking beats middle and prefix masking (Table 3).
2. Consistent direction. PAINT is above OPSD in all nine benchmark–scale cells of Table 1. If the comparison is controlled, this is the strongest evidence in the paper.
Critical weaknesses
1. The comparison may not be controlled, and the gains are within noise.
Baselines. All 48 baseline numbers in Table 1 (No update, SFT, GRPO, OPSD × 3 scales, including the macro average) are identical to Table 2 of the OPSD paper (arXiv:2601.18734v3). The paper cites Zhao et al. (2026) for the method but never states that these numbers are taken from it. The text presents them as the authors' own runs, and OPSD as run in the same pipeline as PAINT: see the Table 1 caption (same training split, rollout budget, and fixed privileged teacher), §4.1 ("We evaluate four updates from the same base checkpoints"), and App. B (the vanilla baseline keeps the same training budget and teacher anchoring). In the OPSD paper, the OPSD numbers are the best of checkpoints evaluated every 20 steps up to step 100, and the GRPO numbers are the peak within 500 steps (v3, Table 2 caption). These are checkpoints selected on the test benchmarks. The ablation reference value of 57.2 (Table 2, first row; §4.3) is the same number.
Selection. For PAINT, App. B says only that checkpoints were evaluated every 20 steps up to step 100. It does not say which checkpoint is reported. With no validation split mentioned, the reported checkpoint was likely chosen on the test benchmarks. The final hyperparameters are also the best of the 15 PAINT variants in the ablations, all on 1.7B AIME24. On 1.7B, the gain on this tuning benchmark is +2.5, versus +0.3 and +0.5 on the other two.
Noise. With 30 problems × 12 samples, sampling noise alone gives a standard error of per benchmark (: per-problem accuracy). That is roughly 1–2 points, and at most about 2.6. The standard error of a difference between two models is about times larger. The OPSD paper itself shows fluctuations of this size: within one run, its 1.7B AIME25 score was 43.9 at step 50 and 41.1 at step 100 (v3, Table 3, forward-KL row). Seven of the nine gains in Table 1 are 1.4 points or less, and the ablation gains are 1.4 (masking) and 1.1 (interpolation). No seeds, confidence intervals, or training curves are reported.
Why this matters. The central claim that PAINT outperforms OPSD is not established by the current evidence. The 9/9 consistency is suggestive, but what it means depends on how the baselines were obtained and how checkpoints were selected.
2. The mechanism is not isolated, and the theory does not support the design.
Adaptivity vs. average mask ratio. The adaptive range is , but Table 2 has no fixed ratio in that range (0.30, 0.35, 0.40). The fixed ratios are only 0, 0.25, 0.50, and 0.75, and there is no reversed schedule. The +0.8 over fixed 0.25 may simply reflect a better average ratio. The distribution of is not shown either. If rollouts are cut at 1024 tokens before reaching the later anchors (including the boxed answer), may measure how far the rollout progressed rather than how closely it followed the reference.
Confounded with hiding the answer. A 30–40% suffix mask usually hides the final answer at the end of the solution, while prefix and middle masks leave it visible. The paper's own Fig. 2 example hides everything after , so the answer is not shown, yet the prompt asks the teacher to "derive the same final answer." The OPSD paper attributes SFT's degradation to the reference solutions being concise (v3, Table 2 caption and §4.2), which suggests is short and ends with the answer. Tables 2 and 3 are therefore consistent with two explanations. One is Thm 3.3's (keep the opening setup visible). The other is simpler (do not show the teacher the answer).
Theory–method gap.
- Prop 3.1 is the definition of conditional mutual information plus data processing.
- Prop 3.4 is the standard geometric-mixture / KL-barycenter identity. Moving the target toward the student lowers the loss at that position by construction, so it is not evidence that training improves. It also concerns the unclipped KL, whereas the loss actually used (Eqs. 3 and 12) is clipped.
- Thm 3.2 is a result for a posited surrogate objective, not one derived from the training objective. Nothing links the character-level mask ratio to the mixture weight, or to , so it cannot justify the direction of the schedule.
- Thm 3.3 follows directly from Assumption D.8, which is not tested. By Lemma D.7, , so the assumption is equivalent to the conclusion.
- The interpolation shifts the target 3% of the way toward the student at 3% of positions, a very small change. Its only support is a 0.8–1.1 point difference on one benchmark.
Why this matters. "Information allocation" is presented as the explanation for the gains, but it is not distinguished from simpler explanations (the average mask ratio, hiding the answer).
Questions for authors
Clarification requests (answers could change my assessment)
Q1. Baselines. Did you run the No update / SFT / GRPO / OPSD baselines yourselves, or take the numbers from the OPSD paper? If they were taken, please say so. At minimum, please re-run OPSD with the same code, seeds, and checkpoint rule as PAINT.
Q2. Selection and variance.
- Was the PAINT checkpoint chosen per benchmark or per model?
- Were the hyperparameters chosen on 1.7B AIME24?
- Please report all five checkpoints (steps 20–100) with at least three seeds (or, at minimum, bootstrap CIs), and select hyperparameters on held-out data.
Q3. What the teacher sees.
- Is a short solution or a long reasoning trace?
- For each mask type, how often is the final answer visible to the teacher?
- A mask that hides only the answer would directly test the answer-hiding explanation.
Q4. The overlap signal.
- Please report:
- the distribution of ;
- the fraction of examples with no anchors;
- whether training rollouts use thinking mode;
- the fraction of rollouts hitting the 1024-token cap;
- the fraction containing a boxed answer;
- the correlation between and correctness.
- Please also add fixed = 0.30, 0.35, 0.40 and a reversed schedule to Table 2.
Q5. Selected positions and clipping.
- How is handled when is close to 0?
- Are the selected positions mostly formatting/LaTeX tokens or reasoning decisions?
- Eq. (3) clips each vocabulary term at , and a term can only be clipped when . A clipped term has no direct gradient. Through the softmax normalization, the loss gradient on that logit is, up to a positive constant, (: the clipped set). So these tokens, which the teacher prefers, are only pushed down. Since interpolation changes which terms are clipped, please report the clipping frequency at selected vs. unselected positions.
Suggestions
- Relate the work to hint / partial-solution methods:
- QuestA (Li et al., ICLR 2026) adds partial solutions to the prompt during RL.
- UFT (Liu, Farina & Ozdaglar, NeurIPS 2025) uses solution prefixes as hints.
- Guide (Nath et al., 2025) adds guidance only when all unguided attempts fail.
- Explain why the cited self-distillation methods (Hübotter et al., 2026; Shenfeld et al., 2026) are not compared.
- Test the motivating claim that the full solution concentrates the target on a single trace, e.g., with pass@k, output entropy, and response length.
- Describe the GRPO setup, and compare compute including the teacher prefill.