- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposed a self-distillation method called Shared Residual Distillation (SRD). The method uses the model’s own multiple answer trajectories (peer trajectories for the same task). While each peer answer is unverified, the authors argue that the trajectories may contain useful local information for self-training. In particular, if multiple trajectories agree on the direction of change for the next token probabilty, in relation to a solution to a different problem, then this direction change is retained in the self-training update. They conducted experiments on SRD and compared its performance with GRPO and OPSD.
Critical strengths
-
The paper proposed an interesting self-training method. The main idea of the method is intuitive.
-
The experiments show substantial gains using this method, especially when there is clipping.
Critical weaknesses
-
I would have liked to see an experimental comparison between SRD and some other label-free distillation method (maybe U-OPSD or CoDA). This will help to understand whether it’s really the local information that makes SRD work and assess SRD’s novelty relative to other label-free methods.
-
The paper seems to demonstrate that the procedure as a whole works well experimentally. However, whether its strong agreement is due to better identification of reliable direction, or stronger shrinkage or some other choices in the experiment is unclear.
Questions for authors
-
Can you separate the effect that SRD selects more reliable directions vs. strong shrinkage avoids big intermediary mistakes? Maybe having a control in the experiment where the control matches the magnitudes of changes while randomizing their locations can help separate the effects.
-
Have you compared SRD with other lable-free distillation methods?
3 comments
Thank you for the additional results. These are helpful. In particular, it's interesting to see that Scalar mean itself achieves such good performance already.
The strong scalar-mean result, together with SRD’s further gains, highlights the potential of learning from unverified peers.
Thank you again for your thoughtful feedback and constructive discussion!
Thank you for reviewing our paper and for your valuable feedback!
To W1 & Q2: Other label-free distillation methods (U-OPSD or CoDA)
Our submission already cites and discusses U-OPSD and CoDA. We share the broad question of how to construct training supervision without external labels. These two methods construct supervision using answer-level consensus, whereas SRD uniformly samples unverified peers and aggregates their local residuals without selecting a consensus solution or routing supervision by answer agreement. This distinction concerns how supervision is constructed, rather than simply whether the method uses external labels.
Both U-OPSD [1] and CoDA [2] appeared as arXiv preprints in August 2026, less than two months before the submission deadline. The ICLR 2027 reviewer guidelines [3] do not mandate comparisons with contemporaneous or arXiv-only work. We also had not found a released implementation in U-OPSD's official repository [4] as of November 10. To address your suggestion, we implemented U-OPSD from the paper and completed an additional Qwen3-4B comparison. Where details are unspecified, we use the shared SRD/OPSD setup and will document these choices explicitly. All scores are AIME avg@12 under the same evaluation protocol.
SRD exceeds this label-free distillation baseline by 19.26 percentage points in mean AIME avg@12, with shorter average outputs.
[1] U-OPSD: https://arxiv.org/abs/2608.06296
[2] CoDA: https://arxiv.org/abs/2608.08764
[3] ICLR 2027 reviewer guidelines: https://iclr.cc/Conferences/2027/ReviewerGuidelines
[4] U-OPSD official repository: https://github.com/williamium3000/u-opsd
To W2 & Q1: Separating informative residual directions from strong shrinkage
To separate these effects, we added three controls under the same training settings:
The following results use Qwen3-4B and report AIME avg@12.
SRD retains a clear advantage over all three controls. In particular, permuting the residual values lowers the average from 45.46% to 17.22%, despite preserving their numerical distribution. Thus, the benefit cannot be explained by the residual magnitudes or sparsity alone: their assignment to particular vocabulary coordinates matters. Uniformly shrinking the mean residual performs substantially better than permutation, but still trails SRD by 7.69 percentage points. Together with the signed-mask mean ablation in the submission, this supports retaining the full combination of shared direction and conservative magnitude selection.
The cross-pair control reaches 19.45%, close to the base model's 19.07%. Keeping the residual construction while removing the problem–peer correspondence does not reproduce SRD's improvement. This provides additional evidence that the supervision uses problem-related peer content, rather than merely the presence of extra solution text.
We will add the results and implementation details to the revision.
We would be grateful if you could let us know whether our response addresses your concerns. We are happy to discuss any remaining issues.