- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper studies when sequence-level feedback in OPSD translates into improved parameter updates and downstream math reasoning performance. The authors introduce RS-OPSD, which combines on-policy self-distillation with verification-gated reference cross-entropy on failed problems. The experiments on Qwen3-4B and Qwen3-8B show improvements of RS-OPSD compared to standard OPSD on math reasoning benchmarks.
Critical strengths
RS-OPSD is simple and easy to implement.
Critical weaknesses
- The paper reads like it was mostly written by AI, without careful check by human. There is clear evidence about this, such as lines 347-350 or the last sentence in the AI use statement. The content lacks intuition and explanation.
- There are several problems with the design of the diagnosis, method, and experiments/evaluation, such as:
- The diagnosis is not connected to the actual method. Moreover, different analyses use different protocols and prefix sets, which makes it hard to have a conclusion about the actual limitations of OPSD.
- The proposed method is not well justified by the results. For instance, in Figure 4, random selection achieves essentially the same performance as error selection.
- The evaluation is very limited: only 1 seed, no same-protocol step-0 scores (lines 351-353).
Questions for authors
See Weaknesses.