- Rating
- 2
- Confidence
- 4
- Informativeness
- 2(+1)
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
OpenSIR studies verifier-free self-improvement for mathematical reasoning. One policy generates and solves questions, using majority-vote pseudo-answers and rewards based on solvability, solution length, and embedding distance from earlier questions. The paper reports gains on mathematical and general-reasoning benchmarks across four post-trained models. The results support the recipe in the tested setting.
Critical strengths
-
The paper addresses the important challenge of improving mathematical reasoning without a large labeled question pool or an executable verifier. Its shared-policy training loop is concrete, and the reported gains span four starting models.
-
Diversity and teacher-training ablations, seed sensitivity, and pass@k analyses give useful evidence that the full recipe works in the tested mathematical setting.
Critical weaknesses
-
The incremental contribution is not yet convincing to me. Seed-based augmentation and embedding diversity have precedents in DaaR; RL-based mathematical question synthesis with complexity and answer-consistency signals appears in MathSmith; and Absolute Zero and R-Zero already explore self-play problem generation. OpenSIR's verifier-free shared-policy combination may be useful, but the paper does not isolate a sufficiently distinctive insight from it.
-
The central scores are defined mathematically but insufficiently validated as proxies for what the paper claims to reward. Majority answer agreement does not establish answer correctness or a well-posed question; longer solutions need not mean harder reasoning; and question-embedding distance need not imply a new solution skill (pages 3–4). Table 3 calls external-model agreement “validity,” while the topic and embedding analyses do not rule out familiar templates (pages 7–9). With a shared generator/solver and a pool containing every format-valid question, independent checks of question quality, pseudo-answers, and skills are needed to support the proposed mechanism.
-
The strongest efficiency and open-endedness claims outpace the experiments. The “exactly 2×” cost claim counts selected update rollouts, while Algorithm 1 samples solutions for candidate questions before selection; total generated tokens and equal-compute results are missing (pages 22–23). Performance plateaus after roughly 300 steps in the longer run, so sustained open-ended discovery is not shown. A per-example evaluation-overlap audit is also absent; notably, the separate AIME 2024 seed ablation evaluates on AIME 2024 (page 16). These gaps limit the interpretation of the gains as efficient, sustained transfer.
Questions for authors
-
What is the specific insight beyond the closest data-diversity and mathematical-synthesis methods? Could you separate the diversity reward from the prompt's instruction to generate a conceptually different question, and test whether it yields distinct solution skills at a matched question count?
-
On a representative sample across training stages, could you independently assess well-posedness, pseudo-answer correctness, and solution-skill novelty? How often do mathematically invalid questions remain in the historical pool, and how do length and embedding-distance scores relate to these checks?
-
Could you report total candidate questions, solution rollouts, generated tokens, and GPU hours, clarify what the 2× cost figure includes, and provide an equal-compute comparison? Please also report per-example evaluation-overlap checks and whether the AIME 2024 seed ablation's result changes after excluding that seed from its AIME 2024 evaluation (Appendix B.5, B.9).