- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper investigates why R-Zero-style self-evolution suffers from performance collapse after multiple rounds. The authors identify two problems, invalid questions being increasingly selected by answer-consistency filtering, and repeated mathematical tasks that cannot be detected by lexical similarity. R-Quest introduces validity-aware solver training and semantic novelty assessment to improve generated question quality. Experiments show better results on math, code and general reasoning, and more stable performance over ten rounds.
Critical strengths
- The analysis of performance collapse is interesting. Especially the finding that answer-consistency filtering actually increases the proportion of invalid questions. The repaired-question experiment in Fig. 3 provides some evidence that these invalid questions really hurt training.
- The long-term results are quite good. I also like the observation in Fig. 7 that stronger novelty control does not always help. Too much diversity may also prevent learning harder variants of existing problems. This part makes sense.
Critical weaknesses
- One concern is what really prevents the performance collapse. R-Quest uses GPT-5.6-Sol to construct validity training data, and keeps replaying these annotated samples during every round of solver training. So there is still external supervision continuously involved. It is unclear how much of the improvement comes from this replay and how much from the proposed online validity feedback. The existing ablations do not clearly separate these effects. This matters particularly when the main failure of R-Zero is accumulating incorrect pseudo-labels.
- Another issue is the validity assessment itself. From Fig. 6(a,b), the final balanced accuracy and invalid recall imply that recall on valid questions is only around 60%. So quite many valid questions are also rejected. I wonder whether difficult but valid questions are more likely to be rejected. Also, Fig. 6(d) compares pseudo-label accuracy on questions generated by different methods. These question distributions could have very different difficulty. Higher accuracy may partly come from generating easier or more predictable questions, rather than improved solver reliability. The current evidence cannot fully distinguish these explanations.
Questions for authors
-
Can you compare R-Quest with a baseline using the same validity-initialized solver and 10% replay data, but without the online validity gate? Does it still collapse after ten rounds?
-
In Fig. 6, the valid-question recall seems quite low. What is the false rejection rate under the actual nine-response majority gate? Are the rejected valid questions systematically more difficult?
-
For Fig. 6(d), can you evaluate the pseudo-label accuracy of both solvers on the same set of valid generated questions? This would help understand whether the improvement mainly comes from better question selection or better solver supervision.
I thank the authors for the thorough rebuttal and the extra compute put into addressing my concerns.
⚬ Multi-seed stability: The 3-seed runs (Figure 9, Table 17) confirm the stability gains and margins over baselines are solid and well beyond run-to-run noise. Though it would be nice to see per-benchmark score+spread.
⚬ Validity mechanism: The new ablation (R-Zero + Init. + Replay dropping to 27.90% in Table 13) is convincing. It shows that replay alone doesn't fix things; the online validity gate is what actually stops the questioner from exploiting bad questions.
⚬ Internalization vs. external verifier: Table 18 neatly proves that internalizing validity matches a strong external judge while replacing a recurring ~$1,400 API cost with a one-time ~$200 setup.
The paper is stronger now empirically. Score raised.