- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies the diminishing learning signal in RLVR when training problems become saturated. The authors propose Failure-Prefix Conditioning, which constructs new training prompts by appending prefixes of incorrect reasoning trajectories to saturated problems. Experiments show that the method improves performance where standard RLVR stalls, achieves gains comparable to training on newly collected medium-difficulty problems, and enhances recovery from misleading reasoning prefixes.
Critical strengths
-
The paper identifies an important challenge in RLVR: extracting useful learning signals from saturated problems. Failure-Prefix Conditioning offers a straightforward approach that does not require modifying the RL algorithm.
-
The method substantially outperforms standard RLVR on saturated problems and achieves results comparable to training on newly collected medium-difficulty problems. The comparisons with multiple baselines and ablation studies support its effectiveness.
Critical weaknesses
-
Limited analysis of improvements in reasoning capability versus reliability. Failure-Prefix trains on failures from saturated problems, which may primarily improve reasoning reliability rather than the ability to solve more challenging problems. Improving reliability is itself valuable, but the paper could provide a deeper analysis distinguishing these effects, such as through pass@k, pass^k, and per-problem success rate distributions.
-
Experiments are restricted to a 1.5B model, leaving scalability to stronger models with rarer failures unclear.
Questions for authors
-
How does Failure-Prefix compare with the Medium baseline in terms of pass@k, pass^k, and per-problem success rate distributions? Does it primarily improve reasoning reliability or expand the range of solvable problems?
-
Can combining Failure-Prefix with the harder problems used in the Medium baseline improve both reasoning reliability and problem-solving capability?
Please engage with the authors' rebuttal
Dear reviewers,
Given that we have only a few days until the closing of the rebuttal period, please take into account the effort that the author spent on their rebuttal, and feel free to engage with them (or at least give an acknowledgement that you have read it). Feel free to ignore this message if you have done so.
Best regards,
Your AC.