- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes CRL for initially weak LLMs, combining progressively shortened reference prefixes with adaptive prefix-loss weighting and reinforcement of successful generated infixes. Experiments on Qwen3.5-0.8B and 2B show improved mathematical reasoning performance and training efficiency.
Critical strengths
- The paper addresses the practical cold-start problem in RL training and presents a relatively simple approach for incorporating reference solutions into policy optimization.
Critical weaknesses
- Limited novelty. CRL closely builds on existing prefix-based approaches such as UFT and R³. The adaptive weighting and infix reinforcement appear to be incremental modifications, with limited additional methodological insight.
- Modest performance gains. The aggregate improvements over SFT-RFT are only 0.71 and 0.61 percentage points, and CRL does not consistently outperform strong baselines across individual benchmarks. These results provide limited evidence of a substantial practical advantage.
- Limited evaluation scale. Experiments cover only 0.8B and 2B models from the same family. Although these models fit the motivation of weak initial policies, the evaluation does not establish whether the benefits extend to larger models facing similarly difficult tasks.
Questions for authors
- Do the authors have evidence that CRL remains effective on larger models in settings where their initial success rates are low?
- Does the preference for a 10% infix persist when the effective auxiliary-loss strength is controlled across different infix lengths?