- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper studies the prerequisites for efficient RLVR: pretraining or midtraining with the so-called source examples are needed to obtain the operation rules. Furthermore, the paper suggests that the model needs to retrieve information from the proper memory and use it to make a decision. Therefore, the source examples teach retrieval and the rewards teach the decision rule.
Critical strengths
- This paper studies how pretraining and midtraining influence the follow-up RL training, and provides theoretical characterizations and supporting experiments on how the source observations resolve the ambiguity where identical training rewards are obtained but different test answers are required.
Critical weaknesses
- The presentation of the paper is poor.
- Many notions are used before introduction. For instance, it is unclear the meaning of Line 32 “Prediction may supply information absent from rewards, build a computation that uses it, or already teach part of the task.”
- From the description in the second paragraph alone, it is not perfectly clear to me what is source prediction. In addition, it is also not clear to me how Figure 1 explains this notion.
- This paper lacks comparison with the existing works like [1] and the novelty of the research needs to be justified.
[1] Yuan, et al. From f(x) and g(x) to f(g(x)): LLMs Learn New Skills in RL by Composing Old Ones. 2026.
Questions for authors
- Can the authors compare their conclusions with [1]? [1] indicates that the atomic operations are needed before RL training, and RL training enables composition operations of these atomic operations.