- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes a post-training framework that uses in-context learning as a training signal to encourage transferable reasoning. The model first generates a response to an initial question, and this question–response pair serves as an in-context demonstration for a set of subsequent probe questions. The initial response is rewarded according to the model’s performance on these probes.
Critical strengths
Using reinforcement learning to improve the usefulness of model-generated in-context demonstrations is a valuable research direction.
Critical weaknesses
- The contribution of the initial response is not sufficiently established. Beyond the intuitive explanation based on in-context learning, the paper does not clearly demonstrate why or how much the initial response helps solve subsequent questions. Concrete examples and direct empirical evidence would strengthen the proposed motivation.
- The experimental evaluation is limited in scope. The experiments rely substantially on toy reasoning tasks, and model coverage is narrow, with only small models (<2B, non-thinking).
Questions for authors
- The actual teaching effect of the first response remains insufficiently demonstrated. Could the authors directly evaluate the teaching effect of the initial response by comparing performance and model behavior with and without a Gen-RL-style prefix, as well as across different candidate prefixes?
- What are the main advantages of the proposed framework over approaches that extract reusable skills or consolidate knowledge into a knowledge base?
- The compute-matched (G=72) control generally performs worse than the (G=8) control. Could the authors explain this counterintuitive result? Could insufficient hyperparameter tuning of the (G=72) baseline contribute to the performance gap?