- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper studies whether an agent’s own capability can help decide which training data to use for post-training. The authors design methods for both SFT and RL to select better training data, which also improve the post-training results.
Critical strengths
-
They design separate methods for SFT and RL, and both work and improve the training quality.
-
The practical selector in the RL setting does not need verified rewards during selection, which is very useful in practice.
Critical weaknesses
-
The paper provides a code repo, but it only contains a README and is almost empty. I cannot find the supporting evidence mentioned in the paper, such as the prompts in Appendix F and the other artifacts, to verify that the results are sound. This also makes me concerned about whether the authors treat this submission seriously.
-
Both the SFT and RL experiments use a single fixed seed with only a few runs on a single dataset, which cannot demonstrate the significance of the authors’ methods.
Questions for authors
- Could you please provide an accessible code repo that contains the artifacts mentioned in the paper?
- For the SFT experiment, only Qwen gets improved but Gemma does not. Could you add at least one more model to demonstrate the significance of your method?
- Both the SFT and RL experiments are tested on only a single dataset. Could you add at least one more dataset for each setting to demonstrate the significance of your method?