- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The authors introduce RL-environments which aim to train LLM agents to perform AI research. They focus on generalizing from low-fidelity to high-fidelity settings. Low-fidelity here means simple environments and high-fidelity means complex environments, closer to frontier research. Although the high-fidelity environments are also very simple / small-scale.
Critical strengths
None
Critical weaknesses
- very simplistic environments, with only 1 tool and maximum 5 steps
- the high-fidelity settings are not really high-fidelity, but still quite small. E.g. they use models of up to 537M parameters in the pre-training task and GSM8K in the GRPO task.
Questions for authors
- How could you create stronger high-fidelity settings, as opposed to having simple high-fidelity baselines as in the current setting?
- How could the setting be made open-ended? Right now the action space of the agents is very limited.