- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes Faraday, a 27B AI Scientist trained to perform scientific research replication. The authors introduce Replica, a benchmark consisting of 310 tasks derived from ML and AI-for-science papers, where agents are asked to reproduce experimental results under limited time and computational resources. To evaluate these tasks, the authors develop a rubric-based LLM judge and use it as the reward for GRPO-based post-training. Faraday is trained to use a frontier coding agent as a tool for implementing and running experiments. Experiments show that Faraday outperforms strong coding agents such as Claude and Codex on both in-distribution and held-out tasks. Human studies and qualitative analyses further suggest that the rubric judge aligns reasonably well with human preferences and that Faraday exhibits more rigorous scientific behaviors, such as implementing the underlying experimental mechanisms rather than simply matching expected results.
Critical strengths
-
The research problem has strong practical significance.
This paper focuses on training an AI Scientist capable of conducting scientific research replication. This problem is important for improving the scientific capabilities of large language models, addressing the rapid growth in the number of research paper submissions, and improving the reproducibility of scientific findings. -
The experimental design is comprehensive and convincing.
The authors compare Faraday with strong coding agents such as Claude and Codex, and further conduct a human study to validate the agreement between the rubric judge and human evaluations. In addition, the paper provides several qualitative analyses to further examine Faraday's improved capabilities in scientific research replication.
Critical weaknesses
-
The experimental results rely heavily on the LLM-based rubric judge, leaving some uncertainty about whether the results reflect true scientific capabilities.
Although the authors conduct a human study showing that the rubric judge has good agreement with human evaluations, the judge is not perfectly reliable. Moreover, both the training reward and the final evaluation of Faraday rely on the same judge. Therefore, it remains difficult to completely rule out the possibility that part of Faraday's performance gain comes from adapting to the characteristics of the benchmark and the judge, rather than from genuinely improved scientific capabilities. -
The coverage and generalization of the test tasks remain limited.
Replica mainly focuses on a specific form of scientific task: reproducing a results figure from a given paper. Although these tasks involve a certain degree of openness, they still differ substantially from real scientific research in aspects such as problem formulation, hypothesis generation, experimental design, result interpretation, and the discovery of new knowledge. Therefore, the current experimental results are not yet sufficient to demonstrate that Faraday can generalize to a broader range of open-ended scientific research tasks.
Questions for authors
-
How do the authors rule out the possibility that Faraday's performance improvement is partly due to overfitting to the rubric judge or the specific characteristics of the Replica benchmark?
In particular, have the authors considered evaluating Faraday with an independent judge, human evaluators who are unaware of the model identity, or alternative evaluation criteria that were not used during training? -
To what extent can the capabilities learned from Replica generalize beyond paper replication tasks?
Have the authors evaluated Faraday on other types of open-ended scientific tasks, such as hypothesis generation, experimental design, or scientific discovery, that are not directly based on reproducing a figure from an existing paper?
2 comments
Thank you for your reply. My concerns have been addressed, and I will update my rating to a higher score.
Thank you for your careful review and useful feedback.
W1/Q1 – Potential overfitting to the judge
We accept that using the same rubric judge for training and evaluation was a weakness of the paper. Our human studies were a step towards validating this approach; our evaluators were unaware of the model identity. Additionally, we have run a suite of additional experiments to demonstrate that the improvement in capabilities that arises from our training regime generalises to independent judges. We invite you to take a look at the shared response above and let us know if these suitably address your concerns or if you would like to discuss further.
W2/Q2 – Task coverage and generality
Displaying generalisation to broad, open-ended scientific discovery is out of scope of this paper. We felt that constraining to a more open-ended variant of paper replication than in previous work would be more than sufficient as a first step, and that the quest for fully creative discovery of new knowledge should inevitably continue in future work. We discuss this in the opening paragraph of Section 5 in the paper.
However, we did include some preliminary experiments exploring early methods towards innovation in Figure 5. While we would not necessarily expect generalisation beyond our replication tasks, we do see some signs of it, including on the perturbed tasks (Figure 5, right), on the much larger compute budget experiments (Appendix A.1), and on our new PaperBench results (see the shared rebuttal response).
The contribution of this paper, though, centres on showing that it is possible to train with RL on highly underspecified, open-ended tasks, and identifying the ingredients we needed to make this work (e.g. judge access to the agent's whole trajectory, multiple judge samples for sufficient SNR, judge-generated turn-level credit weights). We view replication as an initial step towards training for creativity and innovation.