- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper documents an agent-human workflow to improve crystal structure prediction. At the end of this agent-human collaboration, the authors introduce MaskGXT, a crystal structure prediction model that matches or slightly outperforms the current SOTA from earlier this year.
Critical strengths
A few strengths of this paper:
- I think this paper is interesting in this moment as more and more research is done through agent-human collaborations, and I appreciate how this paper acknowledges and reports that.
- The ablations are informative—because the authors did lots of experiments through agent experimentation, there is more rigorous accounting of the effects of different methods than in many other works.
- The figures are well done and the text is comprehensible.
Critical weaknesses
- The method they ultimately report only barely beats the state of the art from earlier this year, and only on select metrics and subsets of standard benchmarks.
- The paper reads more as a research diary than major research contribution. The authors mix together a variety of approaches introduced by others to optimize training workflows for this problem. I think I would be more convinced if this were an underserved research area or if the improvements were significant, but this is a very active field and the methods introduced seem more like hyperparameter optimization than any major insight.
Questions for authors
I don’t really have many questions as the paper is clear, and I don’t doubt the integrity of the experiments. I think that in the era of agent-enabled work, though, a more significant contribution is required in order to be accepted to ICLR main track.