acceptodds
Withdrawn from ICLR 2027

Developing a Crystal Structure Prediction Algorithm with Agent-Guided Search

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Language-model agents show promise for machine learning research, but evidence that agent-guided research can advance the state of the art on established tasks remains limited. We investigate this possibility in crystal structure prediction (CSP), a central task in materials discovery that combines affordable experimentation with a comparatively underexplored generative modeling landscape. An agent-guided search explores eighteen generative framework–architecture pairs and identifies masked generative modeling as the strongest approach under the initial training budget. Subsequent experimentation, combined with four targeted human interventions, produces MaskGXT, a masked generative crystal transformer that jointly predicts crystallographic symmetry, lattice parameters, and atomic coordinates. MaskGXT incorporates circular ordinal label smoothing, sub-bin coordinate refinement, and space-group-stratified sampling to address periodic geometry, coordinate precision, and polymorph coverage. Our experiments show that MaskGXT achieves the highest match rates compared to an extensive set of CSP baselines on MP-20 and MPTS-52 benchmarks.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Crystal generative model, Crystal structure prediction, AI-guided search, generative model, masked diffusion model

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_YwMKOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
3
Read review
Paper summary

This paper documents an agent-human workflow to improve crystal structure prediction. At the end of this agent-human collaboration, the authors introduce MaskGXT, a crystal structure prediction model that matches or slightly outperforms the current SOTA from earlier this year.

Critical strengths

A few strengths of this paper:

  1. I think this paper is interesting in this moment as more and more research is done through agent-human collaborations, and I appreciate how this paper acknowledges and reports that.
  2. The ablations are informative—because the authors did lots of experiments through agent experimentation, there is more rigorous accounting of the effects of different methods than in many other works.
  3. The figures are well done and the text is comprehensible.
Critical weaknesses
  1. The method they ultimately report only barely beats the state of the art from earlier this year, and only on select metrics and subsets of standard benchmarks.
  2. The paper reads more as a research diary than major research contribution. The authors mix together a variety of approaches introduced by others to optimize training workflows for this problem. I think I would be more convinced if this were an underserved research area or if the improvements were significant, but this is a very active field and the methods introduced seem more like hyperparameter optimization than any major insight.
Questions for authors

I don’t really have many questions as the paper is clear, and I don’t doubt the integrity of the experiments. I think that in the era of agent-enabled work, though, a more significant contribution is required in order to be accepted to ICLR main track.

Reviewer_E7hxOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper explores the use of agent-guided search for developing new crystal structure prediction algorithms. Through automated experimentation and targeted human interventions, the authors develop MaskGXT, a masked generative model that achieves competitive performance on several CSP benchmarks. I find the exploration of agentic frameworks for scientific algorithm development valuable and promising, as it demonstrates the potential of human-agent collaboration in advancing scientific research. However, I have several concerns regarding the methodological novelty and experimental evaluation.

Critical strengths
  1. The paper studies an interesting and timely problem, namely whether agent-guided search can help develop scientific algorithms rather than only assist with coding or experimentation.
  2. The final model, MaskGXT, shows competitive performance on several CSP benchmarks, and the polymorph setting is particularly relevant for evaluating diversity and coverage.
  3. The paper provides a concrete human-agent collaboration workflow, which makes the work more interesting than a standard CSP model paper.
Critical weaknesses
  1. The paper presents agent-guided search as a main contribution, but Section 3 feels a bit too brief given its importance. Since the framework builds heavily on AIDE, the paper should more clearly distinguish what is inherited from AIDE and what is newly introduced here.

  2. Figure 2 shows that performance improves along the search tree, but from the figure it is not very clear how the agent actually interprets human insights and translates them into meaningful design changes. Continuing with the masked generative path also seems like a fairly straightforward choice once it shows a clear numerical advantage.

  3. The space-group-stratified sampling strategy is interesting, but I am not fully convinced that it really addresses polymorph diversity. Different polymorphs can share the same space group, while the current strategy generates only one candidate per selected space group. This may limit diversity within the same space group.

  4. The model explicitly predicts space groups and Wyckoff positions, but these are not guaranteed to be consistent with the generated coordinates. Table 5 shows that only 86.4% and 83.9% of the generated structures have their predicted space groups confirmed by spglib. This is worth discussing more carefully, especially in terms of structural validity and final prediction quality.

  5. The reported sampling speedup over Crystalite is impressive, but the comparison uses different decoding budgets (150 vs. 400 steps). A more direct quality-efficiency comparison under matched computational budgets would make this part more convincing.

Questions for authors
  1. Can the authors clarify more explicitly which parts of the agent-guided search framework are inherited from AIDE and which parts are new in this work?
  2. How does the agent incorporate human-provided insights in practice, and can the authors provide more detail in the main text rather than only in the appendix?
  3. In the polymorph benchmark, how often do multiple reference polymorphs share the same space group, and how does MaskGXT perform in those cases?
  4. Can the authors provide a more direct comparison between MaskGXT and Crystalite under matched sampling budgets, for example using match rate versus runtime or number of function evaluations?
Reviewer_jySROpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper uses AI agents and human guidance to explore different generative models for crystal structure prediction. The authors develop MaskGXT, which uses masked modeling, crystal symmetry, and coordinate refinement to improve crystal structure generation.

Critical strengths

The paper explores the use of AI agents with human guidance to design and improve generative models for crystal structure prediction.

Critical weaknesses

see my question

Questions for authors
  1. When and under what conditions are human interventions introduced during the agent search?
  2. The reported DiffCSP results seem inconsistent with the original paper, which makes me question the reliability of the overall results. Could the authors explain this difference?
  3. The main contribution is unclear, as the agent framework seems to be based on existing methods. If the goal is to show the effectiveness of AI agents, why not compare different agent frameworks? Also, the final model does not outperform existing methods on all metrics.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.