- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
The paper proposes a compositional framework for crystal space exploration and generation. Generation is done through known diffusion processes. Crystal concepts are extracted through a VQ-VAE. These are then used for composition through a diffusion model with the generator being restricted to valid compositions. The main contribution is to bring together well known techniques for this task. The chosen task is not novel as people have been attempting this for at least 3 years. The validation is done on the MP20 dataset by collecting 1000 crystals in each sampling run. Then they perform another test by training on MP-20 and seeing how much of the ALEX MP 20 data they can recreate.
Critical strengths
None
Critical weaknesses
• The paper seeks to discover new crystals. But the methodology , encoding representation and generation are quite standard. I derived no new insights into how useful concepts are generated as compared to prior methods and why. • The most important part is to show new crystals can be discovered. They used the training set MP 20 crystals to evaluating stability, novelty etc using VSUN. What were the 1000 crystals found? Were they really new (not in terms of distance in structure but in material science terms?)? What were the material scientist view on these structures? These are not given, or even discussed. This is important in this context.
Questions for authors
Many aspects are poorly formulated –(eg. Derivative reference used for DFT). What is the utility of Figure 2, appears to be of little value to the understanding
3 comments
Rebuttal 2
Q3: The most important part is to show new crystals can be discovered. They used the training set MP 20 crystals to evaluating stability, novelty etc using VSUN. What were the 1000 crystals found? Were they really new (not in terms of distance in structure but in material science terms?)? What were the material scientist view on these structures? These are not given, or even discussed. This is important in this context.
We thank the reviewer for raising this important question.
- Clarification of novelty evaluation. First, we clarify that novelty in our final V.S.U.N. evaluation is not measured against the training set. Instead, we use substantially larger reference databases: the Materials Project (MP, February 7, 2023 snapshot, 154,718 structures) for MP-20, and MP + Alexandria (845,997 structures) for Alex-MP-20. These evaluations follow MatterGen.
- Novelty beyond structural distance. To further assess the materials science significance of the generated crystals, we conduct additional analyses on all V.S.U.N.-qualified crystals with nontrivial symmetry (space group other than P1). We evaluate novelty at three levels: chemical composition (reduced formula), structural prototype, and full crystal structure. Structural prototypes are identified using Wyckoff-based AFLOW-style labels, incorporating anonymous formulas, Pearson symbols, space groups, and Wyckoff positions.
We compare the generated crystals against two reference sets: (1) the corresponding training dataset and (2) an expanded collection comprising the original evaluation references, the latest MP release used in our analysis, the full Alexandria database, and GNoME.
| Novelty level | MP-20: vs. training | MP-20: vs. expanded references | Alex-MP-20: vs. training | Alex-MP-20: vs. expanded references |
|---|---|---|---|---|
| New reduced formula | 58.8% | 2.4% | 27.8% | 5.2% |
| New structural prototype | 27.5% | 11.3% | 15.0% | 13.6% |
| New crystal structure | 100% | 47.1% | 100% | 74.3% |
These results provide three important observations:
- Exploration beyond the training distribution. Compared with their respective training sets, 58.8% / 27.8% of the generated crystals have unseen reduced formulas, while 27.5% / 15.0% exhibit previously unseen structural prototypes on MP-20 / Alex-MP-20.
- Novelty beyond the evaluation references. Even after comparison with the expanded databases, 47.1% / 74.3% of the generated structures remain unmatched, indicating that their novelty is not merely an artifact of the original evaluation reference.
- Novel structural prototypes. Notably, 11.3% / 13.6% of the crystals exhibit structural prototypes absent from the expanded references, while genuinely new reduced formulas are less frequent (2.4% / 5.2%). This suggests that an important contribution of our method is discovering new structural arrangements, rather than simply generating new chemical compositions.
- External evidence of physical plausibility. Interestingly, 52.9% (MP-20) and 25.7% (Alex-MP-20) of the analyzed crystals, although absent from both the training set and the original novelty references, match independently reported, DFT-evaluated structures in Alexandria, GNoME, or more recent MP releases. Two of these also have experimental counterparts in ICSD.
These independent matches provide encouraging evidence that our model can generate physically meaningful crystal structures beyond its training distribution. Although they do not directly validate the remaining unmatched structures, they support the effectiveness of our generation framework in exploring chemically plausible regions of crystal space.
We acknowledge that experimental synthesis and expert assessment of the newly generated candidates remain important directions for future validation. We will include these additional analyses, along with representative CIF files of crystals exhibiting previously unreported structural prototypes, in the revised appendix.
I thank the authors for their responses, and have reflected on them. I would like to retain my original score.
Rebuttal 1
Discovering new crystal structures remains an open and challenging problem in materials generation. Most existing generative methods focus on learning the training data distribution, which limits their ability to explore structures that differ significantly from known materials. To overcome this limitation, we propose a novel framework that learns local structural concepts and combines them in new ways to generate previously unseen crystal structures. Our experiments show that this compositional approach achieves substantially higher novelty than existing methods. Therefore, the main contribution of our work is therefore not a new VQ-VAE, Transformer or diffusion architecture, but a new approach to crystal generation that uses learned concepts to explore structural possibilities beyond the training distribution.
As discussed in Section 3.4, existing conditioning signals, such as physical properties, space groups, and predefined structural motifs, are not directly suited to our compositional setting. In contrast, our learned concepts have three important properties: (1) locality, allowing multiple concepts to coexist within a crystal; (2) discreteness, providing identifiable and reusable structural units for explicit recombination; and (3) data-driven learning, avoiding reliance on predefined structural patterns.
We intentionally adopt standard VQ-VAE, Transformer, and diffusion architectures to isolate the contribution of compositionality. Although more advanced architectures could potentially improve generation performance, they would also introduce additional factors, making it harder to attribute improvements specifically to concept composition.
Our experiments demonstrate that introducing concept composition into the same base generative architecture substantially improves structural novelty. Moreover, our concept analyses and verification experiments in Sections 4 and 6 show that the learned concepts capture coherent local structural patterns and can be recovered from generated crystals. These results support both the usefulness of the learned concepts and their role in enabling compositional generation.
Therefore, the key novelty of our work lies in how learned structural concepts are represented, recombined, and used to guide crystal generation, rather than in the individual backbone architectures.
DFT evaluation. We follow the Materials Project computational settings for our DFT calculations, with implementation details provided in Appendix C.5. The reference datasets used for the final stability and novelty evaluation are consistent with those adopted by MatterGen. We will clarify the distinction between DFT calculations and the reference datasets used for evaluation to avoid confusion.
Utility of Figure 2. Figure 2 illustrates the Transformer architectures used in the VQ-VAE, composition generator, and base generative model. Its main purpose is to clarify the inputs, outputs, and conditioning mechanisms of these components, rather than to introduce novel backbone architectures. We will revise the figure caption and accompanying text to make this purpose more explicit.