- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper addresses the scaling problem of 3D crystal material generation. Existing generative models, during generation implicitly model how a noise particle should become which atom, and since there are no atom indices, this matching gets harder as the number of atoms grows.
Authors proposed GLASS as a fix to this problem, where they first encode the crystal structure into a single global latent vector that doesn’t depend on atom ordering. Next, a flow matching model learns the latent space distribution and generates new latent vector. Following that, a slot-based decoder then turns the latent into all atoms at once.
Critical strengths
- Overall, the problem is interesting and prominent. Existing generative models suffer as the crystal structure grows larger.
- The paper is very well written with clearly motivating the problem and enugh background about crystal generation
- The idea of a slot-based decoder is really novel and interesting.
- The authors perform detailed experiments with MP-20 and MOFs datasets to empirically evaluate their model.
- Codebase provided.
Critical weaknesses
See the Questions.
Questions for authors
-
Crystal Materials are translationally invariant as well; how does the AE encoder maintain that? Is translation augmentation or canonicalization used?
-
The autoencoder is deterministic with no KL Loss; how smooth is the latent space? Is there any way to analyse that?
-
What's the significance of Pair-distance loss? What happens if we remove it? It would be great if the authors do an ablation study on that and checked the stability, novelty, and uniqueness of the model.
-
What is d_z for a single vector encoding up to 150 atoms, and what is the held-out reconstruction accuracy?
-
In Fig-3, authors stated GLASS at 50k and 1M flow-training steps. What are the steps for other models? Are they also similar or very less? If less, then the comparison is unfair.
-
Tables 1 and 11 use different models in the pre-relaxed and unrelaxed groups. The same baselines should be evaluated under both.
-
Stability, uniqueness, and SUN look good, but novelty of the model is only 45–53%. Any justification from authors would be good.
-
Finally, the paper misses some important papers on material generation like DiffCSP++, SymmCD, CrysLLMGen, CrysBFN, TGDMat, FlowMM, FlowLLM, OXtal,Wyckoff Transformer
-
Finally, what is the sampling time and computation needed for GLASS?
5 comments
Q5: In Fig-3, authors stated GLASS at 50k and 1M flow-training steps. What are the steps for other models? Are they also similar or very less? If less, then the comparison is unfair.
We compared the reported training lengths and found a large range. Zatom-1 uses about 43k steps, MatterGen 47k, and DiffCSP 106k. ADiT trains for about 488k steps, Crystalite for 1.33M, and OMatG for 1.7M. The GLASS 50k checkpoint is at the short end of this range, and the 1M checkpoint is below the two longest, so GLASS is not favored by training length. However, we also note that step counts are not directly comparable across batch sizes and model sizes.
Q6: Tables 1 and 11 use different models in the pre-relaxed and unrelaxed groups. The same baselines should be evaluated under both.
The reference values in Tables 1 and 14 (Table 11 in the original submission) are taken from the LeMat-GenBench leaderboard, where each model appears once, as submitted by its authors. The number of released models and the cost of evaluating the benchmark with relaxation prevented us from re-running every model under both settings. We have amended the relevant section to mark this more clearly. Table 16 complements this with a selection of models evaluated under the unified relaxation and evaluation pipeline of the Crystalite protocol.
Q7: Stability, uniqueness, and SUN look good, but novelty of the model is only 45–53%. Any justification from authors would be good.
Thank you for raising this point. Across the flow training, the model moves along a validity-novelty frontier: Early in training, validity is lower, but many samples are novel. As training progresses and the model closely approximates the training distribution, samples become more valid but less novel, shifting the point on the frontier. This transition is shown in Figure 6, showing a peak and eventual decline in novel and valid structures. Therefore, we treat the novelty of a structure set as an optimizable parameter via the training steps.
Furthermore, the selected checkpoint exhibits a novelty comparable to other models such as Crystalite or OMatG, showcasing its strong positioning in the high novelty and high validity regime. We also note that novelty is computed against the LeMat-Bulk reference set, not the MP20 training set.
Q8: Finally, the paper misses some important papers on material generation like DiffCSP++, SymmCD, CrysLLMGen, CrysBFN, TGDMat, FlowMM, FlowLLM, OXtal, Wyckoff Transformer.
Thank you for raising this point. DiffCSP++, FlowMM, and Wyckoff Transformer were already cited. We have incorporated the relevant missing literature in the appropriate section (Appendix A).
Q9: Finally, what is the sampling time and computation needed for GLASS?
On a single H100 GPU, sampling 1,000 MP20-size crystals takes 0.21 s, and sampling 1,000 QMOF150 MOFs takes 0.32 s (Table 21). Training the autoencoder takes about 20 minutes for MP20 and 3.5 hours for QMOF150. Flow training takes about 50 minutes for 1M steps on either dataset (Appendix I). For comparison, Table 16 lists 5 to 22 s per 1,000 structures for Crystalite and 85 s for ADiT.
Thank you for the detailed responses. Some concerns remain:
Q1 (Translation invariance). I consider this a significant limitation: the same crystal under a different origin is physically identical, yet the encoder treats it as a different input. I would ask the authors to quantify this directly by translating held-out structures with random shifts and reporting (i) the latent distance between original and shifted copies, relative to typical distances between different crystals, and (ii) the reconstruction accuracy of the shifted inputs. Without this analysis, it is unclear whether the latent space represents crystal structure or a dataset-specific coordinate convention. In addition, the authors state that shifting inputs and targets together made training collapse, and shifting only the inputs lowered held-out reconstruction. Could the authors explain why this happens? It suggests that the model relies on the absolute coordinate frame of the stored structures rather than on the crystal structure itself.
Q2 (Latent smoothness). Thank you for the qualitative analysis and the KL-regularized comparison. The observation that a small edit to a single atom changes the composition or displaces several atoms after re-encoding suggests the latent space may not be locally smooth. Could the authors provide a PHATE visualization of the encoded latents, colored by properties such as composition, space group or formation energy, to show whether similar crystals lie close together?
Q4 (Reconstruction). The gap between training and held-out reconstruction (0.019 Å vs. 1.25 Å RMSD; 99.99% vs. 58.68% type accuracy) indicates strong overfitting of the autoencoder, which may also explain the high fraction of generated MOFs that replicate training structures. Could the authors report validation curves and test whether early stopping on a held-out validation set, or regularization (e.g., weight decay, dropout, or the KL term of Table 8), improves held-out reconstruction, and how this affects the novelty of generated structures? Could they also report the corresponding held-out reconstruction numbers for MP-20?
Q7 (Novelty). The decline in novelty as training proceeds suggests that the model increasingly reproduces training-like structures, consistent with the reconstruction gap noted in Q4. I would encourage the authors to discuss this connection in the revised paper.
Dear Reviewer vpVs,
Thank you for the careful evaluation of our manuscript and for the positive assessment of the problem and the slot-based decoder. We address each question below.
Thank you for raising this important point. The encoder is not translation invariant, and we use neither translation augmentation nor canonicalization (Appendix D.1). We use the fractional coordinates exactly as stored in the datasets, without any shift. Structures are typically stored with the cell origin at a standard origin of their space group. In MP20 and LeMat-Bulk, many atoms occupy special Wyckoff positions, so fractional coordinates recur across structures, and the decoder slots specialize to these positions (Figure 12). In QMOF150, atoms mostly occupy general positions, and the shared structure is the placement of symmetry elements. Translation augmentation removed this anchor in our tests: shifting inputs and targets together made training collapse, and shifting only the inputs lowered held-out reconstruction.
Thank you for this interesting question. While defining smoothness in this latent space is not a trivial task, we have investigated the behavior of decoded structures around encoded data latents. We find that several behaviors emerge in the latent space: Some latent directions primarily activate/deactivate slots or change the chemistry. Other directions mostly change the cell. For larger structures, latent variables can also shift the atomic structure by moving several atoms. However, targeted displacements of individual atoms are challenging. When editing a specific position in a target atom of a training sample and subsequently encoding and decoding it, the decoded structure exhibits additional artifacts, such as an altered chemical composition or several displaced atoms. Therefore, changes in the latent vectors allow for structured changes in decoded crystals, but precise edits can be challenging.
In addition, we also compare a KL-regularized version in Table 8. For a KL weight of 1e-5, we find a slight gain in the valid and novel yield, while a higher 1e-4 weight increases validity at the expense of novelty.
Thank you for this suggestion. We ran the requested ablation with three training runs per setting. On MP20, the pair-distance loss has no significant effect, and mean SUN and MSUN are slightly higher without it (Appendix E.4):
On QMOF150, the pair-distance loss has a mild effect on validity, but no noticeable effect on novelty, uniqueness, or VNU:
Overall, this loss term has a marginal effect and is not essential to the performance of GLASS. We added the ablation to Appendix E.4 (Tables 10 and 11) and keep the original recipe for all other results.
For QMOF150, we use a latent dimensionality of 64, reaching a held-out reconstruction RMSD of 1.25 Å and 58.68% type reconstruction accuracy, compared to 0.019 Å RMSD and 99.99% accuracy in training. A comparison of different latent dimensionalities is shown in Table 9, and shows a negligible impact.