- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes ghost atoms, which allow common generative models for crystal architectures to vary the number of atoms in a crystal unit cell during generation. The core idea of this research is Voronoi-guided ghost-atom placement, in which ghost atoms are placed in the largest empty spaces inside the unit cell, thereby avoiding overlap with real atoms. The authors claim that this design avoids non-physical behavior of atoms inside crystals and provide experiments intended to:
- isolate the effect of their core design choices in comparison with concurrent approaches;
- demonstrate state-of-the-art model performance.
The authors begin by explaining the problem, present their solution design, choose the OMatG generative methodology as the baseline model, design an iterative procedure for adding ghost atoms into the current largest empty space in the crystal, and provide a set of experiments comparing models trained on MP-20 and evaluated using SUN, MSUN [1], and SUN+MSUN in the LeMatGenBench variant [9] (with LeMatBulk [10] as the novelty reference dataset).
Critical strengths
S1
An interesting idea that addresses an important limitation.
S2
The paper is well written and easy to follow.
Critical weaknesses
The following weaknesses concern the experimental part of the work, which should provide a clear justification that the proposed methodology for constructing generative models is better than existing ones.
W1
Effect isolation
The random-ghost control usefully isolates placement under supervised coordinates, but it is not a matched MiAD-style baseline: MiAD pairs uniform/random placement with masking of the coordinate loss for ghosts, whereas the paper’s control keeps that loss unmasked. The paper states that this is not a "reimplementation of MiAD", so that is not a problem.
However, MiAD [6] proposes a particular methodology for working with ghost atoms: uniform/random placement of ghosts together with masking of the coordinate loss for those ghosts. The paper’s "random-ghost control" differs from that design mainly by not masking the coordinate loss. Therefore, a comparison with the complete MiAD-style setup on the same baseline is expected: OMatG + random ghost placement + coordinate-loss masking. Uniform placement in MiAD was proposed together with coordinate-loss masking, and MiAD demonstrates both conceptually and experimentally that this masking is important (MiAD [6], Table 5 — in the case of uniform placement, loss masking increases the metric by +35% on average). So the proposed "random-ghost control" experiment is a comparison with a weakened version of the methodology proposed in MiAD. The paper already includes MiAD in Table 1; what is missing for a fair methodology comparison is specifically this matched OMatG + random + masking ablation.
W2
Metric choice
LeMatGenBench [9] is a well-known benchmark in the field of de novo crystal generation. In my view, it is a good tool for analyzing the performance of particular generative models on the task of discovering materials that are nearly "unknown to humanity". However, its applicability to comparing generative methodologies seems questionable for the following reason. Many existing papers that propose a generative model design train and evaluate their models on the MP-20 dataset [2–8]. This dataset is well known to the community and cheap enough for training, and many previous papers compare the SUN of models trained on MP-20 using MP-20 itself (or the training set) as the novelty reference [2–8] — an entirely logical choice, since we clearly do not want a model trained on MP-20 to reproduce crystals from MP-20. (Note: MatterGen [1] is an important exception, using a larger reference set rather than MP-20 for novelty.) However, using LeMatGenBench, which employs LeMatBulk [10] (a dataset much larger than MP-20) as the novelty reference, means that we penalize models for discovering materials from LeMatBulk even when those materials are not present in MP-20. This aspect of SUN in LeMatGenBench is not an undoubted benefit. Notably, the LeMatGenBench paper itself reports that stability rates drop by factors of 2–3× and S.U.N. rates collapse even further when evaluated against LeMatBulk instead of smaller references (e.g., DiffCSP S.U.N. from 4.1% to 0.1%) [9], which confirms that the choice of reference set materially affects model rankings. Quoting the paper that proposed LeMatGenBench [9]:
"As generative models scale toward deployment in real discovery settings, evaluation against a comprehensive reference set becomes essential. MP-20 remains valuable for controlled methodological benchmarking, but LeMat-Bulk provides a more reliable basis for assessing discovery-oriented performance."
Since your work proposes a new methodology for constructing generative models, I believe comparison using SUN on MP-20 is necessary.
Also, the fact that the proposed model leads in SUN + MSUN is almost entirely due to its higher MSUN. The scale of MSUN is larger than that of SUN by design (as seen in the paper's results table, MSUN is roughly 15–85 times larger than SUN across the reported models). Therefore, leading in MSUN will almost always result in leading in SUN + MSUN, which means that introducing SUN + MSUN adds no new perspective to the model comparison and is redundant once MSUN is reported. If we remove SUN + MSUN, the proposed model is better on one metric (MSUN) and worse on another (SUN) — not better on both, and not better on one while comparable on the other. These results raise the suspicion that the model merely represents a shift along the trade-off between SUN and MSUN — which could still be a valuable result if you:
- justify that this shift is important; and
- justify that other models — or at least the main competitors — cannot make this shift.
References
[1] Zeni, C., Pinsler, R., Zügner, D., Fowler, A., Horton, M., Fu, X., Wang, Z., Shysheya, A., Crabbé, J., Ueda, S., Sordillo, R., Sun, L., Smith, J., Nguyen, B., Schulz, H., Lewis, S., Huang, C.-W., Lu, Z., Zhou, Y., Yang, H., Hao, H., Li, J., Yang, C., Li, W., Tomioka, R., & Xie, T. (2025). A generative model for inorganic materials design. Nature, 639(8055), 624–632. https://doi.org/10.1038/s41586-025-08628-5
[2] Miller, B. K., Chen, R. T. Q., Sriram, A., & Wood, B. M. (2024). FlowMM: Generating materials with Riemannian flow matching. Proceedings of the 41st International Conference on Machine Learning (ICML), PMLR 235, 35664–35686. arXiv:2406.04713
[3] Sriram, A., Miller, B. K., Chen, R. T. Q., & Wood, B. M. (2024). FlowLLM: Flow matching for material generation with large language models as base distributions. Advances in Neural Information Processing Systems 37 (NeurIPS), 46025–46046. arXiv:2410.23405
[4] Joshi, C. K., Fu, X., Liao, Y.-L., Gharakhanyan, V., Miller, B. K., Sriram, A., & Ulissi, Z. W. (2025). All-atom Diffusion Transformers: Unified generative modelling of molecules and materials. Proceedings of the 42nd International Conference on Machine Learning (ICML), PMLR 267. arXiv:2503.03965
[5] Kazeev, N., Nong, W., Romanov, I., Zhu, R., Ustyuzhanin, A., Yamazaki, S., & Hippalgaonkar, K. (2025). Wyckoff Transformer: Generation of symmetric crystals. Proceedings of the 42nd International Conference on Machine Learning (ICML), PMLR 267, 29495–29526. arXiv:2503.02407
[6] Okhotin, A., Nakhodnov, M., Kazeev, N., Lazarev, M., Ustyuzhanin, A., & Vetrov, D. (2025). MiAD: Mirage Atom Diffusion for de novo crystal generation. arXiv preprint arXiv:2511.14426
[7] Yi, X., Xu, G., Zhang, Z., Liu, L., Bian, Y., Xiao, X., & Zhao, P. (2026). CrystalDiT: Simple diffusion transformers for crystal generation. Proceedings of the AAAI Conference on Artificial Intelligence. arXiv:2508.16614
[8] Hadži Veljković, T., Rosenthal, J., Lončarić, I., & van de Meent, J.-W. (2026). Crystalite: A lightweight transformer for efficient crystal modeling. arXiv preprint arXiv:2604.02270
[9] Betala, S., Gleason, S. P., Ramlaoui, A., Xu, A., Channing, G., Levy, D., Fourrier, C., Kazeev, N., Joshi, C. K., Kaba, S.-O., Therrien, F., Hernandez-Garcia, A., Mercado, R., Krishnan, N. M. A., & Duval, A. (2025). LeMat-GenBench: A unified evaluation framework for crystal generative models. arXiv preprint arXiv:2512.04562
[10] Siron, M., Djafar, I., Ramlaoui, A., du Fayette, E., Rossello, A., Fako, E., McDermott, M., Therrien, F., Barroso-Luque, L., Cipcigan, F., et al. (2025). LeMat-Bulk: Aggregating, de-duplicating, and curating quantum chemistry materials databases. arXiv preprint arXiv:2511.05178
Questions for authors
Q1 The text claims several times that
Introduction, Contributions paragraph:
We place these ghosts at well-spaced interstitial sites found by iterative Voronoi tessellation, choosing at each step the site furthest from every existing atom. Because these are the largest voids in the cell, the padded training data never contains the overlapping or nonphysically close atoms that arbitrary placement would produce. This lets fixed-cardinality generative models adjust the number of atoms during generation, and we measure the benefit by their rate of producing stable or metastable, unique, and novel structures. We compare Voronoi padding with random padding and show that the Voronoi-trained model learns to keep its ghosts out of the space occupied by real atoms.
Results, Reproducing the Tessellation paragraph:
Where ghosts are placed matters because their coordinate loss is not masked: they are supervised targets for the coordinate field rather than inert padding, and a model trained on physically implausible targets learns to reproduce them. We see this directly in where each model places its ghosts relative to real atoms.
Overlapping and physically implausible atom configurations are unintended behavior — but why is this important for ghost atoms? Especially when they are erased from the unit cell anyway, and the final structure will not contain overlaps or physically implausible configurations.
Q2 Results, Reproducing the Tessellation paragraph:
We also verify that the Voronoi-trained model reproduces the geometry rather than memorizing positions by taking the generated structures across all 5 seeds, stripping the model’s ghosts, and re-running the Voronoi maximin procedure of Section 3.5 on the remaining real-atom cell. Then, we compare the ghost–real distance distribution the model produced against the one the algorithm would have produced on the same real atoms. Over 11,593 structures containing 121,489 ghosts, the model’s median -normalized ghost–real distance is 0.357, against 0.359 for the re-tessellation — a difference of 0.6%. The agreement is stable across all five seeds (Table 4 in Appendix G).
Why is a 0.6% difference enough to support such a claim?
Q3 The text (Appendix I) states:
3000 epochs; best-validation checkpoint (epoch 1963)
Does this mean the checkpoint was selected based on the best loss on the validation set?
3 comments
I thank the authors for the additional experiments and clarifications.
W1. I appreciate the matched MiAD-style ablation. A remaining concern is training budget. The original MiAD work trains for 8000 epochs and reports that, at 3000 epochs, performance is still substantially below the peak (MiAD, Fig. 2 / Fig. 3). While the number of epochs required for a MiAD-style objective on the OMatG backbone and diffusion process may differ, a single matched run at 3000 epochs does not yet establish which placement-loss combination is preferable at peak performance. A more conclusive comparison would report performance over training for a sufficiently long run, identify the peak for each configuration, and compare MiAD-style random placement against Voronoi placement at those peaks.
I would also ask the authors to explicitly document the reproduced MiAD-style configuration and how it differs from the original MiAD setup (e.g., number of epochs, batch size, learning rate, learning-rate schedule, and related optimization choices). In particular, MiAD reports that learning-rate schedulers hindered further gains in their experiments, whereas the Ghosts-OMatG pipeline uses a learning-rate scheduler. Such differences do not confound the within-pipeline Voronoi vs. random comparison under a shared training recipe, but they do matter for interpreting how closely the matched baseline follows the original MiAD protocol, and they should be stated clearly in the revision, since incomplete matching of MiAD’s training protocol may underestimate the MiAD-style baseline relative to its peak performance under a more faithful setup.
W2. Thank you for the additional MP-20-reference results. To clarify: I did not claim that ghosts improve only mSUN. My concern was specifically that leadership on SUN+mSUN can be driven largely by mSUN because mSUN is much larger in scale than SUN (roughly 15–85× across the reported models), so the sum can add little new information once both components are reported. I also note that SUN+mSUN is used in LeMat-GenBench, which I had overlooked; given that, and given that the additional results support gains beyond mSUN alone, I no longer view the use of this aggregate as a substantive issue for the paper’s main claims.
Q1. Understood; thank you for the clarification.
Q2. Thank you — the Hungarian-matching analysis addresses my concern.
Q3. Confirmed. As a suggestion for the revision: consider selecting checkpoints by a generation-quality metric (e.g., SUN+mSUN, if that is the primary target) rather than by validation loss. For diffusion/flow models, validation loss is often weakly correlated with sample-quality metrics [1–4]. Metric-based checkpointing would also put the Voronoi and MiAD-style runs on a more equal footing: long training, periodic evaluation every K checkpoints, selection of each run’s best checkpoint by the primary metric, and comparison at peak performance.
[1] Ho et al., DDPM (2020).
[2] Nichol & Dhariwal, Improved DDPM (2021).
[3] When Are Two Scores Better Than One? (2026).
[4] Bigger Isn’t Always Memorizing (2025).
Thank you for the follow-up and the careful reading.
W1. We agree that the OMatG MiAD-style runs use our training recipe rather than MiAD's: AdamW (learning rate 1.97e-4, weight decay 3.1e-4), cosine annealing over 2000 epochs, batch size 256, best checkpoint by validation loss. All OMatG variants share this recipe, and we will state it clearly in the revision. To test MiAD under its own protocol, our DiffCSP runs use MiAD's released code and its MP-20 configuration (8000 epochs, Adam with learning rate 1e-3, no learning-rate scheduler, batch size 256; we only use 20 instead of 25 atom slots). There, random ghosts with the MiAD loss give 23.32 ± 0.52 and Voronoi ghosts with the standard loss give 23.31 ± 1.23 (random ghosts with the standard loss: 21.10 ± 1.34). So when the MiAD baseline is trained as MiAD recommends, it matches but does not exceed Voronoi placement, and Voronoi placement needs no change to the loss. We agree that validation loss is an imperfect selection criterion. In the revision we will select checkpoints by SUN+mSUN for all runs, evaluating the saved checkpoints of the existing OMatG and DiffCSP runs over training, and compare each configuration at its best checkpoint. We will also train both the MiAD-style and Voronoi configurations on OMatG with MiAD's recipe (8000 epochs, Adam with learning rate 1e-3, no scheduler) and compare them the same way.
W2. Thank you for clarifying, we agree. Since mSUN is one to two orders of magnitude larger than SUN for current models, a ranking by the sum is mostly a ranking by mSUN, and SUN carries little weight in it. We still see raising mSUN as useful in its own right, and Table 1 reports SUN and mSUN separately; in the revision we will discuss both rather than only the sum.
Q3. Thank you for the suggestion, we will adopt it (see W1).
We thank the reviewer for the careful feedback and address their concerns below. All new numbers come from our own runs (N = 2500 structures, 5 seeds).
W1 (MiAD-style baseline).
We agree this was a missing comparison, so we trained the appropriate MiAD baseline. The ghost-coordinate loss is masked as in MiAD, for both random and Voronoi ghosts, with the position loss averaged either over all atoms or over real atoms only (3000 epochs). MiAD uses the real-atom average. SUN+mSUN (%):
The MiAD baseline on our model (random ghosts, MiAD loss) gives 24.21; our submitted model gives 25.0. With Voronoi ghosts, the supervised and MiAD losses agree within error bars (25.0 vs 25.66), so Voronoi ghosts do not require modifying the loss. Random ghosts do, and with the standard loss they give 22.80. With the same loss, Voronoi placement matches or beats random placement in every column. On DiffCSP the effect is the same. With DiffCSP's architecture and loss unchanged, Voronoi ghosts give 23.31 against 21.10 for random ghosts, and random ghosts reach 23.32 only with MiAD's modified loss.
W2 (LeMat-GenBench, SUN on MP-20). With MP-20 as the novelty reference:
Ghosts raise SUN, not only mSUN. On the OMatG framework we measure 2.42 and 2.68 vs 1.49 SUN, and 5.35 (Voronoi) and 6.11 (random, MiAD loss) vs 4.76 on DiffCSP. So the gain is not a trade of SUN for mSUN.
The large stability gap in the submission came from comparing against a different OMatG model (encoder-decoder interpolant, ODE sampling) than the one our model is built on (linear interpolant, SDE sampling). Run through our pipeline, the matching OMatG checkpoint is 7.75–8.14% stable, against 7.3% for our model. On DiffCSP, ghosts do not lower stability (about 14–15% with and without). Ranking by SUN+mSUN and using LeMat-Bulk as the reference are choices made by the benchmark. We will report both references in the revision.
Q1 (why does ghost placement matter if ghosts are removed?). During generation, a ghost can turn into a real atom wherever it currently sits. So the ghost positions during generation are part of the structure being built. The model's predictions for the real atoms also depend on the ghost positions at every step. In practice, Voronoi placement does better than random placement with the same loss (W1).
Our analysis of late ghost→real switches did not show that late insertions hurt stability. We will therefore reword the sentence about keeping ghosts out of occupied space as a description of what the model does, not as the reason it works.
Q2 (is a 0.6% difference enough?). We agree it is not, so we tested it directly. For each generated crystal we remove the model's ghosts, rerun Voronoi maximin placement on the real atoms, and match the model's ghosts to the new sites with a periodic-distance Hungarian assignment. On a scale where 0 is chance and 1 is agreement within the placement jitter (Voronoi training data scores 1, random training data scores 0):
(5 seeds, about 122,000 ghosts per model.)
The model puts most of its ghosts in the Voronoi cavities of new structures it has never seen, while the random-ghost model stays at chance. That is not memorization, and we will remove that word.
Q3 (checkpoint selection). Yes, it is the checkpoint with the lowest total validation loss.