- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes CrystalDLM, a framework that combines diffusion language models with periodic coordinate modeling and physical-feedback learning for crystal generation. The method uses a lattice-conditioned periodic module to improve the geometric consistency of generated structures, followed by continuous diffusion refinement. It further introduces a feedback-learning stage that uses physically evaluated structural modifications to train a reconstruction model and a relative verifier for proposing and selecting improved candidates. Experiments on MP-20, Perov-5, and MPTS-52 show improvements in structural validity and several generation metrics. On MP-20, the complete framework achieves a S.U.N. rate of 10.50%, outperforming the evaluated baselines. Overall, the paper aims to improve crystal generation by incorporating periodic structural constraints and learning from physical evaluation outcomes.
Critical strengths
-
The paper explores an interesting direction by incorporating periodic structural constraints and physical feedback into diffusion language models for crystal generation. The proposed framework provides a reasonable attempt to improve both geometric validity and physical quality.
-
The paper conducts experiments on three crystal generation benchmarks, including MP-20, Perov-5, and MPTS-52. The results demonstrate improvements in structural validity and stable, unique, and novel crystal discovery compared with several existing baselines.
Critical weaknesses
-
Unclear advantages of diffusion language models for crystal generation. The paper does not convincingly demonstrate the unique advantages of using diffusion language models (DLMs) for crystal generation. The proposed framework combines masked prediction, periodic interactions, and continuous diffusion refinement, but these techniques are not inherently specific to DLMs. Although the periodic DLM improves structural validity, its contribution to stable and novel crystal discovery is relatively limited. More importantly, the paper lacks controlled comparisons with alternative generative backbones under the same refinement and feedback pipeline. It therefore remains unclear whether DLMs offer meaningful benefits over existing autoregressive or diffusion-based crystal generative models.
-
Insufficient validation of the physical-feedback learning mechanism. The physical-feedback stage appears to be the primary source of improvement in stable and novel crystal discovery. However, the proposed approach mainly relies on reward-weighted supervised fine-tuning and a learned verifier for candidate selection, rather than introducing a substantially new feedback-learning algorithm. More importantly, the current ablation studies do not distinguish the contributions of feedback-trained reconstruction, candidate sampling, and verifier-based selection. For example, it remains unclear whether comparable improvements could be obtained by simply generating more candidates and selecting them using the same verifier, without updating the generative model. Without these essential comparisons, the actual contribution of feedback learning remains insufficiently established.
-
Limited performance improvements relative to framework complexity. The complete framework involves a pretrained 8B diffusion language model, a periodic interaction module, continuous diffusion refinement, and an additional feedback-learning pipeline with a reconstruction model and verifier. Despite this considerable complexity, the reported improvements over existing methods are relatively modest. For example, the S.U.N. rate on MP-20 increases from 9.42% for CrysLLMGen to 10.50% for CrystalDLM. Furthermore, the paper does not provide a comprehensive analysis of training costs, inference overhead, or the computational resources required for physical-feedback collection. Without a clear cost-performance comparison, it is difficult to justify the complexity of the proposed framework over simpler alternatives.
-
Lack of standardized evaluation for stable and novel crystal discovery. The paper evaluates S.U.N. and M.S.U.N. on MP-20 using its own evaluation protocol rather than the standardized framework provided by LeMat-GenBench. Although both evaluations can be performed on MP-20, differences in relaxation procedures, stability assessment, and novelty criteria may substantially affect the reported results. Consequently, the reported S.U.N. improvements may not be directly comparable with results obtained under other evaluation settings. Given the relatively modest performance gains, evaluating CrystalDLM and its baselines under the same LeMat-GenBench protocol would provide more reliable evidence of its effectiveness in stable and novel materials discovery.
Questions for authors
-
What are the unique advantages of using a diffusion language model (DLM) for crystal generation? Could the authors clarify which capabilities are specifically enabled by the DLM architecture and whether similar results could be achieved using alternative generative backbones under the same refinement and feedback pipeline?
-
How much of the improvement comes from feedback learning itself rather than candidate sampling and verifier-based selection? Could the authors provide additional ablation studies comparing feedback-trained reconstruction with a simpler generate-and-rank baseline that uses the same candidate budget and verifier without updating the generative model?
3 comments
W2 & Q2. Why train proposals with physical feedback when a verifier can rank generated candidates?
Physical-feedback training improves the proposals supplied to the same verifier, including at a smaller candidate budget. Whereas geometric pretraining recovers clean fields from corrupted training crystals, feedback training learns where to edit, which fields to change, and what replacements to propose from the measured outcomes of edits to generated references (§4.3.2, Eqs. 12–13). Appendix C.1 connects these executed edits and their relative utility to conditional prediction targets, while forward-KL regularization retains the starting model's geometric prior.
Holding the final encoder, relative verifier, references, and candidate budget (up to eight candidates) fixed, Table R3-3 compares geometrically pretrained and feedback-trained reconstruction across three runs before bounded single-field revision.
Table R3-3. Proposal learning with the same verifier (mean ± sample SD over three runs).
| Proposal model | S.U.N. (%) | M.S.U.N. (%) |
|---|---|---|
| Before physical-feedback training | 9.60 ± 0.24 | 48.27 ± 0.13 |
| After physical-feedback training | 10.54 ± 0.26 | 49.05 ± 0.71 |
Under the identical verifier, proposal training improves both discovery metrics in all three runs. To verify that proposal quality improves independently of learned verifier ranking, Table R3-4 replaces the verifier with random selection in Run 1.
Table R3-4. Proposal learning with random selection, Run 1 (%).
| Proposal model | S.U.N. | M.S.U.N. |
|---|---|---|
| Geometrically pretrained reconstruction | 9.50 | 41.50 |
| Feedback-trained reconstruction | 9.90 | 43.50 |
Furthermore, when paired with the verifier, feedback-trained reconstruction using up to four candidates reaches 10.02% S.U.N. / 48.44% M.S.U.N., exceeding geometrically pretrained reconstruction using up to eight candidates (9.86% / 48.24%). Proposal training thus makes useful edits accessible within a smaller candidate budget, while relative verification selects among them.
W3. What benefits do the additional components and computation provide?
The added computation improves periodic geometry, learns reusable edits, and increases effective conditional-generation throughput. On MP-20, CrystalDLM achieves 10.50% S.U.N. / 51.24% M.S.U.N., compared with CrysLLMGen's 9.42% / 44.78%, gains of 1.08 and 6.46 percentage points.
1. Module capacity. Both the periodic head and the relative verifier are lightweight modules built on the backbone representations.
Table R3-5. Parameter counts of the periodic head and relative verifier.
| Component | Parameters |
|---|---|
| Lattice-conditioned periodic head | 0.537M |
| Relative verifier | 1.049M |
The lattice-conditioned periodic head adds 0.537M parameters (0.0067% of the backbone) and improves draft structural validity by 8.68 percentage points on 33–52-atom MPTS-52 cells.
2. One-time physical-feedback cost. Appendix A and Table 5 give the upstream models' training settings; Table R3-6 reports physical-supervision collection and feedback training alongside the LLaMA-3-8B baseline training cost.
Table R3-6. Physical-feedback preparation and baseline training costs.
| Computation | A100 GPU-hours |
|---|---|
| Physical-supervision collection | 5.56 |
| Reconstruction and verifier training | 0.65 |
| Supervision collection + feedback training | 6.21 |
| LLaMA-3-8B baseline training | 6.71 |
This 6.21 A100 GPU-hour investment (versus 6.71 A100 GPU-hours for LLaMA-3-8B baseline training) learns proposal and selection abilities that are reused on new structures without adding physical relaxations during candidate ranking; the reported relaxation workload includes reference and final-output assessment.
3. Conditional construction throughput. Table R3-7 compares prediction modes under the same chemical conditions on the same physical A100 GPU with batch size one. Wall-clock time includes failed attempts and construction recovery, and valid drafts must satisfy both the specified chemical condition and structural validity.
Table R3-7. Conditional construction efficiency.
Here, condition matching means that the generated structure has exactly the specified chemical composition and total number of atoms.
| Dataset | Constructor and prediction mode | Condition & Struc. Valid (%) | Seconds per condition-matched valid draft |
|---|---|---|---|
| MP-20 | LLaMA-3-8B | 44.38 | 18.644 |
| MP-20 | Periodic DLM, sequential fields | 99.80 | 6.025 |
| MP-20 | Periodic DLM, joint coordinate axis | 71.52 | 4.061 |
| MPTS-52 | LLaMA-3-8B | 8.76 | 228.282 |
| MPTS-52 | Periodic DLM, sequential fields | 92.96 | 14.901 |
| MPTS-52 | Periodic DLM, joint coordinate axis | 92.10 | 2.463 |
Sequential-field Periodic DLM produces condition-matched valid drafts 3.1× faster on MP-20 and 15.3× faster on MPTS-52 than LLaMA-3-8B. Joint-axis prediction reaches 2.463 seconds per valid draft with 92.10% validity on MPTS-52, and 4.061 seconds with 71.52% validity on MP-20, offering a direct speed–yield trade-off.
For CrysLLMGen's native free-generation task, batch-16 generation of 5,000 outputs takes 36.46 minutes, with 89.94% validity and 0.4865 seconds per valid draft. Common continuous refinement takes 12.31 minutes, yielding 94.78% validity. This native task is reported separately from conditional construction.
W4. Evaluation with LeMat-GenBench
Physical-feedback gains persist under a shared LeMat-GenBench configuration. All methods use CHGNet 0.3 relaxation, the Materials Project phase reference (2026-04-13), the MP-20 novelty reference, and common validity criteria. Novelty and uniqueness are evaluated on relaxed structures using StructureMatcher, with lattice, site, and angular tolerances of 0.1, 0.3, and 5°, respectively.
Table R3-8. MP-20 results under the shared LeMat configuration (%).
| Method | Novelty | S.U.N. | M.S.U.N. |
|---|---|---|---|
| CrystalDLM before feedback | 90.98 | 6.38 | 39.90 |
| CrystalDLM | 92.89 | 6.96 | 42.96 |
| CrysLLMGen | 81.59 | 5.82 | 40.70 |
| ADiT | 37.24 | 3.70 | 21.92 |
| MatterGen | 89.73 | 6.90 | 45.40 |
Novelty is measured over valid relaxed structures; M.S.U.N. combines LeMat's stable and metastable discovery categories.
Feedback raises novelty from 90.98% to 92.89%, S.U.N. from 6.38% to 6.96%, and M.S.U.N. from 39.90% to 42.96%. CrystalDLM achieves the highest observed novelty and S.U.N. among the compared methods; its M.S.U.N. exceeds CrysLLMGen and ADiT, while MatterGen reaches 45.40%. The feedback improvement therefore extends to evaluation on relaxed structures.
We thank the reviewer for the detailed assessment. We hope these results clarify our design choices and their empirical benefits, and we would welcome further discussion of any remaining questions.
We sincerely thank the reviewer for the careful evaluation of our paper and for the detailed comments and questions.
W1 & Q1. Why use a diffusion language model?
A masked DLM provides a common conditional-prediction interface for crystal construction and local revision. By holding specified composition, atom count, and visible geometry fixed as observed tokens while predicting masked fields, it supports both lattice-conditioned periodic coordination of unresolved positions and local re-masking of selected fields within the surrounding structure (§4.2.2; Appendix D.3).
Table R3-1 compares Periodic DLM with crystal-adapted LLaMA-3-8B. In the MPTS-52 backbone comparison, both receive the same composition and atom count and share the continuous refiner, reconstruction model, and relative verifier.
Table R3-1. Discovery results (%).
1. Satisfying the specified chemical condition is a key difficulty for the tested LLaMA-3-8B constructor. On MP-20, 99.90% of LLaMA-3-8B outputs are parseable, 53.42% match the specified composition and atom count, and 44.38% additionally pass structural validity; on MPTS-52, these rates drop to 99.62%, 10.58%, and 8.76%. Conditioning geometry prediction directly on observed composition and atom-count tokens avoids this failure mode and contributes to Periodic DLM's higher discovery rates in Table R3-1.
2. Periodic coordination becomes more valuable as cells grow. Periodic DLM achieves 100.00% structural validity on MP-20 after continuous refinement. Table R3-2 shows its MPTS-52 construction results stratified by unit-cell size, reaching 97.14% overall validity with local re-masking.
Table R3-2. MPTS-52 structural validity by cell size (%).
Periodic construction improves single-pass validity by 1.60, 5.30, and 8.68 percentage points across the three size groups, and local re-masking further raises 33–52-atom validity from 60.34% to 83.20%. A separate fixed-sampler control likewise shows gains of 2.96 points on MP-20 and 2.84 points on MPTS-52 from learned periodic interactions. Together, these results support combining DLM conditional prediction with explicit periodic coordination as unit-cell geometry grows more complex.