acceptodds
Withdrawn from ICLR 2027

RMD: Relaxation-Inspired Masked Diffusion Language Model for Crystal Generation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Language models offer a promising approach to crystal generation through unified sequence representations and flexible conditional modeling. However, standard autoregressive generation follows a unidirectional, left-to-right order, restricting predictions to the generated prefix and typically preventing earlier outputs from being revised using later structural information. To address these limitations, we present the first application of a discrete masked diffusion language model to crystal structural text generation, enabling bidirectional prediction and iterative recovery. To further address the challenge of identifying token positions that require continued adjustment, we propose Relaxation-inspired Masked Diffusion (RMD). Inspired by the assess–update–reassess organization of physical relaxation, RMD combines relaxation-score-driven position selection with selective state updates, allowing existing content to be revised as the visible context evolves. State-of-the-art results on the evaluated de novo generation and composition-conditioned crystal structure prediction benchmarks, support RMD's potential to produce high-quality candidates for stable crystal discovery. Our code and datasets are available at https://anonymous.4open.science/r/RMD.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Mask diffusion, Language model, Crystal structure prediction

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_7agtOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes Relaxation-inspired Masked Diffusion (RMD) for crystal generation. Crystals are serialized into structural text containing composition, lattice parameters, atomic sites, and symmetry information, and a bidirectional network is trained to reconstruct masked tokens. The main methodological component is an inference-time token-selection and revision strategy. Candidate log probabilities computed from Gumbel-perturbed logits determine which positions remain masked or are remasked, allowing previously generated content to be reconsidered as the visible context evolves. The analogy to physical relaxation concerns this repeated assessment and updating procedure; the sampler does not explicitly evaluate energies or forces. Experiments cover de novo crystal generation and composition-conditioned crystal structure prediction. The reported results show improvements in validity and the yield of stable structures over a masked-diffusion baseline with random position selection, competitive structure-prediction performance, and fewer subsequent CHGNet relaxation steps than the evaluated baselines.

Critical strengths

This paper firstly try the discrete mask-based diffusion language model for crystal generation. It proposes a atom-wise correction method in sampling.

Critical weaknesses
  1. The physical-relaxation analogy is weakly justified, and Figure 1 provides limited technical insight. I find Figure 1 largely schematic: it illustrates the general distinction between autoregressive generation and bidirectional revision, but provides little explanation of the proposed mechanism. The actual selection score is derived from Gumbel-perturbed token probabilities, with no demonstrated relationship to physical energy, forces, stress, or geometric errors. An actual refinement trajectory would be more informative.

  2. The representation and implementation are insufficiently specified. The manuscript describes preprocessing, tokenization, decoding, and postprocessing through abstract mappings, but does not provide enough detail about numerical precision, structural ordering, canonicalization, or malformed-output handling. The advantages and potential information loss of representing numerical geometry through text tokens are also insufficiently evaluated. Missing architectural and training details further limit reproducibility and make it difficult to assess how much performance depends on representation and postprocessing choices.

  3. The computational-efficiency evidence is incomplete. Fewer downstream CHGNet relaxation steps are encouraging, but they do not establish lower computational cost for the complete pipeline. RMD requires repeated language-model evaluations, yet the numerical sampling budget, runtime, throughput, and combined generation-plus-relaxation cost are not reported. Table 2 also omits the MD baseline, making it difficult to attribute the reduced relaxation requirement specifically to the proposed selection strategy. Energy-change units, evaluated sample counts, and failure handling require clearer reporting.

Questions for authors
  1. Can the authors provide a complete, reproducible structure-to-token-to-structure example? Please specify all pre-postprocessing or repair operations, and identifying which components are learned. What encoding–decoding error does the representation introduce, and what benefit does textual tokenization provide for geometric modeling?

  2. What is the complete computational cost of obtaining useful candidates?

  3. Why is confidence in a proposed token an appropriate criterion for retaining a potentially different existing token? Equation (10) scores a newly proposed token, whereas Equation (11) can use that score to retain an existing token that differs from the proposal.

Reviewer_QYydOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper applies a discrete masked diffusion language model to crystal structural text, as an alternative to autoregressive crystal LMs. At each iteration it scores all positions from the model's candidate distribution and keeps the lowest-scoring ones masked, or remasks them. Because generated tokens can be remasked, earlier unmaksed tokens can be revised. The training objective is derived through routing variables and reduces to masked-token cross-entropy. Experiments covers de novo generation on MP-20, evaluated for validity and DFT stability, a comparison against CHGNet-relaxed structures, and composition-conditioned CSP on four datasets. The paper claims state-of-the-art results, and also compares against a masked diffusion model that selects positions at random.

Critical strengths

The paper shows the potential of masked generative modeling for crystal text. A masked diffusion LM outperforms the autoregressive crystal LMs (CrystaLLM, Mat2Seq) on nearly every reported CSP and stability metric. It is a first step for this modeling choice in the domain.

Critical weaknesses

1. The relation of RMD to prior masked diffusion methods is unclear.

  • Connection to prior work is not discussed. RMD arrives at the standard masked diffusion objective of MDLM (Sahoo et al., 2024, arXiv:2406.07524) and MD4 (Shi et al., 2024, arXiv:2406.04329). Remasking the lowest-confidence tokens, including generated ones, under a decaying budget is a common practice in masked discrete diffusion, and the top-k routing of RDM (Zheng et al., 2023, arXiv:2302.05737) does the same in discrete diffusion. But these works are not discussed.

2. The empirical comparisons are not made under comparable conditions.

  • A unified benchmark would make the SOTA claim more convincing. Table 1 collects results from several papers that differ in sample counts, DFT protocols and novelty reference sets, and RMD's own DFT setup is not described. Evaluating on a unified benchmark such as LeMat-GenBench (Betala et al., 2025, arXiv:2512.04562) would be a stronger basis for the claim. It applies the same validity checks, MLIP-based stability against a large common reference hull, and novelty and uniqueness criteria to all models.
  • Missing CSP baselines. Recent CSP models are absent from Table 3: FlowMM, CrystalFlow (Luo et al., 2025, arXiv:2412.11693), OMatG (Höllmer et al., 2025, arXiv:2502.02582), MCFlow (Seong et al., 2026, arXiv:2602.20210). Several of them report higher match rates than RMD on shared benchmarks, so the CSP claim should be revisited against them.
  • AR advantage untested. AR results come from other papers with different setups. There is no controlled comparison, with everything fixed except the generation paradigm, showing that masked generation improves results or reduces inconsistencies between fields.
Questions for authors
  1. Could you discuss RDM, MDLM and MD4 in the related work, and state which elements of RMD differ from them?
  2. Can you report results on LeMat-GenBench, and describe the DFT protocol used for Table 1?
  3. Can you add CrystalFlow, FlowMM, OMatG, and MCFlow to the CSP comparison?
  4. What are K, T, the architecture, and the inference cost? How is the sequence length chosen in de novo generation?
  5. Can you compare with an AR model trained with the same tokenizer, data and size, and show where bidirectionality helps (e.g., lattice–site consistency, or infilling beyond composition prefixes)?
  6. How often are previously generated tokens remasked, and how often are they refilled with a different token?
Reviewer_Z5z5OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

In this work, the authors have proposed RMD (Relaxation-inspired Masked Diffusion) which implements a masked diffusion language model for crystal structure generation.

Critical strengths
  1. The paper is well written and easy to follow.

  2. The masked diffusion formulation addresses a limitation of sequential language modeling by allowing previously generated tokens to be rectified at a later diffusion timestep.

  3. Experimental results show that crystal structures generated by this method are able to achieve better performance with respect to metastability as well as stability compared to existing baseline models.

Critical weaknesses
  1. The major difference between RMD and diffusion based models is that it treats the crystal as a sequence of tokens. This means that coordinates and lattice parameters which are continuous variables are broken down into discrete tokens and masked independently. How do different tokenization strategies (at least for the real numbers) affect the stability of generated crystals?

  2. The authors have used a linear schedule for masking, which has been generally observed to not show the best performance for image generation models. Have the authors tried a different schedule?

  3. The authors have neither mentioned the number of diffusion timesteps for training nor the sampling steps for inference. A study of model's performance over different number of sampling steps would help in understanding the inference cost.

  4. The authors have also shown that the generated structures are closer to CHGNet relaxed structures than existing models, but this observation has not been justified. The authors have themselves noted in the limitation section that the updates are not driven by physically computed quantities, hence it is unclear what drives this behaviour. The authors should compare the MD model in table 2, and also think of ablations to find out the contributing factor to this behaviour.

  5. Table 2 shows different variants of the model that have been trained with a subset of the training data, but these variants are not compared in table 1. It would be interesting to see how these variants compare to other models in terms of stability and validity.

  6. Some preprocessing and postprocessing steps have been mentioned in section 2.2 without further details. A brief elaboration of these steps might help in understanding the format of data that the model works with.

  7. The underlying model architecture has not been mentioned in the paper. Without specific details it is difficult to understand how the diffusion timestep has been incorporated, and whether the model is scalable or not.

Questions for authors

See Weakness

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.