- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes CrystRL, an RL-based fine-tuning procedure for pretrained crystal diffusion models, demonstrated on MatterGen. Instead of conditioning generation on a target property value, the reverse denoising process is formulated as an MDP and fine-tuned with an actor–critic algorithm so as to maximize the property of the relaxed final structure — bulk modulus throughout the experiments. The method handles the hybrid action space by treating continuous variables (coordinates, lattice) and discrete variables (atomic species) as separate policy factors with their own clipped surrogate losses, stabilizes training by freezing the backbone and updating only ~133k LoRA parameters on the output projections, and trains a time-conditioned critic, initialized from the score network, to estimate the terminal reward from intermediate noisy states. On Alex-MP-20, CrystRL raises the yield of high-modulus candidates relative to MatterGen conditioned at 320 GPa while keeping the usual stability/uniqueness/novelty (S.U.N.) metrics within a reasonable range; the top C–Re candidates were additionally checked with VASP.
Critical strengths
1.The problem being attacked — saturation of guided diffusion at extreme conditioning values — is a genuine bottleneck for property-targeted materials generation, and Figure 1 demonstrates it convincingly with the official checkpoint. 2.The hybrid update scheme is well matched to crystallographic data. Separating continuous coordinate updates from discrete species updates with distinct clipping thresholds ( versus ) reflects the actual geometry of the two spaces rather than forcing one framework onto both. 3.Freezing the backbone and training a small set of LoRA parameters is a pragmatic and, judging by the training curves, effective guard against policy collapse. Initializing the time-conditioned critic from the score network is economical.
Critical weaknesses
1.Reward-metric circularity, with DFT verification on six hand-picked structures only. The observed 10–15 GPa ML–DFT offset on those six is small enough to accept at the top, but the bias across the rest of the range is unknown. 2.No selection baseline. Training consumed roughly twenty thousand rollouts, each scored by a MatterSim relaxation. Under the same budget, sampling from the baseline and keeping the best is the natural null, and since MatterGen's best (405.09 GPa) already sits within ~3 GPa of CrystRL's best (408.21 GPa) it is not obvious that the RL run wins on the maximum. The comparison should be made, and the baseline should also be shown at conditioning values beyond 320 GPa. 3.All ≥400 GPa structures are C–Re binaries, and three of the four highlighted compositions (C$_2$Re$_4$, C$_3$Re$_6$, C$_3$Re$_9$) are at Re:C = 2:1, i.e. the Re$_2$C stoichiometry, a composition with a substantial prior literature as an ultraincompressible carbide. Novelty assessed by structure matching against Alexandria+MP-20 cannot see that literature, nor ICSD or OQMD. The generated phases should be compared against known and predicted Re–C phases before any novelty claim is made. 4.No ablations (critic on/off, clipping thresholds, LoRA rank, KL weight, entropy coefficient — the last is not even reported), and everything comes from a single training run with no error bars. The ≥400 GPa claim is 6 out of 512. 5.The critic's error grows to ~62 GPa MAE at full noise (Appendix B), and the states the policy visits as training progresses are out of distribution for the critic by construction. The possibility that the actor exploits this is acknowledged implicitly but never probed. 6.The generated structures require post-hoc relaxation before their properties are realized; the model produces pre-relaxed states rather than local minima natively. The authors are upfront about this in the limitations section.
Questions for authors
1.Did you observe reward hacking during training — structures that MatterSim scores highly but that are unphysical or clearly non-synthesizable when inspected? 2.How sensitive are the results to the asymmetric clipping ( versus )? What happens with equal thresholds or a single unified ratio? 3.Optimizing bulk modulus alone pushes compositions toward heavy, dense elements; the C–Re concentration is predictable in hindsight. How easily could a hull-distance or synthesizability term be folded into the reward without destabilizing PPO? 4.Where exactly does the 405.09 GPa MatterGen structure come from (conditioning value, sample count, compute), and how should it be reconciled with the 0.00% entry in Table 1? 5.Can you provide DFT moduli for a random (not top-scored) subsample of generated structures, and values for the Appendix D structures against all competing C–Re phases? 6.Under a matched relaxation budget, how does best-of- selection from the baseline compare? 7.Is the fine-tuned policy conditional or unconditional during rollouts? The abstract says the unconditional reverse diffusion is fine-tuned, but Section 4 initializes from a model equipped with property-conditioning adapters.