- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper introduces FieldForge, a flow-based training framework that unifies pretaining, quality-based supervised fine-tuning, and reinforcement learning in order to train a transformer-based backbone to generate inorganic crystalline materials with variable atom counts. The authors carry out a detailed study of the effect of every training stage on the final performance and report state-of-the-art performance in the de novo generation and crystal structure prediction tasks.
Critical strengths
- The de novo generation benchmark results from LeMat-GenBench are strong, and the study of the influence of the different training stages on the final performance is interesting.
- The quality-based supervised fine-tuning training stage is novel and appears to have a favorable effect on FieldForge's competitive performance.
Critical weaknesses
From a technical perspective, the paper mainly combines pre-existing ideas from generative models for inorganic crystalline materials:
- The transformer-based architecture is very similar to Crystalite.
- The stochastic flow policy in the reinforcement-learning setup in Eq. (3) seems very similar to the stochastic perturbations in OMatG-IRL.
- The rewards in the reinforcement-learning setup are taken from Chemeleon2.
- The anti-annealed sampling is taken from FlowMM.
- The trade-off between stability and novelty was already noted in the Crystalite paper and in the paper on the continuous SUN metric.
While I do think that such a combination of pre-existing knowledge in one framework is worthwhile because it can lead to state-of-the-art performance as in FieldForge, I do not think that this is enough for this ICLR conference. Besides the quality-based supervised fine-tuning step, I do not see any new technical insights that are informative for the generative modeling community for inorganic crystalline materials. With respect to the quality-based supervised fine-tuning, it is unclear whether similar performance could be achieved by using classifier-free guidance on the energy above hull as in MatterGen.
Questions for authors
- How does the performance of classifier-free guidance on the energy above hull as in MatterGen compare to the performance of quality-based supervised fine tuning?
- The CDVAE splits of MP-20 and MPTS-52 are flawed because different polymorphs of the same composition are separated across training, validation, and test sets. What is the performance of FieldForge on the polymorph-split datasets from Martirossyan et al., "All that structure matches does not glitter," NeurIPS 2025, including the METRe and cRMSE metrics?
5 comments
I thank the authors for their reply. However, I am still not convinced that FieldForge introduces any foundational new approach to generative modeling for inorganic materials. Like I said before, it does combine lots of scattered ideas from various papers in a single framework (my previous list did not mention MiAD for a model with variable atom counts). This is a practical thing but, in my opinion, not enough methodological novelty for ICLR.
I do agree with the authors that their QB-SFT is actually novel and I appreciate the new comparison with energy-conditioned CFG. Two follow-up questions about this that might nudge my score:
- The realized on-hull fraction in QB-SFT appears to be quite important. In fact, for an on-hull fraction of 1, SUN+MSUN performance drops to around 30% on MP-20 which curiously enough is quite close to FieldForge-CFG with a target hull energy of 0 eV/atom. Does FieldForge-CFG improve if one increases the target hull energy (which seems similar to reducing the on-hull fraction in QB-SFT)?
- If the authors would like to claim that QB-SFT is helpful independently of the model, I would be curious to see MatterGen's performance under QB-SFT.
Energy-Guided Sampling in FieldForge and Transfer of QB-SFT to MatterGen
Thank you for suggesting these experiments. We extended the CFG target sweep on FieldForge and tested whether QB-SFT transfers to MatterGen on MP-20.
1. Higher CFG targets improve novelty and bring FieldForge’s MSUN to a level comparable to QB-SFT.
The sweep supports the reviewer’s intuition. Our previous reply considered only a target energy above hull of 0 eV/atom. Here we vary the target at a fixed guidance scale of 2.
| FieldForge setting | Novelty | Metastable | SUN | MSUN | SUN+MSUN |
|---|---|---|---|---|---|
| CFG, target 0 eV/atom | 49.28 | 57.96 | 4.64 | 25.84 | 30.48 |
| CFG, target 0.025 eV/atom | 64.08 | 57.88 | 1.92 | 33.40 | 35.32 |
| CFG, target 0.050 eV/atom | 68.48 | 57.00 | 0.88 | 34.28 | 35.16 |
| CFG, target 0.075 eV/atom | 71.12 | 56.44 | 1.00 | 34.72 | 35.72 |
| ────── | ── | ── | ── | ── | ── |
| QB-SFT | 63.56 | 57.00 | 1.88 | 33.96 | 35.84 |
At 0.025 eV/atom, the metastable fraction is almost unchanged from the zero-target run, yet novelty and MSUN both rise. CFG at this target and QB-SFT give similar results across all five metrics. Increasing the target further continues to improve novelty, with little change in combined yield. Our earlier zero-target comparison therefore understated what CFG can achieve on FieldForge.
QB-SFT also provides an effective starting point for RL. With the sampler fixed, subsequent RL raises MSUN from 33.96% to 41.12% while maintaining a similar metastable fraction. In our initialization ablation, RL starting from QB-SFT gives higher SUN+MSUN than RL starting directly from pretraining on both MP-20 and Alex-MP-20.
2. QB-SFT transfers from flow-based FieldForge to diffusion-based MatterGen.
Our previous reply used MatterGen’s released energy-conditioned checkpoint, trained on Alex-MP-20. For this comparison, we followed MatterGen’s setup to train an energy-conditioned model on MP-20. We also applied QB-SFT to MatterGen on MP-20, varied the on-hull fraction in the fine-tuning set, and included random-subset SFT as a control. The table reports results after MatterSim relaxation. Released checkpoint configuration
| MatterGen setting on MP-20 | Novelty | Metastable | SUN | MSUN | SUN+MSUN |
|---|---|---|---|---|---|
| Base model | 69.44 | 37.88 | 0.20 | 19.88 | 20.08 |
| ────── | ── | ── | ── | ── | ── |
| CFG, target 0 eV/atom | 67.96 | 39.08 | 0.52 | 20.08 | 20.60 |
| CFG, target 0.025 eV/atom | 70.88 | 37.20 | 0.28 | 19.52 | 19.80 |
| CFG, target 0.050 eV/atom | 73.40 | 35.28 | 0.36 | 19.16 | 19.52 |
| CFG, target 0.075 eV/atom | 74.48 | 35.88 | 0.56 | 20.16 | 20.72 |
| ────── | ── | ── | ── | ── | ── |
| Random-subset SFT, on-hull fraction 0.438 | 68.44 | 40.04 | 0.48 | 21.68 | 22.16 |
| QB-SFT, on-hull fraction 0.30 | 71.00 | 38.00 | 0.40 | 21.68 | 22.08 |
| QB-SFT, on-hull fraction 0.50 | 72.76 | 36.28 | 0.48 | 21.64 | 22.12 |
| QB-SFT, on-hull fraction 0.70 | 71.52 | 34.68 | 0.56 | 20.28 | 20.84 |
| QB-SFT, main setting | 69.36 | 42.12 | 0.92 | 23.36 | 24.28 |
On MatterGen, novelty also rises with the CFG target, but the metastable fraction generally falls and MSUN changes little. Our main QB-SFT setting retains the base model’s novelty while generating a larger fraction of metastable structures. It gives the highest SUN and MSUN among the settings tested.
The random-subset control helps separate the benefit of data selection from that of additional training. With all SFT runs trained for 1,800 steps, the 30% and 50% on-hull mixtures give combined yields close to random-subset SFT. At 70%, SUN improves slightly but MSUN falls. The main QB-SFT setting improves both metrics over random-subset SFT, showing that the gain on MatterGen depends on the fine-tuning mixture.
MatterGen before relaxation. We evaluated both sweeps on the structures as generated. Row labels follow the table above.
| Unrelaxed MatterGen | Novelty | Metastable | SUN | MSUN | SUN+MSUN |
|---|---|---|---|---|---|
| Base model | 69.80 | 33.44 | 0.12 | 14.48 | 14.60 |
| ────── | ── | ── | ── | ── | ── |
| CFG, 0 | 67.80 | 35.80 | 0.44 | 16.08 | 16.52 |
| CFG, 0.025 | 70.88 | 32.60 | 0.16 | 14.92 | 15.08 |
| CFG, 0.050 | 73.44 | 30.84 | 0.32 | 14.32 | 14.64 |
| CFG, 0.075 | 74.48 | 30.24 | 0.36 | 14.52 | 14.88 |
| ────── | ── | ── | ── | ── | ── |
| Random-subset SFT, 0.438 | 68.72 | 35.32 | 0.24 | 16.16 | 16.40 |
| QB-SFT, 0.30 | 71.04 | 33.72 | 0.16 | 16.24 | 16.40 |
| QB-SFT, 0.50 | 72.76 | 31.20 | 0.20 | 15.84 | 16.04 |
| QB-SFT, 0.70 | 71.40 | 30.16 | 0.16 | 14.44 | 14.60 |
| QB-SFT, main setting | 70.20 | 37.60 | 0.28 | 18.60 | 18.88 |
Before relaxation, zero-target CFG gives the highest SUN. The main QB-SFT setting already generates the largest fraction of metastable, unique, novel structures and gives the highest combined yield.
Technical Contributions, CFG Comparison, and Polymorph-Aware Generalization
We thank the reviewer for the assessment and address the novelty concerns and requested experiments below.
Technical contributions
FieldForge introduces a unified PT–SFT–RL framework for flow-based crystal generation, in which a single joint velocity field over elemental features, coordinates, and lattice variables supports pretraining, quality-based supervised adaptation, and reward-driven optimization. The framework connects explicit control of the stable–metastable balance through data selection with joint RL over composition and geometry. Specifically, QB-SFT adjusts the proportions of on-hull and near-hull structures selected from the original training set to reshape generation preferences without changing the architecture or flow-matching objective, providing an effective initialization for subsequent RL.
Flow Transformer (Crystalite). FieldForge learns a joint velocity field over elemental features, coordinates, and lattice variables, whereas Crystalite uses a diffusion Transformer. Beyond GEM, crystal-periodic 3D-RoPE and lattice-metric flow paths encode periodicity and non-orthogonal geometry. A learnable null-species prototype enables atom counts to be generated jointly with composition (Appendix B).
Joint RL and periodic likelihoods (OMatG-IRL). Our policy jointly reinforces all three crystal variables, unlike Chemeleon2’s latent-space RL and OMatG-IRL’s DNG extension, which fixes discrete species generation (Appendix K). Channel-wise transition likelihoods use a nearest-image approximation for periodic coordinates, active-atom masks, and reference-policy regularization (Eqs. 3–7; Appendix E).
Reward design (Chemeleon2). Shared objectives do not imply identical rewards. We combine dense energy, near-hull and stability bonuses, and SMACT validity; diversity uses element-fraction and AMD descriptors instead of VAE-derived features. Chemeleon2’s default DNG reward lacks the SMACT term. Removing our near-hull, stability, or validity terms reduces SUN+MSUN by 2.24, 4.80, or 3.68 percentage points in the 625-structure internal diagnostic (Appendix F.2).
Anti-annealing (FlowMM). We adopt FlowMM’s linear velocity scaling and show that anti-annealing extends the observed PT metastability–novelty frontier relative to the unannealed checkpoint trajectory. It also increases MSUN across matched training checkpoints without retraining (Figures 2 and 5).
Stability–novelty trade-off. The PT–SFT–RL mean lies beyond the observed non-RL frontier across PT checkpoints (0.9M–2.5M training steps) and PT/PT–SFT anti-annealing sweeps (Appendix F.3/Figure 5). With the sampler fixed, RL after QB-SFT raises MP-20 novelty from 63.56% to 75.48% and MSUN from 33.96% to 41.12%, while maintaining similar metastability (57.00% → 57.32%).
The full pipeline raises MSUN over PT from 29.88% to 41.12% on MP-20 and 42.04% to 53.20% on Alex-MP-20, and achieves higher SUN+MSUN than direct PT–RL on both datasets.
Energy-conditioned CFG versus QB-SFT
We tested CFG with guidance 2 and target hull energy 0 eV/atom. FieldForge-CFG and QB-SFT use the same pretrained checkpoint, matched post-training budgets, and identical LeMat-GenBench evaluation. MP-20 results (%) are:
At similar metastability, QB-SFT exceeds FieldForge-CFG by 14.28 points in novelty and 5.36 points in SUN+MSUN; CFG has higher SUN. Thus, tested CFG settings do not reproduce QB-SFT’s higher novelty and combined discovery yield. The equal-sized random-SFT control on MP-20 further supports the selection rule (Appendix D.1). And QB-SFT’s selector is model-agnostic by design.
Polymorph-aware generalization
We trained FieldForge from scratch on MP-20-polymorph-split, with no SFT or RL. Revised Table 3 includes the baselines from Martirossyan et al., Table 2:
FieldForge achieves the highest METRe and lowest reported RMSE/cRMSE among the compared models. Following the benchmark protocol of Martirossyan et al. cited by the reviewer, we evaluate all 9,032 test structures, generating one candidate per reference, using OMatG’s official all/unfiltered evaluation (Appendix G.4). Martirossyan et al. do not provide a polymorph-aware split for MPTS-52, so we retain its standard chronological split.