- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposed MeanFlux, which is used for crystal generation and crystal structure prediction. In this work the author uses a frozen VAE to encode the crystal into a fixed latent representation and then applies a DiT transformer to denoise the latent representation using mean flow, and then finally it applies RL fine-tuning. Experimental results on MP20 are promising, and the author explores an interesting idea for crystal generation.
Critical strengths
In this work, few-step sampling helps RL fine-tuning cheaply. The main reason is each rollout is cheap, and NFT fine-tuning is practical. It roughly doubles relaxed LeMat-GenBench mSUN at the same sampling cost. Experimental results are promising on MP-20 from Table 1 for de-novo material generation. Also this paper shows that using their model crystals can be generated faster than existing baseline models like Chemeleon2.
Critical weaknesses
This paper compares only against diffusion and flow-based models in Table 1 and Figure 1 and omits recent LLM-based generative models; these are Crystal-text-LLM [1], FlowLLM [2], CrysLLMGen [3], and PLaID++ [4]. These methods obtain strong stability and novelty on MP-20, and PLaID++ is the direct reward-based fine-tuning, which makes it directly relevant to this paper. Since this paper claims the best mSUN at a given sampling cost, for fair comparison, these models should be placed on the compute-quality frontier (Figures 1 and 2) under the same relaxation protocol. Without them, it is unclear whether MeanFlux's advantage holds against the strongest current generators or only against diffusion-based baselines.
The main weakness of this work is that it lacks methodological novelty. This work uses existing components from prior work. It uses ADiT and Chemeleon2 for latent space design; the generative objective is MeanFlow, and reward fine-tuning uses NFT taken from MeanFlowNFT. Thus, this paper's contribution primarily applies existing techniques to the generation of crystals. The paper should clarify which design choices are specific to crystals (e.g. the reward design) and why they are non-trivial.
[1] https://arxiv.org/abs/2402.04379 [2] https://arxiv.org/abs/2410.23405 [3] https://arxiv.org/abs/2510.23040 [4] https://arxiv.org/abs/2509.07150
Questions for authors
Could the authors compare MeanFlux with more recent LLM-based crystal generators like PLaID++, FlowLLM, Crystal-text-LLM, and CrysLLMGen? Could the authors include the stability metric in the Table 1 for all baselines and MeanFlux? Could the authors include a methodology diagram or any pseudocode for the better understanding of their proposed framework? Otherwise, it is hard to follow the workflow.
4 comments
Hey Reviewer 7g3f,
As the rebuttal period is drawing to an end, would you please let us know if we have addressed your concerns in our previous response, where we have benchmarked additional baselines, clarified our contribution, and crafted a methodology chart for easier understanding?
Best, Paper Authors
Comment on Revision
Thank you for the revisions. While the revision has improved clarity, my overall assessment remains unchanged due to the following concerns:
Comparing against pre-released Crystal-text-LLM samples does not constitute a controlled evaluation under identical budgets.
The crystal-specific design choices like multiplicative reward in my opinion is a practical engineering adaptations and not something fundamental advancement.
It is still not clear whether the performance gains are reward, objective or interaction.
Thank you for reading our revision so closely and for the follow-up. We address your three points below.
A controlled comparison with Crystal-text-LLM. Since the model weights are not public, we now directly ran the authors' own stability protocol from their released code (M3GNet relaxation against the Materials Project hull) on both models, through one pipeline with the same validity filter and relaxation settings, on 1,024 random samples each.
| Model | Metastable (M3GNet) | Below hull (M3GNet) |
|---|---|---|
| Crystal-text-LLM 70B | 45.9% | 8.4% |
| MF-NFT | 78.5% | 32.3% |
Under the very same protocol, MF-NFT reaches 78.5% metastable, about 33 points higher. A third of its samples fall below the hull, compared with 8% of theirs, and each one costs roughly 1/1,600 as much to generate. Their protocol does not require novelty or uniqueness. Once both are required, as on LeMat-GenBench, the gap widens to 11.4% SUN against 0.5%. Their DFT column needs VASP, which we could not run within this short period, so we report the M3GNet column here.
Reward, objective, or both. Thanks for asking this directly. Both really mattered in our runs, which we demonstrated on their own.
- Objective. Before NFT, we fine-tuned the same model with GRPO (the objective Chemeleon2 uses) in eleven variants. The best of them gained only 4 points of mSUN on the MP-20 suite, while NFT gained 24. GRPO needs a stochastic sampling trajectory, and adding noise to an 8-step sampler pushes samples off the data manifold. NFT has no such requirement, so that it suits few-step models naturally.
| Objective | mSUN before | mSUN after | SUN before | SUN after |
|---|---|---|---|---|
| GRPO (11 attempted variants) | 0.351 | 0.327–0.391 | 0.087 | 0.070–0.100 |
| NFT (MF-NFT) | 0.335 | 0.575 | 0.072 | 0.141 |
- Reward. With NFT fixed, our improved recipe raised relaxed mSUN from 22.5% to 41.2%, and the diversity weight alone adds 9 percentage (Table 2). The gain also holds up well. Every setting in our sensitivity analysis raises mSUN at least 2.9 times over the starting model (Appendix B.3, Table 2), and even with no LeMat-GenBench metrics in the reward (i.e. only mp20), MF-NFT still reaches 6.2% SUN.
We tried and finally adopted the effective objective with careful reward design to achieve the frontier performance. Similar rewards gave little under GRPO and paid off only once the objective fit a few-step sampler.
On the design choices. We understand the view that some of these are engineering adaptations, and we are not claiming a new learning principle. Still, each of them came out of extensive experiments and turned out to be necessary. Training does not even converge without the per-crystal cap (Appendix B.9), and the reward and objective above account for most of the gain. Beyond the method itself, the paper also changes how generators are compared. Rather than ranking models by hit rate alone, we measure how many SUN crystals each one yields for a fixed compute budget, which is what a screening pipeline actually is paying for. We re-score every baseline under one protocol, measure its cost on the same hardware, and place all of them on this discovery-efficiency frontier, both for generation alone and with relaxation included (Figures 1 and 12). On this frontier, MF-NFT yields the most SUN crystals per unit of compute among all models we evaluated, 9 times as many as Chemeleon2 with RL from generation alone and 2.5 times as many with relaxation. We believe this result, together with a frontier that future models can be placed on directly, is a useful contribution to the community.
Thank you again for your time and for helping us strengthen the paper.
Thank you for your review, for the four additional references, and for noting that few-step sampling makes our reward fine-tuning cheap. We have added the comparison where weights were available, a Stable column and a pipeline figure.
W1/Q1. LLM-based generators. Of the four models, only PLaID++ has released weights, so we ran it with the authors' code (Qwen2.5-7B with the DPO adapter, temperature 1.3). Since PLaID++ is preference-aligned for stability, we compare it with MF-NFT. We scored it on relaxed LeMat-GenBench and added it to Table 11 (Appendix B.13), and to the new end-to-end comparison in Figure 12 (Appendix B.14). Values are % of 1,024 draws.
MF-NFT reaches four times the SUN rate of PLaID++ at about 1/152 of its cost per structure. The gap remains once relaxation is included, where MF-NFT needs 25 TFLOPs per SUN crystal compared with 178 for PLaID++ (Table 4, Figure 12). Most PLaID++ samples are not novel against LeMat-Bulk (27% novel, compared with 84% for MF-NFT).
We also scored 1,024 of the LLaMA-2 70B samples that the Crystal-text-LLM authors published, under the same protocol. It reaches 0.5% SUN and 3.9% mSUN, far lower than our method.
W2. Novelty. Please see the general response. In short, we made three design choices specific to crystals. First, crystals have different numbers of atoms, and training will diverge unless we cap the JVP correction for each crystal (Appendix B.9). Second, our reward multiplies the stability terms by validity, uniqueness and novelty, so a crystal is rewarded only if it is new and stable at once, whereas using an additive counterpart rewards a known low-energy crystal. With this reward, NFT raises SUN 7.8 times over its starting model, compared with 1.7 times for GRPO on Chemeleon2 (Table 11). Third, for structure prediction we retrain on a geometry-only latent (from a new VAE) with the known species attached to each atom, which beats Crystalite on MP-20 at about 90 times less compute (Section 3.5).
Q2. Stability. We added a Stable column to Table 1. MeanFlux at eight steps has the highest SUN and the second-highest stable rate (0.160). Only ADiT has a higher stable rate (0.170), but it is also the least novel model.
Q3. Pipeline. We added Figure 6 (Appendix B.1), which now thoroughly shows the training, sampling and fine-tuning stages, each with its pseudocode. We hope this adds more clarity.
Please let us know if you have further questions. We are happy to address them!