- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The authors propose an all-atom joint generative model, OCFlow, for inorganic materials de novo generation (DNG) which leverages torus-aware optimal transport coupling to yield stronger out-of-distribution inference and more efficient sampling measured in terms of NFE. They demonstrate the efficacy of their proposed architecture on unit cells of varying size, showcasing strong size-transferability performance and claim robustness at low NFE.
The main contributions are:
- A torus-aware optimal transport coupling.
- Quantitative evidence that OCFlow balances the speed vs. quality tradeoff in DNG).
- An MLIP-based benchmark for evaluating generative models for DNG is proposed and used to stress-test OCFlow.
OCFlow is validated using stability/metastability, uniqueness, and novelty (SUN/MSUN) metric as well as structural and compositional validity rates with respect to competing models. Formation energy of generated structures is also reported demonstrating a lower-shifted density of for OCFlow. OCFlow is competitive with respect to state-of-the-art models.
Overall the paper is well-written and the results support the central claim that optimal transport coupling during training results in more robust OOD sampling. My main concerns regarding this work are the proposed novelty of the optimal transport coupling for periodic fractional coordinates, the purported contribution of a new benchmark, as well as a lacking comparison to existing models. An additional concern, though not critical, is that while validity remains high for low-NFE sampling, the performance of the model in terms of SUN and MSUN drops dramatically.
Critical strengths
The most convincing demonstration of OCFlow is its performance on size-transferable inference for inorganic crystals. it is clear based on the results in Table 2 and Figure 2 that OOD performance is superior to other architectures. It is also clear that OCFlow trained on MP-20 and MPTS-52 is more robust to OOD systems than other models while remaining strong in distribution. I think that size-transferable generation is a consequential and yet relatively underexplored avenue for inorganic DNG and the authors demonstrate that, at least according to validity and MSUN, OCFlow is well-suited to this task.
Critical weaknesses
Broadly speaking I think that some of the central claims are overstated. Firstly, I believe that the proposed optimal transport coupling is not completely novel. While the authors present torus-aware OC for fractional coordinate flow matching as their first novel contribution, it is my understanding that the Open Materials Generation (OMatG) model attempted something very similar to this already. A high-level comparison of the two approaches and how they differ would make it more clear why the optimal coupling proposed by OCFlow is advantageous and distinct. Similarly, I think that the claimed "few-step benchmark" is somewhat of an overstatement. The authors rather seem to apply well-known benchmarks and evaluation metrics to existing models at various NFE but I am not convinced that this is enough to constitute the claims of a new benchmark.
A second key weakness that I believe should be addressed during the rebuttal period is more comparisons with existing models. There are a few noticeable missing competitors like Crystalite, MiAD, and OMatG. Crystalite, for example, is the current SOTA on LeMat-GenBench and, thus, a comparison with them seems quite relevant. How does OCFlow compare to these models when considering standard and low-NFE inference? On this note, the best LeMat-GenBench SUN rate on MP-20 models has bee reported over 1 % whereas in Figure 3 it does not appear that OCFlow is at this level. Clarification on if OCFlow structures were pre relaxed before SUN computation would help shed light on this.
Questions for authors
- There seem to be many different proxies for thermodynamic stability used in this work. Tables 1 and 2 list stability in terms of formation energy but do these rankings hold when considering energy above the convex hull?
- How does the ablation in Table 5 change if one considers 100 or 1000 NFE? It is a strong finding that optimal transport coupling improves performance appreciably for 10 NFE but is this consistent at greater NFE as well?