- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The authors present an overview of their attempt to train a flow matching model to predict crystal structures from PXRD profiles. They show that the model performs poorly and fails to learn meaningful use of the PXRD conditioning. The manuscript continues with reporting a series of diagnostics to find out why the model doesn't train. Despite this the model never approaches anything comparable to a state-of-the-art. Based on this, the authors then try to make generalizable conclusions about training conditional generative models on scientific data.
Critical strengths
The authors present several diagnostic approaches that may be useful for finding out why a model doesn't learn in general.
Critical weaknesses
The authors never compare to a SOTA model, but checking the literature myself they don't seem to even approach SOTA results. There are several issues in the dataset, acknowledged by the authors themselves. Moreover there can be unaddressed issues in the implementation of the model or training protocol. Any patterns observed in this setup so far away from a valid training regime and cannot readily be assumed to generalize to other applications of conditional generative modelling. Despite this the central thesis of the manuscript is generalizability to other applications.
Questions for authors
- Can the authors compare to SOTA?
- Can the authors fix the issue in the dataset and report performance?
- How are the authors certain their implementation is working?
- Can the authors please roll back generalizability claims or substantiate them with a better model?