- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The author/s take SGEquiDiff-style crystal generation and extends it to layer groups, so 2D materials like monolayers and bilayers come out with exact layer-group symmetry while bulk crystals still work.
Critical strengths
-
As far as I know, this is the first model that enforces exact layer-group symmetry, and the crystallographic groundwork is careful.
-
The benchmark fills a real gap for 2D generation
Critical weaknesses
-
I don't see a consistent gain from layer groups. DiffCSP++ gets a lower layer-group JSD than SLayerGen on every dataset, though the paper marks SLayerGen as best on three. SLayerGen also trails the best P1 baseline on S.U.N. structure rate, and its own P1 variant does as well or better on both S.U.N. rates where tested.
-
the method is mostly SGEquiDiff by Chang et al. (Space Group Equivariant Crystal Diffusion) adapted to layer groups. Symmetry-aware 2D generation also exists already, e.g., Xu et al. (Discovery of 2D Materials via Symmetry-Constrained Diffusion Model) and Rønne and Hammer (Atomistic Generative Diffusion for Materials Modeling, not cited), so saying prior models ignore layer groups is too strong.
Questions for authors
- the layer group is sampled from training frequencies, so why is the layer-group JSD of SLayerGen worse than DiffCSP++ and, on two datasets, SLayerGen3D? -what can the layer-group model do that the P1 variant cannot?
3 comments
the layer group is sampled from training frequencies, so why is the layer-group JSD of SLayerGen worse than DiffCSP++ and, on two datasets, SLayerGen3D?
Thank you for the question - we note that the JSD is computed only over structurally and compositionally valid crystals. We will update the text to specify where SLayerGen actually wins, which is on JSD for the 18 layer groups that don’t map injectively to a space group (table 8). The reason that DiffCSP++ performs well on JSD over all layer groups is that (1) many of them do map injectively to a space group and (2) DiffCSP++ fixes generated crystals to have combinations of space groups and Wyckoff positions in the empirical distribution; in contrast, SLayerGen must learn the Wyckoff positions, which is a more difficult learning task but enables the model to generate more diverse templates. We will promote table 8 to the main text to avoid confusion.
what can the layer-group model do that the P1 variant cannot?
The layer group model can generate realistic, high symmetry crystals and be constrained at inference time to any desired layer group. Given our result that SLayerGen can generalize to rare or unseen groups, we believe this is a strong capability given that many exotic quantum and topological properties of 2D materials are highly symmetry-dependent. In table 6, we also show that the layer group model has significantly better JSD over the layer groups than the P1 model variants, demonstrating improved learning of the distribution of layer group symmetries.
My main concern still stands. The claim of the best overall layer-group JSD was wrong, since DiffCSP++ is lower on every dataset. The response does not explain why SLayerGen trails SLayerGen3D on two datasets, and it does not address the gap to the P1 baselines on S.U.N. structure rate. On top of that, your own P1 variant beats SLayerGen on both S.U.N. rates on both datasets where it was tested. So the main benefit of layer groups is symmetry control, not higher S.U.N. rates than P1 models. I will keep my score. Thanks for the response.
We thank the reviewer for the thoughtful comments and questions. Below we aim to address your concerns. If you find our answers helpful, we would appreciate it if you would consider raising your score.
We highlight that SLayerGen consistently does well on the SUN template rate. We will also update the text to specify that SLayerGen consistently wins on JSD only for the 18 layer groups that don’t map injectively to a space group (table 8). The reason that DiffCSP++ performs well on JSD over all 80 groups is that (1) many of them do map injectively to a space group and (2) DiffCSP++ fixes generated crystals to have combinations of space groups and Wyckoff positions in the empirical distribution; in contrast, SLayerGen must learn the Wyckoff positions, which is a more difficult learning task but enables the model to generate more diverse templates.
We also highlight results from experiments requested by reviewer hdGM, wherein we found SLayerGen is the only benchmarked model able to generalize to rare and held-out layer groups in Alex2D. Specifically, we trained SLayerGen with all crystals from 3 layer groups held out (rare layer groups 31 and 8, and the common layer group 11), and then constrained SLayerGen to generate crystals with those layer group symmetries at inference time. Impressively, we find SLayerGen generalizes to held-out layer groups as evident by nonzero SUN structure/template rates and rediscovery of a few heldout crystals. Meanwhile, DiffCSP and MatterGen achieve zero success at generating any crystals in rare layer groups even when trained on them, while by construction, DiffCSP++ cannot generate held-out groups. "In group” is the rate of generated structures with the prescribed layer group. “Rediscovered” denotes held-out structures generated by the model. Brackets give 95% CIs.
\hline \text{Layer group (train count)}&\text{Model}&\text{Group in training}&\text{In group}&\text{Valid}&\text{UN}&\text{Stable}&\text{SUN}&\text{SUN template}&\text{Rediscovered}\\\\ \hline \text{LG 8 (53)}&\text{SLayerGen}&\text{Yes}&96.5&65.0\ {\scriptstyle[62.9,\ 67.1]}&81.5&57.6\ {\scriptstyle[54.5,\ 60.6]}&41.8\ {\scriptstyle[38.8,\ 44.9]}&8.4\ {\scriptstyle[6.8,\ 10.3]}&3/29\\\\ &\text{SLayerGen}&\text{No}&96.4&58.7\ {\scriptstyle[56.5,\ 60.8]}&80.2&52.9\ {\scriptstyle[49.8,\ 56.0]}&38.0\ {\scriptstyle[35.0,\ 41.0]}&5.9\ {\scriptstyle[4.6,\ 7.5]}&3/29\\\\ &\text{MatterGen}&\text{Yes}&0&\text{–}&\text{–}&\text{–}&\text{–}&\text{–}&\text{–}\\\\ &\text{DiffCSP}&\text{Yes}&0&\text{–}&\text{–}&\text{–}&\text{–}&\text{–}&\text{–}\\\\ &\text{DiffCSP++}&\text{No}&0&\text{–}&\text{–}&\text{–}&\text{–}&\text{–}&\text{–}\\\\ \hline \text{LG 11 (883)}&\text{SLayerGen}&\text{Yes}&99.9&71.0\ {\scriptstyle[69.0,\ 73.0]}&91.8&52.0\ {\scriptstyle[48.9,\ 55.1]}&45.6\ {\scriptstyle[42.5,\ 48.7]}&3.0\ {\scriptstyle[2.1,\ 4.3]}&5/373\\\\ &\text{SLayerGen}&\text{No}&99.8&53.4\ {\scriptstyle[51.3,\ 55.6]}&91.3&41.2\ {\scriptstyle[38.2,\ 44.3]}&34.4\ {\scriptstyle[31.5,\ 37.4]}&3.3\ {\scriptstyle[2.4,\ 4.6]}&1/373\\\\ &\text{DiffCSP++}&\text{No}&0&\text{–}&\text{–}&\text{–}&\text{–}&\text{–}&\text{–}\\\\ \hline \text{LG 31 (42)}&\text{SLayerGen}&\text{Yes}&98.3&71.2\ {\scriptstyle[69.2,\ 73.2]}&79.6&44.8\ {\scriptstyle[41.7,\ 47.9]}&31.1\ {\scriptstyle[28.3,\ 34.0]}&3.7\ {\scriptstyle[2.7,\ 5.1]}&2/17\\\\ &\text{SLayerGen}&\text{No}&99.5&79.9\ {\scriptstyle[78.1,\ 81.6]}&80.7&32.1\ {\scriptstyle[29.3,\ 35.1]}&22.5\ {\scriptstyle[20.0,\ 25.2]}&1.6\ {\scriptstyle[1.0,\ 2.6]}&1/17\\\\ &\text{MatterGen}&\text{Yes}&0&\text{–}&\text{–}&\text{–}&\text{–}&\text{–}&\text{–}\\\\ &\text{DiffCSP}&\text{Yes}&0&\text{–}&\text{–}&\text{–}&\text{–}&\text{–}&\text{–}\\\\ &\text{DiffCSP++}&\text{No}&0&\text{–}&\text{–}&\text{–}&\text{–}&\text{–}&\text{–}\\\\ \hline \end{array}$$ > Symmetry-aware 2D generation also exists already, e.g., Xu et al. and Rønne and Hammer, so saying prior models ignore layer groups is too strong Thank you for your comment. We will update the text to state that prior models do not enforce layer group symmetry during generation. Xu et al. uses the same framework of DiffCSP with post-hoc projection to specified space groups; they are not invariant to layer groups nor do they allow constrained generation to layer groups which map non-injectively to the layer groups. We benchmarked the space group constrained diffusion model DiffCSP++ as a direct analog to Xu et al.’s work. Thank you for sharing the Rønne and Hammer model AGeDi; we will make sure to cite it. However, we highlight that AGeDI is not symmetry constrained and instead softly conditions on layer groups using classifier free guidance; doing so achieves only ~38% accuracy in sampling the correct layer group as noted in their paper, while SLayerGen enforces layer group symmetries by construction.