- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper introduces CrysPFlow, a flow model for crystal structure relaxation. The authors introduce endpoint-preserving physics injection using the MLIP forces to adjust the computed geometric gradients. Specifically, they add the orthogonal component of the physics loss gradient to the geometric gradient, which is passed to the optimizer. The results show that their model has lower structural error on the SrTiO3 and X-Mn-O datasets than the baselines and that their physics guidance helps not only their model but also the adapted flow matching baselines.
Critical strengths
Motivation for endpoint-preserving physics injection is reasonable.
Critical weaknesses
While the proposed method appears technically sound, I have substantial concerns about the implementation and the rigor of the experimental evaluation.
1. Insufficient methodological details and reproducibility
The paper lacks several important details needed to reproduce the work. These omissions raise substantial concerns about the reproducibility of the experiments, the fairness of the baseline comparisons, and the reliability of the resulting conclusions.
- Model architecture: The authors do not describe their own backbone, despite Table 4 showing that it already outperforms the other backbones even without physical guidance. The model architecture, number of layers, and parameter count are not specified.
- Baseline adaptation: The paper refers to an RG-VFM [1] backbone, although RG-VFM primarily introduces a training objective, and its material model builds on MOFFlow [2], which uses an IPA-based architecture modeling building-block translations, rotations, and lattice parameters. It is unclear how this architecture was adapted to inorganic crystals without such building blocks. More broadly, the paper provides insufficient details on how the other flow matching baselines were adapted for crystal relaxation. Furthermore, all adapted baselines use 20 Euler inference steps, deviating from their original inference configurations, without justification or a step-count ablation.
- Dataset construction: The datasets lack important details about their source/curation, preprocessing, and number of data points. In particular, the SrTiO3 dataset uses ground-truth atomic annotations as , but how is the initial structure determined?
- Reproducibility: Although some training hyperparameters are provided, GPU specifications, training time, and inference latency are missing. No reproducible code is provided, making it difficult for practitioners to reproduce the results.
2. Limited evaluation of crystal relaxation
- Missing MLIP baselines: Crystal structure relaxation is regularly performed using iterative optimization with MLIPs. While the authors motivate direct prediction models by arguing that MLIP-based relaxation remains computationally expensive, they do not compare against iterative MLIP relaxation to establish the accuracy–compute trade-off. A matched-budget comparison (e.g., with a few MLIP steps) would also be helpful.
- Limited benchmarks: The evaluation is restricted to SrTiO3 and X-Mn-O. A related structure relaxation task, IS2RS, was introduced in OC20 [3] and also studied in OC22 [4]. Although these benchmarks focus on adsorbate–surface systems rather than bulk crystals, the paper does not discuss this related work or evaluate the proposed method on broader, established relaxation benchmarks.
- Robustness across MLIPs: The proposed method is evaluated using only MACE for physical supervision. Experiments with other MLIPs would help establish whether the improvements generalize across different learned energy landscapes.
[1] Zaghen, Olga, et al. "Riemannian variational flow matching for material and protein design." International Conference on Learning Representations. Vol. 2026. 2026.
[2] Kim, Nayoung, et al. "Mofflow: Flow matching for structure prediction of metal-organic frameworks." International Conference on Learning Representations. Vol. 2025. 2025.
[3] Chanussot, Lowik, et al. "Open catalyst 2020 (OC20) dataset and community challenges." Acs Catalysis 11.10 (2021): 6059-6072.
[4] Tran, Richard, et al. "The Open Catalyst 2022 (OC22) dataset and challenges for oxide electrocatalysts." Acs Catalysis 13.5 (2023): 3066-3084.
Questions for authors
Mainly, my questions are about implementation and experimental design, as stated in the weaknesses section -- e.g.,
- What model architectures do the authors use? What GPU resources did the authors use?
- What do the authors mean by the RG-VFM backbone? The backbone RG-VFM uses for metal-organic frameworks assumes that crystals can be decomposed into building blocks, which isn't obvious for inorganic crystals.
- How did the authors construct the dataset? Did the authors consider evaluating on more established benchmarks like OC20/22?
- How do the model's speed and accuracy compare to MLIPs?
- Does the endpoint-preserving physics injection method show benefit across other MLIPs?
4 comments
4 Single-step inference accuracy, network-call efficiency, and sensitivity to step count (Weakness 1)
We thank the reviewer for drawing attention to the inference configuration and fairness of the comparisons. The 20-step setting is the predefined, uniform network-call budget for the three externally adapted baselines; the main CrySPFlow model uses a single endpoint prediction, with NFE = 1. Section 5.1 of the original manuscript states that the main model uses one forward evaluation, while the cross-backbone experiments specify: “The external adapters used 20 Euler steps at inference.” We will present this accuracy–inference-cost comparison more clearly: CrySPFlow achieves higher structural accuracy with fewer network calls than the externally adapted models using multi-step inference.
The table below directly reports the prediction errors and network-call counts for the “+MACE+Unrolling” configuration in Table 4 of the original manuscript.
| SrTiO₃ model | Coordinate MAE ↓ (Å) | Bond-length MAE ↓ (Å) | Network calls (NFE) ↓ |
|---|---|---|---|
| CrySPFlow | 0.008021 | 0.005415 | 1 |
| FlowMM | 0.014000 | 0.006823 | 20 |
| CrystalFlow | 0.013137 | 0.007448 | 20 |
| RG-VFM | 0.014814 | 0.013920 | 20 |
Specifically, in Table 4 of the original manuscript, the full CrySPFlow configuration achieves a coordinate MAE of 0.008021 Å and a bond-length MAE of 0.005415 Å with 1 network call. Both errors are lower than the respective best results reported for the three external adapters with 20 network calls. Relative to the best coordinate MAE among the external models (CrystalFlow, 0.013137 Å) and the best bond-length MAE (FlowMM, 0.006823 Å), these errors are reduced by 38.94% and 20.64%, respectively, with 95% fewer network calls. This demonstrates the accuracy and network-call efficiency advantages of single-step endpoint prediction under the evaluated configurations.
For each external backbone, the geometric baseline, local-feedback variant, and unrolled-feedback variant use the same sampler, NFE, and sampling seed. None of the physical-supervision variants calls MACE at inference. The uniform 20-step setting therefore also allows same-backbone comparisons to assess the contribution of physical supervision without changes in the number of inference calls.
5 Accuracy and computational-cost comparison with iterative MLIP relaxation (Weakness 2, Question 4)
We have added a MACE+FIRE iterative relaxation baseline, evaluated on all 2,859 X–Mn–O test structures with budgets of at most 1, 5, 20, and 200 steps. These budgets cover both a small number of MLIP iterations and more extensive optimization.
Table R5. MACE+FIRE results on the X–Mn–O test set. Coordinate, bond-length, and lattice errors are in Å, and volume errors are in ų. Atomic positions and the cell are optimized jointly, with no computation or evaluation failures at any budget.
| Maximum steps | Coord ↓ | Bond ↓ | Lat.-R ↓ | Lat.-A ↓ | Vol. ↓ | Rel.V (%) ↓ | Match (%) ↑ | Mean MLIP queries | Mean runtime (ms/structure) ↓ |
|---|---|---|---|---|---|---|---|---|---|
| ≤1 | 0.306436 | 0.432941 | 0.217076 | 0.219603 | 32.150206 | 19.744 | 65.338 | 1.99 | 59.444 |
| ≤5 | 0.263586 | 0.349022 | 0.188040 | 0.187460 | 26.662356 | 15.278 | 68.381 | 5.93 | 185.376 |
| ≤20 | 0.178742 | 0.210359 | 0.128176 | 0.114074 | 11.254481 | 6.456 | 77.125 | 19.24 | 614.411 |
| ≤200 | 0.166928 | 0.179657 | 0.126237 | 0.093545 | 8.039689 | 4.385 | 82.581 | 51.73 | 1691.211 |
The additional MACE+FIRE experiments directly cover low-budget iterative relaxation. As the step cap increases from 1 to 200, coordinate error decreases from 0.306436 to 0.166928 Å, Match increases from 65.34% to 82.58%, and mean runtime increases from 59.44 to 1,691.21 ms/structure. Under the 200-step cap, 98.15% of structures meet the MACE force/stress stopping criteria. Further improvements with a larger budget incur additional cost: increasing the step cap from 20 to 200 reduces coordinate error by 6.61% while increasing runtime by a factor of 2.75.
For inference cost, CrySPFlow's independent endpoint timing records 5.35 ms/structure, with only one network evaluation and no MLIP queries at inference. This timing and the above supplementary model-accuracy results come from independent runs. The recorded FIRE runtimes across the four budgets range from 11.12 to 316.31 times this value.
These results directly support the design of using physical feedback during training while retaining low-cost endpoint prediction at inference. We will include this accuracy–runtime comparison in the camera-ready version.
6 Validation on OC20/OC22 and with other MLIPs (Weakness 2, Questions 3 and 5)
The additional OC22 experiments directly address the suggestion to evaluate on an established surface-relaxation dataset, yielding a coordinate MAE of 0.14711 Å and an ADwT of 47.13%. OC22 provides IS2RS initial–relaxed structure pairs, as well as energy- and force-related tasks. We will add OC20 IS2RS and OC22 to the related work and explain the differences between the fixed-atom/fixed-cell constraints in surface relaxation and the joint recovery of bulk atomic coordinates and cells in X–Mn–O. We selected OC22 for this round.
Summary of reported OC22 results.
| Method / task adaptation | Coordinate MAE (Å) ↓ | Mean displacement error (Å) ↓ | Vector RMSE (Å) ↓ | ADwT (%) ↑ |
|---|---|---|---|---|
| DeepRelax | 0.187175 | 0.378482 | 0.516979 | 40.36 |
| RG-VFM | 0.185394 | 0.376220 | 0.510792 | 40.44 |
| CrySPFlow | 0.147110 | 0.297060 | 0.407720 | 47.13 |
The CHGNet and MatterSim results (see Table 3 in our response to Reviewer 2) address compatibility with other MLIPs. In the tested endpoint-preserving, terminal-local implementation, both teachers improve coordinate and bond-length errors and the matching rate relative to the geometric baseline. These improvements come at the cost of small increases in lattice/volume errors. The revised manuscript will specify the exact variant, without extending this observation to all gradient paths or all teachers.
We thank the reviewer for the suggestions, which help us present the methodological contributions and supporting evidence more precisely. In the camera-ready version, we will consolidate the implementation and data protocols, correct the RG-VFM terminology, include the additional material/teacher/stress and MLIP-budget experiments, and clarify the scope of the statistical and comparative conclusions.
If these clarifications and additional results have addressed your concerns, we would be grateful if you could consider a more favorable overall assessment of our paper.
I thank the authors for their response. My main concerns about baseline fairness and reproducibility are only partly addressed. Most changes are promised for the camera-ready version, but the manuscript has not yet been revised.
1. RG-VFM baseline The response confirms that "RG-VFM backbone" is the RG-VFM objective applied to the authors' own atom-level periodic network. This model does not exist in prior work, yet the paper presents it as an established baseline. Because the network is not described, it is unclear how it differs from "Our backbone." Its weak performance may therefore reflect the authors' implementation.
2. Baseline inference settings The FlowMM and CrystalFlow adaptations are still not described, and the baselines use 20 Euler steps instead of their original inference settings. The response justifies this by noting that CrySPFlow needs fewer network calls. However, this efficiency does not justify running the baselines with weaker settings than those in their original papers.
3. Architecture and reproducibility Thank you for the X-Mn-O details. However, the SrTiO3 backbone is still not described, although it accounts for most of the gain on SrTiO3. Baseline parameter counts and dataset sources, sizes, and splits are also missing.
4. OC22 evaluation I appreciate the OC22 experiments. However, the baselines again include the self-implemented RG-VFM (see point 1) and omit the standard IS2RS baselines from the OC22 paper (e.g., SpinConv, GemNet-dT, and GemNet-OC). The reported 47.13% ADwT is only slightly above the initial-structure baseline, which involves no learning (44%), and well below GemNet-OC trained on OC22 (58%).
Overall, the remaining concerns about baseline implementations and reproducibility limit the credibility of the results, so I will keep my score.
We thank the reviewer for the specific suggestions!
We thank the reviewer for the specific suggestions on reproducibility and experimental rigor. We agree that the original manuscript did not sufficiently distinguish the training objective, prediction network, and task adaptation. Below, we provide the network architecture and computational resources for X–Mn–O and clarify data construction, inference budgets, and comparisons with iterative MLIP relaxation.
1 Network architecture, GPU resources, and reproducibility (Weakness 1, Question 1)
We provide the network architecture and computational resources for the X–Mn–O periodic endpoint model below.
For X–Mn–O periodic crystals, we use an atom-level message-passing endpoint network with a hidden dimension of 512, a time-embedding dimension of 256, and 128 periodic Fourier frequency terms per coordinate axis. The network comprises 7 blocks: 1 initial context block and 6 subsequent message-passing blocks, with LayerNorm, SiLU, and residual connections. The atom-level prediction head outputs fractional-coordinate corrections, followed by removal of global translation and periodic wrapping. The pooled prediction head outputs corrections to the 6 independent components of the symmetric logarithmic cell metric. The predictor has 14,269,190 trainable parameters, approximately 14.27M, excluding the frozen potential model.
The original X–Mn–O training used a single NVIDIA RTX PRO 5000 Blackwell GPU and took approximately 48 hours in total. Single-step endpoint prediction has an average latency of 5.347 ms/structure, requires only one prediction-network call, and makes no MACE queries at inference. The timing scope is detailed in Sec. 5.
We agree that the backbone is already strong without physical supervision, indicating that the network itself also contributes to performance. We therefore assess the contribution of physical supervision through same-backbone comparisons, such as Table 3 of the original manuscript, rather than attributing all performance differences between architectures to physical feedback. In the camera-ready version, we will consolidate the descriptions of the network, computational resources, and baseline adaptations, and provide training/inference implementations, fixed configurations, and details of data processing and evaluation.
2 Meaning of “RG-VFM backbone” and baseline adaptation (Weakness 1, Question 2)
The reviewer is correct that RG-VFM defines a training framework rather than a unique backbone architecture, and our original terminology was imprecise. Its original MOF experiments indeed use MOFFlow's rigid-building-block representation. In contrast, our inorganic-crystal experiments adapt the endpoint objective to an atom-level periodic network, without using MOFFlow's IPA architecture or requiring crystals to be decomposed into MOF building blocks. Atomic species are given as known inputs, and the network directly predicts atomic coordinates and the cell. We will use the term “atom-level adaptation of the RG-VFM objective” and describe the objective and architecture separately. This distinction is consistent with the original RG-VFM and MOFFlow methods.
3 Data sources, construction, and Y₀ for SrTiO₃ (Weakness 1, Question 3)
For SrTiO₃, Y₁ consists of reference atomic annotations, whereas Y₀ is constructed from the observed input. Lattice spacing is estimated solely from input-side Sr/Ti observations in the training region. Missing sites are completed using a lattice fitted to the input, while observation indicators are retained. Target annotations are used for supervision, evaluation, and target-coverage filtering, but do not supply the coordinates of Y₀. After spatial separation and valid-region filtering, no atomic identifiers are shared across splits. Supervised interpolation during training uses the target structure, consistent with Section 5.1 of the original manuscript; inference conditioning remains based solely on the input.