- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3(+1)
- Soundness
- 2
Read review
Paper summary
This paper presents an empirical study of where physics should be incorporated into molecular crystal generators. The authors introduce CrystAF, a few-step all-atom flow-map generator distilled from Clari-M, and use UMA-based physical feedback with negative-aware fine-tuning for post-training alignment. The paper systematically compares three approaches to incorporating physics: training-time learning, post-training alignment, and inference-time correction. It further investigates the contributions of different physical signals through ablation studies, and evaluates the methods in terms of structural validity, crystal packing quality, crystal structure recovery, and computational cost. The results suggest that post-training provides a favorable trade-off between structural quality and inference cost, while inference-time corrections are particularly effective at repairing local geometric defects and enforcing stereochemical constraints.
Critical strengths
-
Comprehensive empirical comparisons. The paper provides a useful framework for systematically investigating where physics should enter a molecular crystal generator. The experimental comparisons are comprehensive and insightful. In particular, Table 1 provides a broad comparison of physical quality and computational cost across different methods, while Table 3 thoughtfully evaluates whether improvements in structural quality translate into better crystal structure recovery. The inclusion of computational cost is especially valuable for practical applications.
-
Distinguishing physical improvement from metric optimization. The authors explicitly recognize that improving an evaluation metric does not necessarily imply improving physical plausibility. This distinction is important and provides useful insights for future model design and evaluation. The ablation studies also offer interesting observations about the different contributions of physical signals and update algorithms.
-
Interesting empirical findings. The paper reports several informative findings, including the effectiveness of simple feasibility filtering, the complementary roles of post-training and inference-time corrections, and the limitations of post-training in recovering stereochemical constraints. The authors also report cases where improvements in structural validity do not consistently translate into better crystal structure recovery.
Overall, this is an interesting exploratory and empirical work.
Critical weaknesses
-
Presentation and terminology could be improved. Some terms, such as "gentle," "strong," "matched," and "released" in Table 1, are not clearly introduced. The table also mixes model architectures, training configurations, and inference-time corrections, making the comparisons difficult to follow. A summary table or diagram clarifying these relationships would help.
-
Limited mechanistic understanding. The paper primarily demonstrates where physics should enter, but provides less insight into what physical properties are learned and how they evolve during generation. For example, Table 2 shows similar performance across different physical signals, but the underlying reasons remain unclear. Analyzing the evolution of energy, forces, stress, and atomic geometry during generation could provide more mechanistic insights. While such analysis is not essential to the paper's primary objective, it would strengthen the findings.
-
Unclear distinction between physical improvement and metric optimization. The term "physics" encompasses physical energetics, geometric validity, and agreement with experimental structures, which are not necessarily equivalent. Since negative-aware fine-tuning is itself a form of reward optimization, it remains unclear whether the model learns generalizable physical properties or merely optimizes physical proxy metrics. Table 2 suggests that feasibility filtering and volume supervision explain much of the improvement, while UMA diagnostics show increased energy, forces, and stress after post-training. Moreover, Table 3 uses UMA for candidate ranking, introducing some overlap between training and evaluation. Independent physical evaluations or out-of-distribution experiments would help substantiate the claims.
-
Limited improvement in downstream crystal structure recovery. Table 3 is valuable but shows that post-training does not consistently improve recovery performance. Further analysis of why improved structural validity does not necessarily translate into better recovery would strengthen the paper's practical implications.
-
Incomplete related work and insufficient positioning of novelty. Several relevant studies are missing from the discussion, including Physics vs Distributions (ICLR 2026), Physics-Constrained Fine-Tuning of Flow-Matching Models (ICLR 2026), Physics-Informed Distillation of Diffusion Models (ICML 2026), and Physics-Informed Diffusion Models (ICLR 2025). More closely related to molecular and crystal generation are Chemistry-Inspired Diffusion with Non-Differentiable Guidance (ICLR 2025), Reinforcement Learning with Formation Energy Feedback for Material Diffusion Models (2026), and Siamese Foundation Models for Crystal Structure Prediction (2026). More discussion and comparisons are needed.
Overall, this paper can have good potential impact, while the demonstrated impact is not enough, which can be improved by adding the above suggested analysis and make everything (i.e., the author's claims, supporting evidence, analysis) clear.
I am open to increasing my score if the authors adequately address the concerns raised above.
Questions for authors
See weakness.
6 comments
W2: mechanistic understanding of physical signals and training evolution
To address why different physical signals in Table 2 perform similarly and what physical properties are actually learned, we conducted two new mechanistic analyses examining within-family signal correlations and epoch-by-epoch training trajectories.
First, we rebuilt the exact training advantage using the UMA single-point scores of generated samples and correlated it within each molecular family against our evaluation metrics across 195 families on CrystAF-base samples (detailed in Appendix A.35 and Section 4.2).
| Reward Signal | ρ(A, −vol. error) | ρ(A, −PDD) | Partial ρ(A, −PDD) given volume | ρ(A, denser cell) |
|---|---|---|---|---|
| E, F, σ | −0.04 [−0.08, −0.00] | −0.01 [−0.05, +0.02] | +0.05 [+0.01, +0.09] | −0.10 [−0.13, −0.06] |
| E only | +0.01 [−0.03, +0.05] | +0.08 [+0.04, +0.12] | +0.15 [+0.11, +0.19] | −0.04 [−0.07, +0.00] |
On CrystAF-base samples, the UMA reward is uncorrelated with cell-volume error (Spearman ρ = −0.04). However, within any given family, cell-volume error and PDD are tightly coupled (ρ = +0.85), and a pure 1% volume error alone costs 1.6 PDD units on an otherwise exact experimental crystal. Even after controlling for volume error, the energy ranking still carries a consistent packing signal, with a partial correlation of +0.15 for the energy-only reward and +0.05 for the combined E, F, σ reward.
This strong coupling explains why the different arms in Table 2 look so similar on PDD. When volume effects are mathematically removed by isotropically rescaling each generated cell to the experimental volume while keeping molecules rigid (Appendix A.34), most of the post-training PDD gain disappears: the full-vs-base gain drops from −0.25 on raw PDD to −0.10 [−0.14, −0.05] on volume-normalized PDD. Moreover, the label-free E, F, σ reward and the experimental-volume label arm become statistically equivalent, differing by just −0.03 [−0.07, +0.02] (well within our pre-specified ±0.15 margin).
Thus, the primary mechanism behind the packing improvement is macroscopic cell-volume correction. CrystAF-base systematically generates cells +3.0% too large on average; the UMA-only arm shrinks this over-expansion to +1.2%, matching the +1.4% of direct experimental volume supervision across three seeds. Within a single family, the potential's ranking slightly favors larger cells (ρ = −0.10), so we report this global contraction as a robust empirical observation rather than claiming a simple within-group ranking mechanism (Discussion).
Because most crystal UMA energy is intramolecular, we also tested whether isolating intermolecular interactions via lattice energy (crystal minus isolated molecules) gives a sharper packing signal across 45 families (Table S22).
| Sample Source | E_int (Lattice) | E (Total) | E_int − E |
|---|---|---|---|
| CrystAF-base, 45 families | +0.151 [+0.055, +0.245] | +0.119 | +0.031 [−0.071, +0.136] |
| Eligibility only, epoch 4, 41 fam. | +0.197 | +0.102 | +0.095 [+0.023, +0.167] |
| E, F, σ, epoch 4, 44 fam. | +0.172 | +0.161 | +0.011 [−0.090, +0.110] |
While lattice energy tracks volume better than total energy (ρ = +0.11 vs. +0.01 with volume error, +0.16 vs. +0.04 with PDD), its partial correlation with volume-normalized PDD on base samples is +0.151 vs. +0.119 for total energy (difference +0.031 [−0.071, +0.136]). Failing our pre-specified go-rule (≥ 0.20 with CI above total energy), we did not train a lattice-energy policy.
Second, we tracked how metrics evolve across epochs 0–6 for the full signal on 200 families (Appendix A.37, Table S24).
| Epoch | PB (%) | Clash (%) | Vol.Err | PDD | Eligible | ΔE (eV/mol) | Med. F_max | Med. ‖σ‖ |
|---|---|---|---|---|---|---|---|---|
| 0 (base) | 89.99 | 11.13 | 1.96 | 10.34 | 0.853 | +0.00 | 8.59 | 0.058 |
| 1 | 90.50 | 10.32 | 1.79 | 10.15 | 0.846 | −0.14 | 8.62 | 0.060 |
| 2 | 91.56 | 10.91 | 1.82 | 10.13 | 0.843 | +0.24 | 9.34 | 0.068 |
| 3 | 92.20 | 9.99 | 1.76 | 10.14 | 0.841 | +0.61 | 10.53 | 0.075 |
| 4 | 92.02 | 9.64 | 1.69 | 10.02 | 0.819 | +0.91 | 11.07 | 0.079 |
| 6 | 89.41 | 10.60 | 1.57 | 9.94 | 0.717 | +3.21 | 12.21 | 0.128 |
Most volume and PDD gains occur in epoch 1 (Vol.Err 1.96 to 1.79, PDD 10.34 to 10.15) with UMA energy flat (−0.14 eV/mol), whereas PB peaks at epochs 3–4. Upward drift in single-point energy, force, and stress starts at epoch 2 and accelerates after epoch 4 (+3.21 eV/mol at epoch 6), where the eligible fraction drops to 0.72 and PB degrades even as volume and PDD tighten. This confirms that packing gains are not a by-product of late energy drift and justifies reading out all arms at epoch 4. We omitted generation-trajectory analyses because intermediate flow-map predictions fall outside the 16-step adjacent-jump training distribution.
W3: disentangling physical preferences, constraints, and metric optimization
We agree that "physics" was used too loosely in the original draft. In the revision (Section 3.2 and Introduction), we explicitly separate three signals: (i) potential-based preferences, represented by within-family rankings of UMA energy, forces, and stress; (ii) feasibility constraints, consisting of minimum distance, density bounds, and finite-energy checks; and (iii) direct supervision from experimental labels (the optional cell-volume term). Table S3 maps these against our metrics, and the Discussion clarifies that the full reward's volume gain partly reflects direct label supervision, whereas UMA-only arms use zero experimental labels.
Since the feasibility gate (ii) alone reproduces the PoseBusters validity gain, does the potential preference (i) contribute anything on its own? To test this directly, we trained a new E,F,σ arm with the geometric gate completely switched off, ranking every scoreable candidate across three independent training seeds (Appendix A.36, Table S18, Table 2).
| Arm (Mean ± SD, 3 Seeds) | Gate | UMA | Vol. | PB ↑ | Clash ↓ | Vol.Err ↓ | PDD ↓ | Norm. PDD ↓ | Signed ΔV/V |
|---|---|---|---|---|---|---|---|---|---|
| CrystAF-base | 89.02 | 10.35 | 1.91 | 10.24 | 9.19 | +3.03% | |||
| Eligibility only | ✓ | 91.98 ± 0.49 | 10.31 ± 0.25 | 1.75 ± 0.06 | 10.05 ± 0.04 | 9.11 ± 0.03 | +2.42 ± 0.23% | ||
| E, F, σ, no eligibility | ✓ | 91.97 ± 0.36 | 9.93 ± 0.52 | 1.76 ± 0.06 | 10.07 ± 0.07 | 9.09 ± 0.01 | +2.25 ± 0.09% | ||
| E, F, σ + eligibility | ✓ | ✓ | 90.91 ± 0.49 | 10.06 ± 0.14 | 1.58 ± 0.06 | 9.94 ± 0.06 | 9.04 ± 0.01 | +1.24 ± 0.30% | |
| Exp.-volume label | ✓ | ✓ | 91.30 ± 0.33 | 9.94 ± 0.36 | 1.61 ± 0.02 | 9.97 ± 0.01 | 9.07 ± 0.02 | +1.37 ± 0.15% | |
| Full (CrystAF-UMA, 4 ep.) | ✓ | ✓ | ✓ | 91.41 ± 0.13 | 9.65 ± 0.48 | 1.64 ± 0.03 | 10.00 ± 0.01 | 9.08 ± 0.01 | +1.93 ± 0.16% |
Remarkably, without any gate, UMA ranking alone achieves 91.97 ± 0.36% PB, matching the gate-only arm (91.98 ± 0.49%, seed differences −0.13, +0.15, −0.06) and beating the 89.02% base across all seeds (CIs > +0.5). Thus, the potential carries a genuine validity signal of its own, ranking gate-rejected base samples below accepted ones with 0.92 probability by energy and 0.97 by max force. However, the gate and ranking do not stack linearly. Without the gate, ranking expends its capacity filtering broken geometries, leaving cell volume at 1.76 Vol.Err (+2.25% expansion). When the gate removes broken structures upfront, the same UMA ranking operates among intact candidates, trading ~1 PB point to drive Vol.Err down to 1.58 (+1.24% expansion). Validity comes from discarding broken samples (via gate or ranking), while volume correction comes from ranking samples already cleaned by the gate.
To verify geometric quality using an independent diagnostic outside the reward, we measured MMFF94 heavy-atom intramolecular strain after relaxing hydrogens across 37 scoreable validation families (Appendix A.38, Table S25). Post-training cuts the tail of strongly distorted molecules (> 3 kcal/mol per heavy atom) nearly in half, from 17.2% in CrystAF-base to 9.3–9.7% across all post-trained arms (vs. 8.1% in experimental crystals), while mean strain falls from 2.97 to 1.85 kcal/mol for the full signal (paired difference −1.12 [−2.15, −0.37]). Because the gate-only and volume-label arms achieve this same reduction without UMA ranking, this independent check confirms a real geometric improvement driven by discarding broken samples.
To resolve why single-point UMA energy rises while cell volumes move toward experiment, we ran full-cell UMA relaxations on the 200 experimental validation crystals (Appendix A.39, Section 4.2). Across 177 converged crystals, median volume changes by just +0.12% (92% within ±1%), proving UMA's equilibrium cell sits at the experimental volume. We removed our earlier claim that UMA prefers larger cells (based on 36 clashing generated structures). The generator's +3.03% expansion is a model bias corrected by UMA, while energy rise reflects unconstrained drift under group-relative ranking (Appendix A.32 notes the lack of an independent periodic organic crystal potential).
Finally, Table 3 conclusions do not hinge on UMA ranking overlap. Unranked recovery (n_s = k = 30, Table S17) leaves CrystAF-UMA and CrystAF-base tied on the 779 teaching targets (0.427 vs. 0.423). Re-ranking the top 200 candidates per target from the 400-sample pool by UMA lattice energy instead of single-point energy (Appendix A.40, Table S27) changes collision-enforced Sol@30 by +0.013 [−0.013, +0.039] for base (0.471 vs. 0.458) and −0.005 [−0.031, +0.022] for CrystAF-UMA (0.447 vs. 0.452), keeping comparisons invariant. We also now describe test sets as "held-out molecular identities" rather than OOD (Appendix A.1, Section 4.4).
W1: clarifying terminology and restructuring Table 1
We sincerely thank the reviewer for such a thorough reading and for providing a clear, actionable list of suggestions. We have organized our response into five posts corresponding directly to W1 through W5. All new experiments reported below follow a strict evaluation protocol that was fixed before any results were observed.
You rightly pointed out that shorthand terms like "gentle", "strong", "matched", and "released" were opaque, and that Table 1 mixed model architectures, training configurations, and inference-time corrections in a way that made comparisons difficult to follow. We have completely removed these ambiguous labels from Table 1, Figure 3, and the main text. Every row now explicitly states what it is, the Table 1 caption provides the exact guidance hyperparameters, and the protocol paragraph at the beginning of Section 4 defines how the two CrystAF-UMA checkpoints were trained.
To resolve the structural confusion in Table 1, we reorganized the rows based on what actually changes in the pipeline. The first group contains interventions that modify the model weights, covering training-time distillation and post-training alignment. The second group contains interventions that alter only the sampling process via inference-time corrections, separated into distinct blocks according to the underlying weights they are applied to (CrystAF-base versus CrystAF-UMA). Finally, we added a dedicated route summary table in Appendix Table S4, referenced at the start of Section 4, which explicitly maps out each route by whether weights are changed, what physical input is required, whether experimental labels are used, what happens at inference time, and when the computational cost is paid.