acceptodds
Under review as a conference paper at ICLR 2027

Zatom-2: Multitask Pretraining on Atomistic Data for Generative Modeling across Domains

Abstract

Unified atomistic modeling has the potential to accelerate discovery in chemistry, materials science, and biology by bridging data-rich chemical domains and data-scarce biological contexts. However, existing generative approaches to atomistic modeling remain highly specialized to scientific disciplines (chemistry vs. biology) or do not leverage both high-volume organic (molecule) and inorganic (material) data for general-purpose pretraining. To this end, we introduce Zatom-2, an atomistic generative model pretrained on approximately five million structures from the OMol25 and OMat24 electronic structure datasets. Zatom-2 features a multiscale Transformer architecture coupled with conditional flow matching that supports force conditioning and foundational pretraining tasks such as generation, structure prediction, and prediction of molecular and material energies and forces. Empirically, Zatom-2 achieves better molecular distribution fidelity than Zatom-1 and achieves strong performance on existing molecule and material generation benchmarks. Zatom-2 demonstrates the ability to control sample generation across low- and high-force regimes, and enhances protein generation in a low-data setting through joint generative-predictive pretraining and transfer learning, increasing protein backbone designability in a length extrapolation setting from 66.9% without pretraining to 75.1% after finetuning on 2,000 protein domains. Code and model weights will be made fully open source.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Generative model, atomistic training, molecule generation, structure prediction, biology, chemistry, flow matching

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_v8PoOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a unified framework Zatom-2 for atomic-level generative modeling across molecules, periodic materials, and proteins via a multitask transformer backbone and mixed training tasks. It combines a generative objective, a structure prediction objective, and a property prediction objective into one loss during training, achieving good performance in a variety of generation tasks.

Critical strengths
  1. This paper introduces a unified pretraining framework that can do multiple atomic tasks, covering generation, structure prediction, and force/energy predictions.
  2. It shows that the ability model obtains through force/energy prediction in pretraining can improve molecule quality and protein generation designability, showing that some physics-based objectives can help downstream molecular generation.
Critical weaknesses
  1. This work has limited methodological novelty. Its main architecture and training paradigms are all from existing works. It feels like what this work does is combining the generation task, structure prediction task, and property prediction task together into one objective without some new insights into the optimization mechanism.
  2. The experiments and ablation studies do not sufficiently demonstrate how each pretraining task contributes to downstream performance improvements. For example, it remains unclear how incorporating energy and force prediction during pretraining affects the conformational properties and physical quality of generated molecules. Moreover, the protein generation experiments are limited to unconditional monomer generation, a relatively well-studied task for which many existing models already achieve strong performance. The designability metrics alone provide limited insight into how pretraining affects the structural and physical properties of generated proteins. Finally, the scaling experiments primarily confirm the well-established observation that increasing dataset size and model capacity improves performance, offering limited additional insights into the effectiveness of the proposed pretraining framework.
Questions for authors
  1. The experiment that benchmarks the pretrained model's performance in protein generation is interesting. I am wondering if the authors have done some energy analysis on the generated proteins to see whether the incorporation of energy and force prediction helps the model generate more low-energy-state proteins. Also, have the authors tried more challenging protein binder design tasks?
  2. It would be helpful to include more ablations to better understand the contribution of each pretraining objective to downstream performance. For example, how does removing the structure prediction objective affect downstream protein generation? Such experiments would help clarify whether all the proposed pretraining tasks are necessary and how much each contributes to the observed improvements.
3 comments
AuthorsOpenReview ↗

Author Rebuttal for Reviewer v8Po

Author Rebuttal

We thank Reviewer v8Po for their constructive review and for noting Zatom-2's unified multi-task design and physics-based pretraining benefits. Below we address W1–W2 and Q1–Q2.

1. Methodological Design and Mechanistic Insights (W1)

Unifying small molecules, periodic materials, and all-atom proteins in one generative model required three design choices beyond combining standard losses. First, discrete atom type or lattice diffusion processes do not extend naturally to variable-length all-atom protein generation. Zatom-2 instead transports molecules (atom1), materials (atom1), and proteins (atom14) through one shared Cartesian velocity field while recovering atom types, amino acid types, fractional coordinates, and unit cell geometry via endpoint-gated readouts (g1 A˚(t)g _ {1\text{ Å}}(t), Equation 5). In short, Zatom-2 introduces atom1 to unify chemical and biological generative modeling. Second, naively combining tasks on non-equilibrium DFT data can cause performance interference: pretraining on OMol25 and OMat24 without force conditioning degrades OOD protein designability below training from scratch (63.0% vs. 67.8%, Table 3). Pairing scalar force conditioning (sFs _ F) with anti-leakage task masks (mE,mF)(m _ E, m _ F) and latent equivariance (Leq\mathcal{L} _ {\mathrm{eq}}) resolves this conflict and unlocks positive transfer (74.8%), while our extended materials training experiment (Appendix E.1, Table 9) shows how endpoint loss weighting governs Cartesian-to-lattice alignment for materials under inference-time guidance.

Third, representation probing suggests why physical supervision aids generation tasks. On 768 OMol25 validation molecules (Appendix F.4, Figure 17), energy/force supervision lifts cross-structure 10-NN chemical-subset purity by 12.6–14.9 points across the elements H, C, N, and O. On 128 unseen SCOPe domains (Section 4.5, Figure 4), protein finetuning from Zatom-2 (V) raises cross-protein 10-NN amino acid type purity in Zatom-2's latent space from 84.8% to 88.1% and Calinski-Harabasz separation by 64% (374.9 vs. 228.1) over SCOPe-only training.

2. Protein Energetics, Molecular Quality, and Ligand-Binder Design (W2, Q1)

Addressing Q1a, we evaluated all of Zatom-2's unrelaxed all-atom proteins and scored 4,800 protonated structures with UMA-S-1p2 (Appendix E.3, Table 13). Energy/force pretraining in Zatom-2 (V) yields the lowest bond/angle RMSZ among Zatom-2 variants, cuts steric clashes 3.6× ID and 2.8× OOD over training from scratch (83.2% ID and 48.9% OOD clash-free), lowers heavy-atom forces by 33%–38% below Zatom-2 (IV) (2.73/2.62 vs. 4.10/4.24 eV/Å), and matches native SCOPe-2k binding energies (−9.04 ID and −9.00 OOD vs. −9.03 eV/heavy atom native, vs. −8.77 / −8.84 when training from scratch), while lifting native sequence designability 4.9× over Zatom-2 (IV) (Table 14).

On OMol25 molecules (W2; Tables 1, 9, 21), adding energy/force supervision (Zatom-2 (IV) to (VI)) improves RDKit validity (83.6% to 85.1%), force AUROC (0.713 to 0.817), and conformer mean forces (2.14 to 1.21 eV/Å), restoring 1-fragment median connectivity, raising PoseBusters pass rates from 30.3% to 34.8%, and cutting mean max atomic forces from 7.31 to 4.12 eV/Å.

As suggested, in a new, challenging ligand-conditioned protein binder design task (Q1b; Appendices B.6, E.9, F.3, Table 24, Figure 14), we finetuned the Base-sized Zatom-2 architecture on SPINDR (23,955 complexes) to diffuse atom14 binders around fixed atom1 ligands across 1,000 test complexes (137 ligands, three seeds): Zatom-2 (V) beats training from scratch in <1.5 Å designability (4.37% to 6.90%, 1.58×), cluster diversity (27.7 to 48.7), novelty (13.7% to 24.7%), PoseBusters clash-free rates (11.1% to 14.6%), and UMA-S-1p2 pocket intermolecular forces (2.23 to 1.33 eV/Å, −40%).

3. Stepwise Task Ablations and Multi-Domain Scaling (W2, Q2)

Table 11 of Appendix E.3 now isolates each pretraining task across three seeds. On OMol25 alone (70M, no materials), training from scratch reaches 53.8% OOD designability (66.6% TMgeo\mathrm{TM} _ {\mathrm{geo}} novelty), generation alone (i.e., Zatom-2 (I)) lifts novelty to 73.6%, and force conditioning without structure prediction (II) reaches 55.3% designability and 80.9% novelty.

Answering Q2 directly, adding structure prediction (III) boosts OOD protein designability by +11.2 points to 66.5%, per-sequence success by +7.7 points to 37.8%, and OOD clusters from 473.0 to 601.0, while raising OMol25 validity (82.3% to 84.6%) and force AUROC (0.662 to 0.730). With 140M parameters, force conditioning remains essential (63.0% without it vs. 68.0% in IV), and energy/force supervision (V) reaches 74.8 ± 1.1% OOD designability. Scaling to 270M (Table 9) also relieves cross-domain contention, lifting OMat24 match rate from 0.294 to 0.460.

Thank you for the substantial additional experiments. The task ablations, physical-quality analyses, and ligand-conditioned generation results address some of my concerns about the contribution and scope of pretraining. While I retain some reservations about methodological novelty, the new evidence strengthens the empirical contribution. Please distinguish improvements in predicted physical quality from thermodynamic stability and clarify the remaining limitations of sequence–structure co-design. I update my score to 3.

AuthorsOpenReview ↗

Author Follow-up: Clarification and Limitations

Thank you for reviewing our additional experiments and for raising your score. In the revised manuscript (Section 4.4, Appendices E.3 and E.9, and Appendix G.2), we have made both distinctions explicit:

  1. Predicted physical quality vs. thermodynamic stability: PoseBusters checks, single-point gas-phase UMA-S-1p2 energies and atomic/pocket force norms, and unrelaxed 0 K MLIP EhullE _ {\mathrm{hull}} estimates (Tables 13, 17, 21–24) measure predicted local geometric plausibility and reduced steric/electronic strain on unrelaxed structures, rather than verified local minima or true thermodynamic folding, binding, or finite-temperature phase stability (ΔG), which require conformational entropy and solvation modeling, full relaxation, and DFT or experimental validation.

  2. Sequence–structure co-design limitations: Because Zatom-2's endpoint-gated sequence readout (λseq=0.1\lambda _ {\mathrm{seq}} = 0.1 vs. λFM=4.0\lambda _ {\mathrm{FM}} = 4.0, active only near t≈1t \approx 1 via g1 A˚(t)g _ {1\text{ Å}}(t) without iterative sequence–rotamer packing refinement) yields low standalone single-sequence refolding rates (≤3.80% ID and 0.00% OOD at <1.5 Å; Table 14), Zatom-2 currently functions primarily as an all-atom coordinate generator paired with post-hoc SolubleMPNN or LigandMPNN inverse folding (Tables 3, 11, 24), leaving self-consistent sequence–structure–side-chain co-design for future work.

Thank you again for your time and feedback. We would be happy to address any other feedback you might have about the manuscript.

Reviewer_Mxu5OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

In this paper, the authors introduces, Zatom-2, an atomistic generative model and pretraining framework for bridging small molecule, periodic crystal, and protein. An atom1+atom14 tokenization scheme and a multiscale Transformer architecture are designed to model atomistic data. A force conditioning mechanism are proposed to enable general control over generated samples' relaxation state. The Zatom2 is trained on a massive, diverse corpus comprising 5 million structures from OMol25 and OMat24. Experimental results on several tasks, including molecule, matrial, and protein genenration, demonstrate the effectiveness of it. The architecture also exhibits performance advantages with data and parameter scaling.

Critical strengths
  1. The paper explores the unified atomistic modeling task, which is an important and challenging direction in scientific intelligence.
  2. Several key techiniques, including tokenization scheme, multiscale Transformer architecture, and the force conditioning mechanism, are introduced for atomistic modeling. The model is trained on a massive dataset, and the experimental results demonstrate its effectiveness.
  3. The paper reports comprehensive experimental study and analysis to evaluate the contribution of multi-task mixtures, model capacity, and data size.
Critical weaknesses
  1. Using a single global scalar to represent a complex system's stress state may lead to physical information noise, since the system may be locally under a high-stress state while its overall average force remains low. Tab.10 shows that the auxiliary MLIP task accuracy lags far behind specialized potentials. Therefore, interagting a non-conservative, low-accuracy potential into multi-task pretraining may introduce negative transfer. As shown in Appendix E.2, while MLIP fine-tuning improves property prediction, it temporarily degrades materials generation and structure prediction metrics unless extended fine-tuning is applied. A deeper discussion on why materials generation metrics are more fragile to task-mixture shifts compared to molecular domains would strengthen the analysis.
  2. The protein transfer learning experiments lack comparisons with state-of-the-art dedicated protein generative models. Moreover, Figure 7 exhibits a overfitting trend (where training curves show performance drops after slightly longer training steps), indicating that pretraining may has not fully eradicated the stubborn issue of overfitting during small-dataset fine-tuning.
Questions for authors

Please refer to the weaknesses.

3 comments
AuthorsOpenReview ↗

Author Rebuttal for Reviewer Mxu5

Author Rebuttal

We thank Reviewer Mxu5 for their positive assessment of Zatom-2's unified atomistic framework, multiscale atom1 + atom14 tokenization, force conditioning, and broad evaluation. Below we address both concerns.

1. Global Force Conditioning, Auxiliary MLIP Supervision, and Crystal Fragility (W1)

We agree that a global scalar sF=log⁡10(1L∑ℓ∥Fℓ∥2)s _ F = \log _ {10}(\frac{1}{L}\sum _ \ell \lVert \mathbf{F} _ \ell \rVert _ 2) cannot uniquely specify every local stress pocket, and we note per-atom vector conditioning as future work in Appendix G.2. Even so, Zatom-2 suppresses and controls local strain in two ways. First, during pretraining (Zatom-2 (V) and (VI)), the force loss (LF\mathcal{L} _ F, Equation 5) supervises 3D per-atom forces F^ℓ∈R3\widehat{\mathbf{F}} _ \ell \in \mathbb{R}^3 on the decoder states QLQ _ L driving coordinates, potentially cutting per-molecule maximum force spikes on OMol25 from 7.31 to 4.12 eV/Å (Table 21) and nearly tripling generated SPINDR protein-ligand interfaces free of >10 eV/Å spikes (8.8% to 24.1%; Table 24). Second, sweeping sFs _ F across five targets on 50 fixed OMol25 graphs yields 100% positive within-graph Spearman correlation (Table 22). Moreover, during single-step zero-shot coordinate refinement with a fixed graph or cell (Appendix E.8, Table 23; 0.075 s on one GPU), conditioning frozen Zatom-2 (VI) on sF=0.05s _ F = 0.05 eV/Å lowers UMA forces and energies on 49/50 to 50/50 rattled molecules (9.66 to 2.86 eV/Å median at σin=0.30\sigma _ {\mathrm{in}} = 0.30 Å) and 47/50 to 50/50 rattled crystals (5.24 to 2.42 eV/Å, ΔE=−0.391\Delta E = -0.391 eV/atom), achieving 3%–16% lower forces than unconditioned denoising on identical inputs.

Regarding MLIP accuracy, Zatom-2 is primarily a generative model rather than a conservative MD potential (Appendix E.2). Even below specialized potentials, energy/force supervision acts as a strong physical regularizer: adding it (IV to V) lifts 10-NN subset purity of the embeddings of the elements H, C, N, and O by 12.6–14.9 points (Appendix F.4, Figure 17), raises unseen-protein 10-NN amino acid type purity in latent space from 84.8% to 88.1% and Calinski-Harabasz separation by 64% (Section 4.5, Figure 4), cuts unrelaxed protein forces by 33%–38% (Table 13), and boosts OOD protein designability (68.0% to 74.8%, Table 3) and SPINDR ligand-conditioned protein binder designability (4.4% to 6.9%, Table 24).

Why are periodic materials in Tables 9–10 more sensitive to task-mixture shifts than molecules? Appendix E.1 traces this to readout asymmetry. Molecules and proteins decode directly in Cartesian space via the velocity field (λFM=4.0\lambda _ {\mathrm{FM}} = 4.0, active across t∈[0,1]t \in [0, 1]), whereas crystals decode at the final step from fractional/lattice heads gated only near t≈1t \approx 1 (g1 A˚(t)g _ {1\text{ Å}}(t), ≈6% of steps) whose weights were cut 20–40× (10/1/1 to 0.25/0.05/0.05) to balance generative and predictive gradients. In Table 9, force-only guidance (scale 2) steers the Cartesian path away from these lightly weighted heads, raising guided OMat24 target median force error from 1.70 to 30.55 eV/Å. Re-evaluating all 500 targets without guidance eliminates this shift: Zatom-2 (V) improves unguided median target force error over IV (1.21 to 1.09 eV/Å) and cuts ≥17-atom mean error 3.6× (24.59 to 6.91 eV/Å), leaving only a mean spike from 26/500 (5.2%) 2–4 atom cells with <1.0 Å clashes. Pretraining Zatom-2 (V) for +10 epochs with restored 10/1/1 weights—keeping MLIP losses active—lifts guided OMat24 match rate from 0.294 to 0.462 and cuts guided target median force error 10.4× to 2.95 eV/Å without hurting molecule tasks (Table 9).

2. Dedicated SOTA Protein Baseline and Small-Dataset Convergence (W2)

Tables 3 and 11–15 benchmark Zatom-2 against the recent, size-matched SOTA protein generation baseline RFdiffusion3 (Butcher et al., Science 2026; ≈170M) on the same SCOPe-2k split across three seeds. RFdiffusion3 holds a slight in-distribution (ID) edge on 50–128 residue samples (91.1% vs. 89.1%), whereas Zatom-2 (V) yields higher ID diversity (453.7 vs. 435.0), novelty (16.3% vs. 12.4%), and clash-free rates (83.2% vs. 63.8%, Table 13), and outperforms RFdiffusion3 in OOD extrapolation (129–256 residues) by +20.8 points in designability (74.8% vs. 54.0%; 71.5% vs. 37.5% at 193–256 residues), +17.4 points in per-sequence success (45.8% vs. 28.4%), and +163.4 in cluster diversity (618.7 vs. 455.3). Appendix E.3 (Figure 8) shows that over 50k steps, pretraining mitigates rather than eliminates memorization: sample novelty when training from scratch rapidly drops after 25k–30k steps, while pretrained Zatom-2 (V) reaches its peak designability faster and uniquely sustains high designability and novelty through its 50k finetuning steps. To complement these results, we now provide SPINDR binder design transfer results (Table 24).

The reviewer appreciates the authors' detailed responses and the additional experimental results. The rebuttal has addressed most of the concerns. The reviewer therefore maintains the Weak Accept recommendation.

AuthorsOpenReview ↗

Author Follow-Up: Closing the Loop

Thank you for your constructive review, engagement with our rebuttal, and supportive recommendation. To close the loop between W1 (crystal fragility vs. negative transfer) and W2 (protein generation vs. RFdiffusion3), we wanted to briefly share one final experiment completed right after our initial response (updated Section 4.4 and Appendix E.3, Tables 11–12):

Finetuning our 10/1/1-restored Zatom-2 (V) checkpoint—which resolved the guided OMat24 regression (0.294 to 0.462 match rate; 30.55 to 2.95 eV/Å target median force AE; Table 9) while retaining λE/λF/λeq=0.1/0.3/0.1\lambda _ E / \lambda _ F / \lambda _ {\mathrm{eq}} = 0.1/0.3/0.1—on SCOPe-2k across three seeds confirms that restoring 20–40× stronger crystal supervision introduces no negative transfer to proteins:

  1. OOD extrapolation (129–256 residues): Designability rises from 74.8 ± 1.1% (80-ep. Zatom-2 (V)) to 80.7 ± 1.4% (+26.7 points over RFdiffusion3's 54.0 ± 1.8%; 82.6 ± 1.4% vs. 37.5 ± 1.2% at 193–256 residues; 49.7 ± 1.5% at <1.0 Å), with 51.2 ± 1.1% per-seq success, 680.7 ± 12.9 clusters (426.3 ± 1.7 at M=500M = 500), and 80.2 ± 1.4% TMgeo\mathrm{TM} _ {\mathrm{geo}} novelty.
  2. In-distribution (50–128 residues): Designability holds at 89.3 ± 0.4% with higher diversity (477.7 ± 12.5 vs. 435.0 for RFdiffusion3) and novelty (18.2 ± 1.5% vs. 12.4% for RFdiffusion3).

Thus, balancing periodic readout weights eliminates crystal fragility without cross-domain trade-offs in a single 140M parameter model, while our revised manuscript (Appendices E.3, E.9, and G.2) also clarifies the scope of scalar force conditioning and auxiliary MLIP supervision relative to specialized potentials and per-atom vector control.

Thank you again for your time and consideration of this work. If you have any other feedback about the manuscript, we would be happy to address it.

Reviewer_9HmrOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

Zatom-2 combines an atom–token–atom Transformer with coordinate flow matching, using atom-level tokens for molecules and materials and atom14 residue tokens for proteins. Element identities, residue identities, and material lattice information are predicted from the shared representations. Pretraining uses approximately four million OMol25 and one million OMat24 structures, with generation, structure prediction, and auxiliary energy/force prediction tasks. A scalar force condition controls the requested mean atomic force norm. The model is evaluated on broad atomistic generation, standard molecular and material benchmarks, and transfer to protein backbone generation after fine-tuning on 2,000 SCOPe domains.

The most interesting result is that joint generative-predictive pretraining improves protein length-extrapolation designability from 66.89% to 75.10%, while generation/structure pretraining alone does not. This suggests a useful interaction between physical supervision and generative transfer. However, the causal explanation, the novelty of the generated proteins, and the extent of positive transfer across all tasks are less settled than the main text implies.

Critical strengths
  1. The research question is important. Establishing when abundant atomistic data can help data-limited biological generation is a substantive problem. A reusable model across chemical domains could have practical value even without a fundamentally new attention mechanism.
  2. The implementation is broad and technically coherent. A common backbone supports several input representations and readouts. The paper gives detailed forward-pass pseudocode, task mixtures, normalization, sampling settings, compute estimates, and comparisons with related architectures.
  3. The transfer result goes beyond a single headline metric. In the extrapolation setting, the best initialization improves both best-of-eight designability and per-sequence refolding success, the latter from 34.91% to 46.50%. This reduces concern that the gain comes entirely from the best-of-eight selection rule.
  4. There are useful negative results and qualifications. The appendix acknowledges weak MLIP accuracy, lack of energy–force consistency, imperfect force matching, and trade-offs in conformer coverage. The paper also supplies a UMA validation-set calibration and finite-sample AFD reference values.
  5. The architecture is competitive on established benchmarks. The unrelaxed MP20 MetaSUN yield of 4.92% is encouraging, and molecular generation performance is competitive. These results help establish that the broad model is not only a conceptual unification.
Critical weaknesses

1. Protein transfer is promising, but the source of the improvement is not yet isolated

The matched-size comparison between pretrained variants IV and V is valuable. Nevertheless, adding the MLIP task also changes the task mixture, introduces latent equivariance regularization, and changes fractional-coordinate and lattice loss weights substantially (Appendix C.2). For example, the fractional-coordinate weight changes from 10 to 0.25. The experiment therefore tests a training package, rather than isolating force and energy labels as the cause.

There is also no same-capacity, matched-budget comparison of OMol-only pretraining against OMol+OMat pretraining for protein transfer. Adding OMat in Table 1 simultaneously changes model size. The current results establish that the joint corpus and recipe can provide a useful initialization; they do not establish that inorganic material data contributes to the biological gain.

The data boundary matters here. OMol25 explicitly includes biomolecules and MD configurations of cropped protein pocket–ligand complexes (Appendix B.1). Please report overlap or homology between the parent protein structures in pretraining and SCOPe, and clarify which biological structures are present. A pretraining ablation excluding biomolecular subsets would help distinguish transfer from general small-system physics from transfer supported by already related biological environments. This is not an allegation of leakage: broad biological pretraining can be legitimate, but the source of the information changes the interpretation.

Finally, SCOPe-2k is one fixed, fold-balanced sample of 2,000 domains from 488 fold groups. That is a useful setting, but not enough to characterize data efficiency generally. Independent subset seeds and a small 1k/2k/4k learning curve would be more informative than additional sample galleries. At minimum, please repeat the central transfer comparison across training seeds and report uncertainty by length bin.

2. The protein novelty metric is strongly affected by the length-extrapolation setup

Appendix D.9 defines novelty by the maximum TM-align score against SCOPe-2k, normalized by the generated query length, with a novelty threshold of 0.5. Training structures are at most 128 residues, while extrapolated queries reach 256.

This creates a mathematical length effect. If a query has length LqL_q and a reference has length LrL_r, the standard query-normalized TM-score is bounded by Lr/LqL_r/L_q, because there can be at most LrL_r aligned residue pairs and each contributes at most one. For a 256-residue query compared with a 128-residue reference, the score cannot exceed 0.5 even under perfect alignment of the entire shorter reference. Almost any mismatch then makes it “novel.” A longer protein can therefore contain a familiar training domain and still receive a low query-normalized score.

The increase from roughly 10–17% novelty in-distribution to 89–96% out-of-distribution cannot be interpreted as evidence of new folds without correcting for this effect. Please compare against a length-matched structural database, report both normalization directions and alignment coverage, and include a domain- or fragment-level nearest-neighbor analysis. Novelty relative only to 2,000 short training structures is also narrower than novelty relative to known protein structures.

This concern does not erase the designability gain, which uses a different metric. It does weaken a major part of the diversity/generalization interpretation. Similarly, the number of designable clusters should be compared after subsampling equal numbers of designable structures; otherwise, a method that simply produces more designable samples has more opportunities to form clusters.

3. The protein evaluation establishes backbone compatibility, not the full all-atom generation claim

The model produces an atom14 representation and includes a sequence head, but Table 3 redesigns eight sequences with SolubleMPNN and evaluates Cα agreement after ESMFold2 refolding. This is a sensible backbone-generation benchmark. It does not directly test whether the model's own sequence, side-chain geometry, and backbone are mutually compatible.

Please evaluate the generated sequence without inverse-folding replacement and report side-chain stereochemistry, clashes, bond geometry, and sequence–structure consistency. A second folding evaluator on a representative subset would help assess evaluator dependence. The current best-of-eight protocol and absence of a pLDDT gate are clearly documented; sensitivity to refolding confidence and the 1.5 Å threshold would still be useful. These computational tests need not be replaced by experimental validation for this methodological contribution.

Checkpoint selection also deserves a precise account. Appendix E.3 evaluates 96 backbones per checkpoint and selects a designability/novelty trade-off. Please specify a deterministic selection rule, the length distribution and random seeds used in the sweep, and whether the final evaluation uses independent samples. The plotted trajectories are useful evidence against a single arbitrary checkpoint, but they do not measure variation across training runs.

The appendix clearly accounts for the 100k pretraining step offset: the final models use approximately 20–30k local fine-tuning steps, versus 25k for Zatom-2 from scratch. This makes the local optimization budgets reasonably comparable; the remaining question is the stability of checkpoint selection and training across runs.

4. The material results reveal substantial negative transfer that the main narrative understates

The most consequential counterexample appears in Table 9. Adding MLIP supervision from variant IV to V changes OMat24 structure prediction as follows:

MetricIV: generation + structureV: + MLIPVI: larger + MLIP
Structure match rate44.6%29.4%46.0%
Median mean force norm, eV/Å2.76532.45818.867
Target median absolute force-norm error, eV/Å1.70330.55317.380

These are large physical-quality regressions, not small fluctuations. Because the reported statistics are medians, rare extreme outliers alone cannot explain them. Scaling restores the match rate but leaves force consistency much worse than the generation/structure-only model. Table 10 likewise reports high OMat24 force norms after MLIP-heavy fine-tuning.

Please discuss this result in the main text and determine whether it arises from loss reweighting, the shared representation, periodic geometry reconstruction, or another aspect of the implementation. A material can pass a tolerant, volume-rescaled StructureMatcher test while still having unfavorable local distances; the disagreement between geometric matching and force evaluation is itself informative.

The separate MP20 generation benchmark also shows a trade-off. Table 12 reports 4.92% MetaSUN, but only 70.24% overall validity and 53.44% uniqueness. The corresponding Zatom-1-L values are 95.0% and 90.4%. A higher yield on the selected metastability metric is meaningful, but it is not an across-the-board improvement.

Moreover, the stated MetaSUN criterion excludes Ehull≤0E_{\mathrm{hull}}\leq0. Please report stable, metastable, and combined stable-or-metastable unique novel yields, with uncertainty from the MLIP ensemble. Otherwise, a metric that excludes stable predictions cannot alone support a broad ranking of physical quality. A stratified DFT check of some generated materials would strengthen the scientific interpretation, particularly near the 0.1 eV/atom threshold.

5. The force metric supports limited controllability, not a complete assessment of physical realism

Mean force norm is a reasonable descriptor of distortion. The validation-set agreement between UMA and the DFT metadata, 0.00587 eV/Å for OMol25 and 0.02462 eV/Å for OMat24, is a useful calibration. However, it evaluates the proxy on reference structures, not on arbitrary generated geometries where extrapolation failures may be more severe.

A high AUROC at a coarse threshold can coexist with poor calibration within each regime, as the authors acknowledge. Matching one scalar also does not establish a realistic force field, the correct conformation, or thermodynamic stability. A small mean can hide a highly strained local region, and a stationary point need not be a stable minimum.

For unconditional generation, composition is generated as well as geometry. A force condition could therefore alter the sampled composition instead of controlling relaxation for a fixed chemical system. The graph-conditioned structure prediction experiments partly address this concern and should be emphasized. Please add repeated force sweeps for the same graphs/compositions, per-atom force-tail statistics, and generated-sample validation with an independent potential or a modest DFT subset.

The coverage of the reported metrics also needs to be explicit. UMA target MAE is computed only for successfully scored samples; report the failure rate and its dependence on force condition. RDKit validity is based on the largest molecular fragment (Appendix D.2), which can conceal poor whole-system outputs. For genuinely multicomponent OMol systems, the solution is not to require a single connected molecule, but to report fragment counts, retained atom fractions, and whole-system chemical/physical checks.

Finally, the conformer-training filter removes disconnected structures after OpenBabel bond inference, excluding 60.32% of OMol25. Some excluded structures are legitimately multicomponent, while some connected inferred graphs may still be wrong. Please characterize the retained subset and avoid equating connectedness with reliably recovered chemistry.

6. Benchmark and scaling results need clearer separation from the pretraining claim

The QM9/MP20 and GEOM-Drugs benchmarks characterize separately trained instances of the architecture. They are useful, but they do not directly demonstrate the benefit of OMol25/OMat24 pretraining. Please label this distinction prominently and compare the new architecture with Zatom-1 under matched data, capacity, training, and sampling budgets where practical.

The scaling experiment also needs a precise budget definition. Appendix C.2 defines an epoch as approximately five million replacement draws, whereas Table 7 lists the 500k-per-domain runs as “16 * 5 = 80 epochs” and reports about five times less compute than the full-data runs: 540 versus 2,850 A100 GPU-hours for the base model and 790 versus 4,090 for the large model. The current presentation does not cleanly separate coverage from optimization budget. Please plot quality against optimizer updates, tokens, and GPU-hours, and include a same-compute smaller-corpus control. It is fair to conclude that more data and compute improve these runs; isolating data coverage requires a stronger comparison.

The reported molecular benchmark gains are also mixed. On GEOM-Drugs, Zatom-2 has lower validity and lower valid-and-PoseBusters-valid yield than TABASCO, although it is competitive. On conformer generation, Table 14 shows improved precision but reduced recall; the appendix appropriately acknowledges this. These results support broad competence, not uniformly superior generation.

Questions for authors
  1. Does the protein transfer gain persist across training/subset seeds, and after removing related biomolecular examples from pretraining?
  2. How much of the benefit comes from energy/force labels, rather than changed losses, latent regularization, or task allocation? Does OMat pretraining add value beyond OMol?
  3. What happens to extrapolation novelty under length-matched and domain-level comparisons?
  4. What causes the severe OMat24 force-consistency regression, and how does resolving it affect the claimed shared-model benefit?
  5. Are the model's own generated sequences and side chains consistent with its generated backbones?
5 comments
AuthorsOpenReview ↗

Author Rebuttal for Reviewer 9Hmr (Part 1/3)

Author Rebuttal for Reviewer 9Hmr (Part 1/3: Protein Transfer, Overlap Controls, and Novelty)

We thank Reviewer 9Hmr for their thorough review. Part 1/3 covers protein transfer, overlap controls, and novelty/diversity (W1–W2, Q1–Q3); Part 2/3 covers all-atom proteins and crystals (W3–W4, Q4–Q5); and Part 3/3 covers force control, molecular checks, and scaling (W5–W6).

1. Multi-Seed Uncertainty, Isolating Pretraining Factors, and OMol25 Overlap Audit (W1, Q1, Q2)

All nine Small (70M) and Base (140M) SCOPe-2k models are now evaluated across three sampling seeds (3,081 ID and 3,072 OOD backbones; 49,224 ESMFold2 refolds/model; Table 3, Appendix Table 11). In OOD extrapolation (129–256 residues), Zatom-2 (V) lifts designability from 67.8 ± 1.3% (scratch) and 54.0 ± 1.8% (RFdiffusion3) to 74.8 ± 1.1%, and per-sequence success from 34.7% to 45.8%; stratified bins (Table 12) show this lead widening at 193–256 residues to 71.5 ± 0.5% vs. 61.5 ± 1.9% (training from scratch) and 37.5 ± 1.2% (RFdiffusion3). While our three SCOPe-2k seeds measure sampling/refolding variance rather than repeating nine 50k-step runs across subset seeds within the rebuttal window (due to time and compute constraints), Figure 8 shows smooth late-training plateaus across pretrained runs, and we independently validated transfer on a new 12× larger SPINDR binder design task (23,955 complexes; Appendix E.9, Table 24), where Zatom-2 (V) beats training from scratch across three seeds in <1.5 Å designability (4.37% to 6.90%, 1.58×), cluster diversity (27.7 to 48.7), novelty (13.7% to 24.7%), and UMA-S-1p2 pocket forces (2.23 to 1.33 eV/Å).

To disentangle pretraining factors (Q2), fractional and lattice losses (λfrac,λlat\lambda _ {\mathrm{frac}}, \lambda _ {\mathrm{lat}}) are masked to zero on molecules and proteins, and restoring 10/1/1 periodic weights in Zatom-2 (V) leaves molecule metrics unchanged (Table 9). Moreover, our 70M OMol25-only model ablations (Training from scratch vs. Zatom-2 (I–III); Table 11) isolates each non-MLIP task with zero crystal data: generation alone (I) raises OOD TMgeo\mathrm{TM} _ {\mathrm{geo}} novelty (66.6% to 73.6%), force conditioning (II) lifts OOD designability to 55.3% and novelty to 80.9%, and structure prediction (III) drives an +11.2-point jump in OOD designability to 66.5 ± 0.9% (+12.8 points over 70M scratch-trained Zatom-2's 53.8%; 417.0 to 601.0 clusters). Since Zatom-2 (III) (70M, OMol25) and IV (140M, OMol25+OMat24; 68.0% OOD) differ in size, our results suggest that OMol25 alone suffices for representation transfer while OMat24 co-training preserves shared-trunk compatibility, rather than proving an independent inorganic performance boost. With 140M parameters, Zatom-2 (V) (74.8% OOD) allocates fewer training steps to generation and structure prediction (34%/33% vs. 50%/50% in IV) while pairing energy/force labels with Leq\mathcal{L} _ {\mathrm{eq}} on t=1t = 1 views; the direct imprint of force/energy supervision appears in unseen-protein clustering (88.1% vs. 84.8% 10-NN purity and 4.3× localized decoding sensitivity, Figure 4) and 33%–38% lower protein forces with native cohesive energies (−9.04 / −9.00 vs. −8.84 / −8.70 eV/heavy atom in IV; Table 13).

Auditing all 3,986,754 OMol25-4M rows (Appendix B.1, Table 15) confirms they contain zero full protein chains or domains: SPICE has zero PDB proteins (≤2\le 2 residues), 56.4% of biomolecules (799,988 rows, max 350 atoms) are synthetic fragments or nucleic acids, and all 348,406 PDB protein entries are spherical 6 Å cutouts tokenized as atom1 point clouds without atom14 frames or sequences. Only 2,890 rows (0.073% of OMol25-4M, across 538 PDBs) share a PDB ID with SCOPe-2k, spanning a median of 111 atoms across 3 fragments and 3 contiguous residues (max 11). Excluding generated backbones homologous to overlapping PDB IDs, superfamilies, or folds via Foldseek and TM-align leaves Zatom-2 (V)'s OOD lead intact across every filter (74.7%–79.1% vs. 67.0%–71.6% scratch; Table 15).

2. Length-Corrected Novelty and Subsampled Diversity (W2, Q3)

Query-normalized TMq\mathrm{TM} _ q against Lr≤128L _ r \le 128 domains is bounded by 128/Lq128/L _ q on OOD queries (Lq∈[129,256]L _ q \in [129, 256]), whereas unrestricted TMt\mathrm{TM} _ t collapses to 0% novelty because 20–30 residue helices align locally inside longer proteins (Appendices D.9, E.3). Symmetric TMgeo=TMqTMt<0.5\mathrm{TM} _ {\mathrm{geo}} = \sqrt{\mathrm{TM} _ q \mathrm{TM} _ t} < 0.5 in Tables 3 and 11 (75.4% OOD / 16.3% ID for V vs. 63.3% / 11.0% Base scratch) and TMavg<0.5\mathrm{TM} _ {\mathrm{avg}} < 0.5 in Table 12 (62.3% vs. 49.5% Base; 75.1% vs. 55.9% 70M) remove both biases relative to SCOPe-2k. Subsampling M=500M = 500 designable backbones/model (Table 12) confirms higher intrinsic diversity for Zatom-2 (V) (281.4 ID / 418.2 OOD clusters vs. 255.6 / 354.6 for scratch-trained Zatom-2 and 265.8 / 414.9 for RFdiffusion3).

AuthorsOpenReview ↗

Author Rebuttal for Reviewer 9Hmr (Part 2/3)

Author Rebuttal for Reviewer 9Hmr (Part 2/3: All-Atom Protein Quality and Periodic Materials)

(Continued from Part 1/3)

3. All-Atom Geometry, Native Sequences, Oracles, and Checkpoints (W3, Q5)

To evaluate Zatom-2's all-atom protein quality without SolubleMPNN, we analyzed unrelaxed atom14 outputs across all 30,765 proteins and scored 4,800 protonated structures with UMA-S-1p2 (Appendix E.3, Table 13). All sample cohorts exceed 99.0% Ramachandran favored residues, while Zatom-2 (V) achieves the lowest Engh & Huber bond/angle RMSZ among Zatom-2 variants (1.870/1.298 ID, 1.915/1.221 OOD), cuts steric clashes 3.6× ID and 2.8× OOD over scratch-trained Zatom-2 (83.2% ID and 48.9% OOD clash-free vs. 56.3%/16.0% scratch-trained Zatom-2 and 63.8%/39.8% RFdiffusion3), lowers heavy-atom forces by 33%–38% below IV (2.73/2.62 vs. 4.10/4.24 eV/Å), and matches native SCOPe-2k binding energies (−9.04 ID and −9.00 OOD vs. −9.03 eV/heavy atom). Refolding Zatom-2's single native sequence without SolubleMPNN (Table 14) yields lower designability across model variants due to light sequence weighting (λseq=0.1\lambda _ {\mathrm{seq}} = 0.1 vs. λFM=4.0\lambda _ {\mathrm{FM}} = 4.0), confirming that Zatom-2's main protein strength is clash-free coordinate generation; even so, Zatom-2 (V) lifts native ID designability 4.9× over IV (3.80% vs. 0.78%) and nearly doubles OOD TM≥0.5\mathrm{TM} \ge 0.5 foldability (16.0% vs. 8.6%).

On the 2,000 ground-truth SCOPe-2k domains, SolubleMPNN + ESMFold2 yields a 58.2% base rate at <1.5 Å (55.9% with pLDDT > 70; 41.1% at <1.0 Å). Across <1.0 Å, <1.5 Å, <2.0 Å, and pLDDT > 70 gates (Table 12), Zatom-2 (V) leads OOD performance everywhere, reaching 44.0 ± 2.0% at <1.0 Å (vs. 31.7% scratch and 27.2% RFdiffusion3—surpassing native SCOPe-2k's 41.1%) and 74.3 ± 1.0% with pLDDT > 70. Cross-checking 200 stratified backbones/model with ESMFold v1 (Table 14) yields 92.8% ID and 88.0% OOD agreement with ESMFold2, with Zatom-2 (V) leading OOD per-sequence success (43.5%), <1.0 Å designability (43.0%), and dual-oracle consensus (66.0%). Appendix E.3 documents our deterministic checkpoint sweep over 96 fixed-seed ID backbones every 5k steps (35k for 70M scratch, 145k/135k/140k for I–III, 25k for 140M scratch, 125k/120k/130k for IV w/o FC/IV/V, 30k for RFdiffusion3).

4. Diagnosing OMat24 Force Outliers and Full MP20 Stability Breakdown (W4, Q4)

We traced the OMat24 regression from Zatom-2 (IV) to V in Table 9 (match rate 0.446 to 0.294; guided target median force AE 1.70 to 30.55 eV/Å) to guidance interacting with periodic readout weights rather than degraded trunk features (Appendices E.1–E.2). Table 9 applies force-only guidance (scale 2) to the Cartesian coordinates path (λFM=4.0\lambda _ {\mathrm{FM}} = 4.0, active across t∈[0,1]t \in [0, 1]), whereas crystals decode at the final step from fractional/lattice heads gated only near t≈1t \approx 1 (g1 A˚(t)g _ {1\text{ Å}}(t)) whose weights were cut 20–40× (10/1/1 to 0.25/0.05/0.05) to balance task gradients, amplifying mismatch. Three tests confirm this: (1) evaluating all 500 targets without guidance eliminates the median regression—unguided median target force AE improves from 1.21 eV/Å (IV) to 1.09 eV/Å (V) and ≥17-atom mean error drops 3.6× (24.59 to 6.91 eV/Å), leaving only a mean spike from 26/500 (5.2%) 2–4 atom cells with <1.0 Å clashes (74.1% of error); (2) fixed-cell refinement with frozen Zatom (VI) lowers rattled forces on 47/50 to 50/50 crystals (5.24 to 2.42 eV/Å median; Table 23); and (3) continuing Zatom-2 (V)'s training for 10 epochs with restored 10/1/1 weights—retaining λE/λF/λeq=0.1/0.3/0.1\lambda _ E / \lambda _ F / \lambda _ {\mathrm{eq}} = 0.1/0.3/0.1—lifts guided OMat24 match to 0.462 (beating IV's 0.446 and 270M VI's 0.460) and cuts guided target median force AE 10.4× to 2.95 eV/Å, with intact OMol25 prediction (1.757 Å RMSD, 0.625 eV/Å MAE; Table 9) and validity (85.8% OMol25, 75.6% OMat24). The same 0.25/0.05/0.05 weights plus a 10% periodic step share explain Table 10's dip before 160-epoch recovery (30.8% to 38.2% match; 38.8 to 25.8 eV/Å).

On MP20 unrelaxed generation (Section 4.2; Appendix E.4.1, Table 17), Zatom-2 trades lower validity (70.24%) and uniqueness (53.44% attempted, 76.09% valid) than Zatom-1-L (95.0%, 90.4%)—reaching 100.0% physical plausibility and 98.52% SMACT neutrality, limited only by unrelaxed distance checks (71.43%)—for 8.7× higher novelty (32.24% attempted, 45.90% valid vs. 3.7%). Whereas Zatom-1-L reaches 3.1% stable (Ehull≤0E _ {\mathrm{hull}} \le 0) yield near training examples for 0.68% SUN + MetaSUN, Zatom-2 reaches 19.76% metastability (0.00% unrelaxed Ehull≤0E _ {\mathrm{hull}} \le 0), yielding 4.92% MetaSUN and 4.92% SUN + MetaSUN (7.7× and 7.2× Zatom-1-L), with tight MACE/ORB/UMA agreement (19.97 ± 0.89% metastable, 0.015 eV/atom EhullE _ {\mathrm{hull}} std); DFT checks near 0.1 eV/atom remain a natural follow-up.

AuthorsOpenReview ↗

Author Rebuttal for Reviewer 9Hmr (Part 3/3)

Author Rebuttal for Reviewer 9Hmr (Part 3/3: Force Controllability, Molecular Checks, and Scaling)

(Continued from Part 2/3)

5. Force Controllability, Scoring Coverage, Whole-System Checks, and OpenBabel Audit (W5)

We agree that matching a global mean force norm does not guarantee locally strain-free geometry, calibrated conformations within each regime, or thermodynamic minima (Section 4.1, Appendix G.2). To test within-system control without composition shifts, Appendix E.7 (Table 22) sweeps sF∈{0.05,0.25,1.00,2.50,5.00}s _ F \in \lbrace 0.05, 0.25, 1.00, 2.50, 5.00 \rbrace eV/Å across 50 fixed OMol25 graphs and 50 fixed OMat24 compositions, reporting coverage and per-atom force tails (max⁡ℓ∥Fℓ∥2\max _ \ell \lVert \mathbf{F} _ \ell \rVert _ 2). RDKit and UMA coverage is strictly invariant to sFs _ F, succeeding on 45/50 (90.0%) OMol25 graphs at every target (87.1% PoseBusters pass; same 5 RDKit valence failures/target) and 50/50 (100.0%) OMat24 compositions. Across the 45 scored OMol25 graphs, 100% (45/45) show positive within-graph Spearman correlation between target and realized UMA forces (mean/median ρ=0.764/0.800\rho = 0.764 / 0.800, 81.6% concordance, AUROC 0.826 at 1.0 eV/Å and 0.977 at 2.5 eV/Å), with median max forces shifting from 2.40 to 4.53 eV/Å alongside mean forces. On OMat24, realized mean and max forces rise monotonically across rattled targets (1.0 to 5.0 eV/Å: median max force 27.19 to 81.67 eV/Å), while low targets plateau at the fractional-head floor. Bypassing that readout in single-step fixed-graph and fixed-cell zero-shot coordinate refinement at sF=0.05s _ F = 0.05 eV/Å (Appendix E.8, Table 23; 0.075 s on one GPU) lowers UMA forces and energies on 49/50 to 50/50 rattled molecules (9.66 to 2.86 eV/Å median at σin=0.30\sigma _ {\mathrm{in}} = 0.30 Å) and 47/50 to 50/50 rattled crystals (5.24 to 2.42 eV/Å, ΔE=−0.391\Delta E = -0.391 eV/atom)—acting as a fast pre-relaxation denoiser rather than a BFGS/DFT substitute, while our 3-MLIP check with MACE and ORB (0.015 eV/atom EhullE _ {\mathrm{hull}} std vs. UMA; Table 17) cross-validates stability outside UMA.

To expose largest-fragment RDKit validity artifacts and verify UMA coverage on unconditional generation, Appendix E.7 (Table 21) evaluates 1,000 OMol25 samples/model, where UMA succeeds on 100.0% of RDKit-valid structures. Zatom-1's 94.1% largest-fragment validity masks severe whole-system fragmentation—56.09 mean fragments retaining 12.5% of atoms, 1.00% PoseBusters pass rate, and 258.43 eV/Å mean max force. By contrast, Zatom-2 restores 1.0 median fragment (≈75% atoms in the largest fragment, reflecting multi-component electrolytes, metal complexes, and solvated pockets in OMol25), while energy/force supervision (IV to V/VI and V+FT extended) lifts whole-system PoseBusters pass rates from 30.3% to 34.7%–34.8% and 37.0% and cuts mean/median max atomic forces from 7.31/5.00 to 4.12/3.29 and 3.57/2.84 eV/Å. Auditing our OpenBabel filter across all 3,986,754 OMol25-4M rows (Appendix E.1) shows that 39.68% (1,581,810) form one connected component: covalent subsets retain 73.5%–99.2% connectivity across SPICE, Transition1x, ANI-2x, ORBNet Denali, and GEOM-ORCA6, whereas 79.6% of excluded entries come from multi-component metal complexes (38.6% connected), biomolecules (12.1%), and electrolytes (9.6%). We treat connectedness only as a single-graph heuristic, keeping all ≈4M rows for unconditional generation and MLIP pretraining.

6. Standalone Scratch Benchmarks, Molecular Trade-Offs, and Scaling Budget (W6)

Section 4.2 labels joint QM9+MP20 and GEOM-Drugs as from-scratch architectural controls. Under Zatom-1's matched splits, 100-step unguided sampling, and comparable size, Zatom-2 surpasses Zatom-1-L on QM9 validity (95.72% vs. 95.3%), GEOM-Drugs validity (96.90% vs. 93.6%), and MP20 SUN + MetaSUN (4.92% vs. 0.68%), while we note mixed results against specialized baselines: TABASCO edges out Zatom-2 on GEOM-Drugs validity (97.6% vs. 96.90%) and Valid & PB-valid yield (89.40% vs. 88.63%, Table 2), and GO-Flow achieves higher conformer recall (94.82% vs. 86.09% COV-R) whereas Zatom-2 achieves higher precision (83.07% vs. 70.13% COV-P, Table 19), supporting broad competence rather than uniform superiority. Finally, Section 4.3 and Appendices C.2–C.3 (Figure 3, Tables 6–7) clarify our scaling budget: each full-dataset epoch draws 4,996,604 examples (≈400M over 80 epochs), whereas 500k+500k runs trained for 16 such epochs (≈80M draws) on a 10610^6-draw effective-epoch axis (following Kaplan et al., 2020 [1]). Comparing full-dataset epoch 16 against subset effective epoch 80 in Figure 3 presents a matched 80M-draw control for data coverage, where pure data coverage differences are modest—suggesting that Zatom-2's largest gains come jointly from data coverage, compute, and model scale.

References:

[1] Kaplan et al. (2020). Scaling laws for neural language models. arXiv preprint arXiv:2001.08361.

Comments on the paper

3 comments
Area_Chair_ABSDOpenReview ↗

Authors: please respond to your reviewers

To the paper authors: Please acknowledge and respond to the thoughtful replies by reviewers. They have raised some going points about strengths and weaknesses of your paper, as well as questions that merit replies.

AuthorsOpenReview ↗

Author Response to Area Chair about Rebuttals

Dear Area Chair ABSD,

Thank you for the reminder. We are currently running the requested experiments and plan to post our detailed responses to the reviewers tomorrow.

Best,

Paper Authors

AuthorsOpenReview ↗

Global Response to the Area Chair and All Reviewers

Global Response

We thank all reviewers and the Area Chair for their thoughtful feedback and for recognizing Zatom-2's unified atom1 + atom14 formulation across molecules, periodic materials, and proteins, its force-conditioned flow matching design, and its transfer to low-data protein generation. We have uploaded a revised manuscript (changes highlighted in red) incorporating new experiments, representation probes, and diagnostic controls, summarized below.

Mechanistic Basis and Task-Level Isolation of Pretraining

To clarify why auxiliary energy/force supervision improves generation, we probed Zatom-2's representations on held-out molecules and proteins (Section 4.5, Figure 4; Appendix F.4, Figures 15–17). On 768 OMol25 validation molecules, energy/force supervision lifts cross-structure 10-NN chemical-subset purity by 12.6–14.9 percentage points across the elements H, C, N, and O. Further, on 128 unseen SCOPe protein domains, SCOPe finetuning from Zatom-2 (V) raises cross-protein 10-NN amino acid type purity in Zatom-2's latent space from 84.8% to 88.1% and increases Calinski-Harabasz separation by 64% (374.9 vs. 228.1) over SCOPe-only training. Notably, within Zatom-2's architecture, perturbing one residue's latent token before decoding it induces 4.3× stronger localized coordinate sensitivity on that residue (12.50 vs. 2.88 Ų MSE at α=0.5\alpha = 0.5) with negligible off-target residue leakage (<0.001 Ų), suggesting that Zatom-2's pretraining supports clean compositional assembly at longer sequence lengths.

We also expanded all SCOPe-2k protein evaluations to three random seeds and added 70M OMol25-only pretraining ablations without material data alongside 140M Base model comparisons to RFdiffusion3 (Section 4.4, Table 3; Appendix E.3, Table 11). On OMol25 alone, force conditioning and structure prediction cumulatively raise 70M OOD protein designability from 53.8% (trained from scratch) to 66.5%. With 140M parameters, adding energy and force supervision in Zatom-2 (V) lifts OOD designability from 67.8 ± 1.3% (trained from scratch) and 54.0 ± 1.8% (RFdiffusion3) to 74.8 ± 1.1%, while raising per-sequence success from 34.7% to 45.8% and fold diversity from 470.7 to 618.7 clusters.

Physical Protein Realism, Overlap Controls, and Ligand-Binder Design

These protein transfer gains hold across length-corrected novelty (TMgeo\mathrm{TM} _ {\mathrm{geo}} and TMavg\mathrm{TM} _ {\mathrm{avg}}), equal-count fold clustering (M=500M = 500), finer length bins (71.5% designability at 193–256 residues vs. 61.5% from scratch and 37.5% with RFdiffusion3), pLDDT > 70 and <1.0 Å gates (44.0% OOD designability for Zatom-2 (V), surpassing native SCOPe-2k's 41.1% base rate), and ESMFold v1 refolding (66.0% dual-oracle consensus; Tables 12, 14). Across 30,765 unrelaxed all-atom proteins and 4,800 UMA-S-1p2-scored structures (Table 13), Zatom-2 (V) cuts steric clashes roughly threefold over training from scratch, lowers heavy-atom forces by 33%–38% below Zatom (IV), and matches native SCOPe-2k binding energies (−9.04 and −9.00 vs. −9.03 eV/heavy atom). Auditing all 4M OMol25-4M rows confirms there are zero full protein chains—only 6 Å cutouts tokenized as atom1 point clouds, with 0.073% sharing a SCOPe-2k PDB ID—and excluding homologous SCOPe folds leaves Zatom-2 (V)'s OOD lead intact (Appendix B.1, Table 15).

To test transfer beyond monomers, we benchmarked ligand-conditioned all-atom binder design on SPINDR across 1,000 generated complexes spanning 137 test ligands over three seeds (Appendices B.6, E.9, F.3, Table 24, Figure 14). Initializing from Zatom-2 (V) improves <1.5 Å holo designability 1.58× (4.37% to 6.90%), nearly doubles fold clusters (27.7 to 48.7) and novelty (13.7% to 24.7%), raises PoseBusters clash-free rates from 11.1% to 14.6%, and cuts UMA-S-1p2 pocket intermolecular forces by 40% (2.23 to 1.33 eV/Å).

Periodic Crystal Readouts, Molecular Coherence, and Force Control

Appendix E.1 traces Zatom-2 (V)'s guided OMat24 force error in Table 9 (30.55 eV/Å target median AE) to force-only guidance steering the Cartesian coordinates path away from endpoint-gated fractional/lattice heads whose loss weights were cut 20–40× to balance gradients: without guidance, median target force error improves from 1.21 to 1.09 eV/Å, and restoring 10/1/1 periodic weights for 10 epochs lifts the guided match rate to 0.462 while cutting guided target median force error 10.4× to 2.95 eV/Å without hurting molecule tasks. Fixed-graph sweeps confirm 100% positive within-graph Spearman force correlation across 50 OMol25 graphs (mean ρ=0.764\rho = 0.764; Table 22), one-step zero-shot refinement lowers rattled forces on 50 molecules and 50 crystals (Table 23), whole-system checks confirm 1-fragment median coherence (Table 21), and Sections 4.2–4.3 clarify baseline benchmarks and effective-epoch scaling.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.