acceptodds
Under review as a conference paper at ICLR 2027

When Forces Disagree: An Out-of-the-Box Stability and Uncertainty Estimate for Direct-Force Machine Learning Interatomic Potentials

Abstract

Machine learning interatomic potentials (MLIPs) that predict forces directly, rather than as the gradient of a predicted energy, accelerate atomistic simulations but are unstable in molecular dynamics (MD). They also lack an out-of-the-box single-model uncertainty quantification (UQ) metric, since existing single-model methods require either non-standard MLIP training or post-hoc fitting beyond energy and force matching. Here, we address both problems with the Force Delta (U∆), the magnitude of the difference between the direct and conservative forces of a single model, which is available out of the box for any direct-force MLIP. We first find that MD stability depends more on conservativity than on force accuracy. We then show that U∆ tracks both the exact but expensive Jacobian-based measure of non-conservativity and the resulting instability. Further, U∆ correlates well with the true force error across equivariant and non-equivariant architectures on in- and out-of-distribution systems, rivaling or outperforming other single-model UQ metrics and ensembles without any additional training or fitting beyond standard MLIP training. Finally, finetuning on only 200 structures that maximize U∆ improves both the stability and the accuracy of direct forces in pretrained universal MLIPs. U∆ therefore offers an out-of-the-box way to monitor and improve fast direct-force MLIPs, enabling their safe application to materials discovery.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
machine learning interatomic potentials, uncertainty quantification, direct-force models, non-conservative forces, simulation stability, molecular dynamics, active learning, out-of-distribution detection

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_bi7zOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper introduces Force Delta (UΔU_{\Delta}), the mean absolute difference between direct and gradient-based force predictions to measure stability and uncertainty in direct force MLIPs. This approach is motivated by showing direct force predictions cause energy drift even with low pointwise force errors. On 50 MPtrj systems, UΔU_{\Delta} has high correlation to an exact Jacobian conservatively measure and to downstream energy drift. As an UQ metric, UΔU_{\Delta} is shown to be superior or competitive over some systems against ensembles and other UQ metrics, including ones requiring training/fitting. On other systems there is low correlation between the metric and the error to DFT. The authors demonstrate using UΔU_{\Delta} for efficient finetuning with an active learning setup, showing reduced force error and energy drift using up to 200 structures.

Critical strengths

Immediate UQ for direct force MLIPs. The method is simple, works out-of-the-box for direct force MLIPs in a single model, and requires no additional training. High correlation with exact non-conservativity measure and simulation drift. On an Orb model with 50 different materials systems, the paper shows high correlation between UΔU_{\Delta} to \lambda and simulation energy drift, offering a cheaper way to measure non-conservativity. The comparisons reported against other UQ metrics are strong, showing UΔU_{\Delta} is competitive across MLIP architectures. Active Learning. The empirical results of the active learning are very good, showing substantial decrease in test error and nearly removing the drift using only 100 structures over 2 generations (200 total).

Critical weaknesses

Instability limited to drift. UΔU_{\Delta} is validated as a stability metric against energy drift, while instability can manifest in a multitude of different ways (unphysical geometries, temperature drift, structural collapse, etc.). Correspondingly, showing just energy drift with accurate direct force predictions in section 4.1 is unsurprising. It is established that low pointwise force error does not guarantee energy conservation, including drift in direct-force models even with lower force errors [1]. While section 4.1 mentions the authors saw increased temperature fluctuations in NVT simulations using the direct forces, it is established in prior work [2] and this analysis is limited to that section only. The authors should show instabilities other than just drift or restrict claims about using the metric to drift and not stability overall. Correlation with \lambda measure is limited to Orb. The high correlation with the exact non-conservativity measure \lambda is only done with two Orb models and MPtrj systems, so it is not established whether these results generalize to other architectures and systems. Weak UQ results on certain systems. Certain systems in the adversarial attack don’t have a strong correlation with εnc\varepsilon_{\mathrm{nc}} (Table 4). This is not reported in the main text, which just states “UΔU_{\Delta} retains a strong positive correlation with εnc\varepsilon_{\mathrm{nc}} across structures”, referring only to Figure 3. This also does not hold up with the results presented in Table 4. The reasoning given in the appendix, that this is due to initial inaccuracy of the model, seems to only hold partially in Figure 7, as some of the systems with low correlation also have high errors. If it is indeed the reasoning, show stronger evidence, and discuss it in the main text. Additionally, the comparison against trained UQ (Table 2) is done only over ammonia, while with other systems it might not perform as well. Active learning ablations. While the active learning results are impressive, without ablations it is unclear whether the improved energy drift (and test accuracy) can be attributed to using the 200 UΔU_{\Delta}-maximized structures, or to the additional training, particularly due to supervising the conserving forces. This is especially true for the second generation results. Including an ablation study (different methods for choosing the fine tuned structures, supervising different combinations of conserving and direct forces, number of generations, etc.) would strengthen this. Active learning drift measurement. Measuring drift for 10 ps can be insufficient and is inconsistent with other drift measurements done in the paper. The authors should use a consistent justifiable drift measurement protocol throughout the paper.
Aligned direct and conservative force errors. The authors acknowledge that a limitation of UΔU_{\Delta} is when the two forces differ in the same direction and magnitude from the ground truth, however they do not characterize this mode of failure (diagnosis, dependence on system? architecture?). Non-equivariant direct force head for MACE. In the MACE setup, the added Orb-style direct force prediction head breaks the equivariance of the model, weakening the claim that UΔU_{\Delta} works well on equivariant architectures.

[1] Zhang, D., Bi, H., Dai, FZ. et al. Pretraining of attention-based deep learning potential model for molecular simulation. npj Comput Mater 10, 94 (2024). https://doi.org/10.1038/s41524-024-01278-7 [2] Bigi, F., Langer, M. F., & Ceriotti, M. (2025). The dark side of the forces: assessing non-conservative force models for atomistic machine learning. Forty-Second International Conference on Machine Learning. Retrieved from https://openreview.net/forum?id=OEl3L8osas

Questions for authors

What is the motivation for using mean absolute difference and not metrics (e.g., root mean square difference)? In Figure 2, can you color the points markers by the system and/or model? Any outliers? What is the spearman correlation of each system? In the MACE setup, why did you use the Orb-style non-equivariant direct force head and not an equivariant alternative? Can you show how UΔU_{\Delta} performs with MACE using an equivariant head? In the active learning fine tuning of Orb, does each setting use the resulting model of the previous setting? So does the finetuning of F^nc\hat{F}_{\mathrm{nc}} and F^c\hat{F}_{\mathrm{c}} together use the model that was trained on the same 100 structures of finetuning F^nc\hat{F}_{\mathrm{nc}} only, and does the second generation use the model that was trained on both? Can you show the computational efficiency of the method compared with the baseline model and other methods? Is there a way to diagnose when a user can trust that a small UΔU_{\Delta} is attributed to a small force error and not to the two predictions deviating in the same direction? Which systems are the results in Table 1 reported over? In section 4.6, what is the timestep used and MD protocol used?

8 comments
AuthorsOpenReview ↗

Replies to Reviewer bi7z (Part 1)

We thank the reviewer for thoroughly reviewing our work and for giving us many valuable suggestions, which led to new experiments that substantially improve the generality of the paper and further support the UΔU_\Delta utility claims in the original submission. We address each point below.

Instability limited to energy drift. We agree with the reviewer and have restricted our stability claims to energy drift in the revised version. We focus on energy drift because Bigi et al. [2] show that it is the root cause of the other instabilities, including the structural and dynamical property deviations that follow from thermostat interference, and that correcting the drift also recovers them. We have also retained our NVT measurement as a confirmation of their result, as originally intended.

We also thank the reviewer for pointing us to Zhang et al. [1], which, together with Bigi et al. [2], shows that a conservative force can give more stable dynamics than a more accurate direct force. We have reframeed Section 4.1 as a confirmation of this finding, which our setting extends: in our work F^c\hat{\mathbf F}_{\mathrm c} is never trained and has large error, unlike the trained conservative force in [1, 2], so exact conservativity matters more than static accuracy for energy conservation even when the conservative force is substantially less accurate. Section 4.1 has also been presented as the motivation for UΔU_\Delta as a cheap proxy for non-conservativity and energy drift in the revised version, in place of the exact Jacobian-based measure λ\lambda, which requires 3N3N backward passes per structure.

Correlation with λ\lambda limited to Orb, and per-system statistics. The reviewer is right about the limited scope. We accordingly extended the Section 4.2 experiments to four architecture/dataset settings spanning Orb and the equivariant eqV2 evaluated on MPtrj and OMat24 [3], with every correlation recomputed within each trajectory so that each value represents a single system. We also extended the NVE simulation time from 5 ps to 50 ps for consistency throughout the paper as suggested by the reviewer, and performed DFT calculations on 100 evenly sampled frames along each trajectory to track the force error εnc\varepsilon_{\mathrm{nc}}.

Median [interquartile range] over 50 trajectories per setting:

Spearman ρ\rhoorb-inf-mpa / MPtrjorb-inf-omat / OMat24eqV2-31M-mp / MPtrjeqV2-31M-omat / OMat24
ρ(UΔ,λ)\rho(U_\Delta, \lambda)0.72 [0.59, 0.86]0.67 [0.48, 0.85]0.79 [0.65, 0.94]0.83 [0.67, 0.95]
ρ(UΔ,εnc)\rho(U_\Delta, \varepsilon_{\mathrm{nc}})0.83 [0.70, 0.92]0.79 [0.67, 0.91]0.83 [0.68, 0.94]0.77 [0.64, 0.92]
ρ(UΔ,energy drift rate)\rho(U_\Delta, \text{energy drift rate})0.75 [0.52, 0.84]0.71 [0.53, 0.89]0.84 [0.61, 0.95]0.87 [0.69, 0.92]

UΔU_\Delta still tracks λ\lambda, energy drift, and also the force error well within each trajectory of a single system, across all four settings. This also explains the active learning results below, where finetuning on high-UΔU_\Delta structures improves energy drift and accuracy together, and considerably more than the MD baseline. UΔU_\Delta can therefore be interpreted as a practical joint indicator of per-material non-conservativity and force error within an MD trajectory, without DFT labels or the expensive Jacobian-based λ\lambda.

A few trajectories fall below the median, in which the drift stays small throughout and the correlation is correspondingly noisy, though still positive:

SettingTrajectories with ρ(UΔ,λ)<0.5\rho(U_\Delta, \lambda) < 0.5Lowest ρ\rho
orb-v3-inf-mpa / MPtrj2 / 500.14
orb-v3-inf-omat / OMat243 / 500.07
eqV2-31M-mp / MPtrj2 / 500.19
eqV2-31M-omat / OMat242 / 500.09

[1] D. Zhang, H. Bi, F.-Z. Dai, W. Jiang, X. Liu, L. Zhang, and H. Wang. Pretraining of attention-based deep learning potential model for molecular simulation. npj Computational Materials, 10:94, 2024.

[2] F. Bigi, M. F. Langer, and M. Ceriotti. The dark side of the forces: assessing non-conservative force models for atomistic machine learning. In ICML, PMLR 267, 2025.

[3] L. Barroso-Luque, M. Shuaibi, X. Fu, B. M. Wood, M. Dzamba, M. Gao, A. Rizvi, C. L. Zitnick, and Z. W. Ulissi. Open Materials 2024 (OMat24) inorganic materials dataset and models. arXiv:2410.12771, 2024.

[4] Y.-L. Liao, T. Smidt, M. Shuaibi, and A. Das. Generalizing denoising to non-equilibrium structures improves equivariant force fields. Transactions on Machine Learning Research, 2024.

[5] A. R. Tan, S. Urata, S. Goldman, J. C. B. Dietschreit, and R. Gómez-Bombarelli. Single-model uncertainty quantification in neural network potentials does not consistently outperform model ensembles. npj Computational Materials, 9:225, 2023.

[6] L. Kahle and F. Zipoli. Quality of uncertainty estimates from neural network potential ensembles. Physical Review E, 105:015311, 2022.

AuthorsOpenReview ↗

Replies to Reviewer bi7z (Part 2)

Weak UQ results on certain systems, and aligned force errors. We agree with the reviewer that the weak correlations in Table 4 and the aligned-error failure mode deserve further investigation than a few sentences in the appendix, and we thank the reviewer for pointing these out. We accordingly extended the adversarial attack experiments to five additional pretrained eqV2 models and two MACE models trained from scratch on an MPtrj subset, one direct-force-only and one hybrid, both with the exactly equivariant head described below, all validated against DFT. Entries are ρ(UΔ,εnc)\rho(U_\Delta, \varepsilon_{\mathrm{nc}}), ρ(UΔ,εc)\rho(U_\Delta, \varepsilon_{\mathrm c}), ρ(εc,εnc)\rho(\varepsilon_{\mathrm c}, \varepsilon_{\mathrm{nc}}):

SystemeqV2-dens-86M-mpeqV2-dens-153M-mpeqV2-31M-mpeqV2-31M-omat-mp-salexeqV2-86M-mpMACE directMACE hybrid
ice0.98, 0.99, 0.950.92, 1.00, 0.940.88, 1.00, 0.900.87, 0.98, 0.830.92, 0.97, 0.930.96, 0.92, 0.961.00, 0.97, 0.97
Mg17Al120.86, 0.98, 0.920.95, 0.99, 0.930.95, 0.99, 0.910.88, 1.00, 0.850.89, 0.98, 0.921.00, 1.00, 1.001.00, 1.00, 1.00
LGPS0.88, 1.00, 0.850.91, 0.97, 0.930.97, 1.00, 0.990.93, 0.98, 0.921.00, 1.00, 1.001.00, 1.00, 1.000.79, 0.87, 0.93
CaPd-NH$_2$0.35, 1.00, 0.330.25, 0.98, 0.160.99, 0.98, 1.000.98, 0.98, 0.961.00, 0.98, 0.970.98, 0.98, 0.971.00, 0.95, 0.95
aspirin0.58, 0.97, 0.600.71, 0.89, 0.700.95, 0.99, 1.000.98, 1.00, 0.970.95, 0.97, 0.960.99, 1.00, 0.990.88, 0.93, 0.91
paracetamol0.07, 0.99, 0.150.87, 0.97, 0.730.75, 0.99, 0.821.00, 1.00, 1.001.00, 1.00, 1.001.00, 1.00, 1.000.94, 1.00, 0.94
MOF-50.13, 0.99, 0.090.24, 0.98, 0.170.89, 0.99, 0.870.96, 1.00, 0.960.92, 1.00, 0.931.00, 1.00, 1.001.00, 1.00, 1.00

Together with the 22 pairs of Table 4, this gives 71 model–system pairs. The correlation with the direct-force error is strong across most of the set: the median ρ(UΔ,εnc)\rho(U_\Delta, \varepsilon_{\mathrm{nc}}) is 0.92 over all 71 pairs, and 0.95 over the 51 pairs from models not trained with a denoising objective (DeNS) [4], with 40 of those at 0.85 or above. The weak correlations concentrate in the DeNS checkpoints, whose median is 0.51 over 20 pairs and which improves monotonically with model capacity: 0.33 at 31M, the checkpoint used in Table 4, then 0.58 at 86M and 0.87 at 153M.

One possible explanation is that DeNS trains the per-atom output head to predict the added position noise and the per-structure energy head to predict the energy of the uncorrupted structure from corrupted input [4], so the two targets are not related by differentiation as they are under standard direct-force training. This may make the two heads behave differently enough that their errors decouple and UΔU_\Delta cannot capture both errors at once, with extra capacity partially absorbing the difference.

Whether the two force errors move together, ρ(εc,εnc)\rho(\varepsilon_{\mathrm c}, \varepsilon_{\mathrm{nc}}), also predicts how well UΔU_\Delta ranks εnc\varepsilon_{\mathrm{nc}}: the two correlations agree at Pearson r=0.985r = 0.985. Both heads read the same representation, so out of distribution their errors usually rise together and UΔU_\Delta tracks both errors well. On a few paths, DeNS-trained models in particular, the two heads behave differently enough that the attack on UΔU_\Delta raises the error of one force while the other (mostly the trained direct force head) stays robust, decoupling the two errors and weakening ρ(UΔ,εnc)\rho(U_\Delta, \varepsilon_{\mathrm{nc}}).

Aligned errors, in which both forces deviate in the same direction with large errors while UΔU_\Delta stays small, are rare: 1 of 71 paths, namely MOF-5 under orb-d3-xs-v2, the smallest and only non-equivariant model in the set, on the most structurally heterogeneous system we test. This suggests that the consensus failure mode may occur when a system is so far from the training prior, especially for a model of small capacity, that the model stops responding to the distribution shift and outputs a constant small UΔU_\Delta while the error keeps increasing as the ground truth shifts further from both forces. This failure mode is inherent to disagreement-based UQ, as with deep ensembles, which are known to underestimate errors that all members share in the OOD regime [6]. Our results nonetheless suggest practical guidelines for avoiding it: prefer larger models, avoid denoising-trained checkpoints where UΔU_\Delta is to be used for the direct force, and treat systems far from the training distribution with caution, for instance by labeling the true error of a few low-UΔU_\Delta structures with DFT. We have added this discussion to the revision and the full results to the appendix.

AuthorsOpenReview ↗

Replies to Reviewer bi7z (Part 3)

Trained-UQ benchmarks beyond ammonia, and an equivariant direct-force head for MACE. The reviewer is right that the MLP head in the submission breaks equivariance of MACE. We used the Orb-style head to keep the MACE and Orb architectures comparable, but we agree with the reviewer and have implemented an exactly equivariant direct-force head: Fi=∑cwic vic\mathbf F_i = \sum_c w_{ic}\,\mathbf v_{ic}, where vic\mathbf v_{ic} are the 128 l=1l = 1 equivariant features of atom ii from MACE's first interaction layer and the weights wicw_{ic} are predicted by an MLP from the invariant (l=0l = 0) features of both layers. Invariant weights multiplying equivariant vectors give F(Rx)=RF(x)\mathbf F(R\mathbf x) = R\mathbf F(\mathbf x) exactly; a rotation test gives a violation of 4×10−154 \times 10^{-15} on ammonia and 1.7×10−151.7 \times 10^{-15} on silica, against 1.5×10−11.5 \times 10^{-1} for the MLP head, as the reviewer correctly points out.

We evaluated this head on ammonia and on two further benchmarks: condensed-phase SiO2 from Tan et al. [5], with large periodic cells and an OOD test set of high configurational complexity from high-deformation-rate trajectories, and a multi-system MACE trained on a 20k subset of MPtrj structures and tested on OMat24 rattled structures. Across all of them, UΔU_\Delta remains superior or competitive against ensembles and other UQ methods while being the only out-of-the-box single-model method available from standard direct-force MLIP training, without additional fitting or specialized training. As with the ammonia benchmark in the submission, its utility also extends to hybrid models in which both forces are trained and can be used.

  • Ammonia with the equivariant direct-force head. UΔU_\Delta remains competitive against the deep ensemble and outperforms GMM, while MVE remains unstable owing to its hard-to-optimize specialized loss. Submitted results are included for comparison. ρ\rho (AUROC):

    MethodOrbMACE, equivariant F^nc\hat{\mathbf F}_{\mathrm{nc}}MACE, MLP head
    UΔU_\Delta0.832 ± 0.026 (0.969 ± 0.007)0.874 ± 0.031 (0.982 ± 0.002)0.861 ± 0.055 (0.966 ± 0.003)
    GMM0.819 ± 0.070 (0.962 ± 0.023)0.841 ± 0.042 (0.979 ± 0.006)0.898 ± 0.030 (0.982 ± 0.003)
    MVE0.692 ± 0.120 (0.897 ± 0.076)0.702 ± 0.157 (0.862 ± 0.124)0.675 ± 0.241 (0.791 ± 0.129)
    Deep ensemble0.825 (0.982)0.933 (0.987)0.873 (0.976)
    ∣F^nc∣‾\overline{\lvert\hat{\mathbf F}_{\mathrm{nc}}\rvert} (baseline suggested by Reviewer B3Fn)0.723 ± 0.024 (0.891 ± 0.005)0.731 ± 0.072 (0.882 ± 0.100)0.773 ± 0.050 (0.902 ± 0.001)
  • Condensed-phase SiO2 [5]. ρ\rho (AUROC):

    MethodOrbMACE, equivariant F^nc\hat{\mathbf F}_{\mathrm{nc}}MACE, MLP head
    UΔU_\Delta0.917 ± 0.012 (0.986 ± 0.006)0.935 ± 0.031 (0.975 ± 0.008)0.908 ± 0.029 (0.982 ± 0.004)
    GMM0.784 ± 0.030 (0.967 ± 0.010)0.775 ± 0.018 (0.914 ± 0.009)0.796 ± 0.022 (0.921 ± 0.012)
    MVE0.635 ± 0.183 (0.827 ± 0.125)0.712 ± 0.243 (0.851 ± 0.243)0.681 ± 0.103 (0.867 ± 0.098)
    Deep ensemble0.951 (0.990)0.953 (0.988)0.932 (0.985)
    ∣F^nc∣‾\overline{\lvert\hat{\mathbf F}_{\mathrm{nc}}\rvert}0.762 ± 0.014 (0.831 ± 0.003)0.744 ± 0.004 (0.865 ± 0.002)0.722 ± 0.012 (0.867 ± 0.003)
  • Multi-system training from scratch, MACE with the equivariant head trained on an MPtrj subset, with an OMat24 rattled OOD test set:

    Methodρ\rhoAUROC
    UΔU_\Delta0.864 ± 0.0230.898 ± 0.002
    Deep ensemble0.9020.926
    ∣F^nc∣‾\overline{\lvert\hat{\mathbf F}_{\mathrm{nc}}\rvert}0.718 ± 0.0320.810 ± 0.021
  • Hybrid models, in which both forces are supervised, as in the MTS scheme of [2]. Practitioners can use either force in these models, and UΔU_\Delta correlates well with both errors, serving as a UQ metric for both simultaneously:

    Method / SystemF^nc\hat{\mathbf F}_{\mathrm{nc}}: ρ\rhoF^nc\hat{\mathbf F}_{\mathrm{nc}}: AUROCF^c\hat{\mathbf F}_{\mathrm c}: ρ\rhoF^c\hat{\mathbf F}_{\mathrm c}: AUROC
    UΔU_\Delta / Ammonia0.908 ± 0.0650.989 ± 0.0030.953 ± 0.0270.989 ± 0.003
    GMM / Ammonia0.847 ± 0.0190.975 ± 0.0050.873 ± 0.0270.975 ± 0.005
    Deep ensemble / Ammonia0.9060.9870.9600.984
    ∣F^∣‾\overline{\lvert\hat{\mathbf F}\rvert} / Ammonia0.791 ± 0.0620.894 ± 0.0780.832 ± 0.0420.926 ± 0.025
    UΔU_\Delta / MPtrj0.934 ± 0.0010.980 ± 0.0020.923 ± 0.0020.980 ± 0.002
    Deep ensemble / MPtrj0.9140.9720.9010.962
    ∣F^∣‾\overline{\lvert\hat{\mathbf F}\rvert} / MPtrj0.826 ± 0.0010.896 ± 0.0050.834 ± 0.0020.903 ± 0.005
Reviewer_B3FnOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Many MLIPs are trained to predict both energy and a non-conservative force. The paper proposes the simple idea that the discrepancy between the non-conservative force and a conservative force estimate obtained by differentiating the energy prediction can serve as a reliable, out-of-the-box uncertainty estimate. The experiments report that the stability of dynamical simulations depends more on energy conservation than on overall force-prediction accuracy, and they also show that the proposed discrepancy correlates with the error in the direct force prediction and the overall energy drift during dynamical simulations.

Critical strengths

The paper’s main idea is simple, the presentation is clear, and the results are interesting.

Critical weaknesses

I have concerns about the claims, mainly what the proposed metric measures, its novelty, and what the experiments really show.

Given that the conservative prediction can have a large error compared to a force head (despite them being conservative), it seems their delta better reflects the conservative force error rather than the force head error, which is what is used in practice. This is also supported by the experiments on the ammonia dataset, where the delta is about ten times the direct force error. For the same reason, the paper’s claim that the delta is like the “non-conservative component” of the direct force is not really correct. To be fair, in section 5 the paper does say that conservative error dominates the delta, but it doesn't consider the implications for the paper's claims.

The experiment's stability directly measures energy drift, which means (ignoring the integration error) any conservative force will be stable by definition, even if it is wildly wrong. What the experiment is really showing is that the direct force is non-conservative and therefore unstable. The idea of using the conservative force to monitor the MD already appears in Bigi et al. They also show that ORB behaved badly in simulations despite being more accurate than MACE. The paper’s lambda is not normalized (unlike Bigi et al.), so rigid structures, which have higher forces, would raise both delta and lambda, creating a correlation if one plots the correlation across materials as in Figure 2a. Instead, if one plots the correlation between lambda and delta for a single material, which is basically the way delta will be used, it seems like the correlation will be lower (assuming each clump in the figure is for one material). The same seems true for the correlations reported in Table 1. It makes sense to instead report correlations per system and then report statistics (mean, std, etc.) on those for the table, and use different colours for different materials in the plot. The statistics within each system will reflect how the proposed delta is used in MD, as they show ranking in the simulations for a specific system. Table 2 does not seem to have this problem, but it also does not seem to reflect an MD situation since the energy range is quite wide; for this table, it would be nice to include a simple baseline like force magnitude, which may also work well.

I’m open to adjusting my recommendation once the above issues are addressed.

Questions for authors

Please see the weaknesses for questions or experiments that would address the concerns.

8 comments
AuthorsOpenReview ↗

Replies to Reviewer B3Fn (Part 1)

We thank the reviewer for reading our work carefully and for making many constructive comments. Following the reviewer's suggestions, we added per-trajectory analyses on the submitted setting and three further architecture/dataset settings, and three additional UQ benchmarks with the suggested force-magnitude baseline. We address each point below.

Error ranking and uncertainty magnitude. The reviewer is right that the magnitude of UΔU_\Delta does not quantitatively predict the magnitude of the direct force error εnc\varepsilon_{\mathrm{nc}}. This is also true of most other UQ methods for MLIPs, including ensembles and distance-based GMM [5], which are not calibrated out of the box and require post-hoc calibration on additional data for quantitative error prediction. UQ methods are therefore commonly compared by how well they rank the error [4, 5], using the Spearman correlation we adopt as the main evaluation metric in this work. For the same reason, many active learning workflows use ranking-based acquisition, as our finetuning experiments do: the top-kk most uncertain structures in the pool are selected for labeling. This depends only on the ordering of the uncertainties, and what it requires, namely that higher uncertainty identifies higher-error structures, is exactly what the Spearman correlation measures.

Across our five UQ benchmarks, UΔU_\Delta ranks εnc\varepsilon_{\mathrm{nc}} well, with ρ\rho consistently in the range 0.8–0.9. This may be explained by the two force errors rising together as the model enters the OOD regime, since both heads read the same learned representation but diverge in different directions because each force is computed differently, as discussed in Section 5. Our active learning experiments further show that this ranking capability translates into model improvement: at a fixed DFT labeling budget, finetuning on the highest-UΔU_\Delta frames from an MD trajectory reduces energy drift by more than an order of magnitude relative to uniform MD sampling, while nearly halving the held-out error.

Energy drift over 50 ps (eV/atom) and held-out liquid-water test set εnc\varepsilon_{\mathrm{nc}} (meV/Å):

ModelAdversarial attackMD-sampledTop-UΔU_\Delta selection
Pretrained1.0090 ± 0.1353 / 40.8samesame
Gen 1, F^nc\hat{\mathbf F}_{\mathrm{nc}}0.2216 ± 0.0418 / 35.61 ± 0.320.7976 ± 0.2755 / 39.47 ± 0.190.0674 ± 0.0031 / 22.73 ± 0.07
Gen 1, F^nc\hat{\mathbf F}_{\mathrm{nc}} & F^c\hat{\mathbf F}_{\mathrm c}0.1531 ± 0.0329 / 31.82 ± 0.240.6781 ± 0.2138 / 35.85 ± 0.130.0504 ± 0.0018 / 21.64 ± 0.11
Gen 2, F^nc\hat{\mathbf F}_{\mathrm{nc}} & F^c\hat{\mathbf F}_{\mathrm c}0.0467 ± 0.0059 / 25.43 ± 0.180.4691 ± 0.1437 / 31.68 ± 0.250.0362 ± 0.0038 / 21.36 ± 0.04

Quantitative error prediction remains a limitation of UΔU_\Delta, as with most UQ methods, and we state this in Section 5. Post-hoc recalibration can be applied when a calibrated magnitude is needed, provided that the metric already ranks the error well, which is what we evaluate in this work. We thank the reviewer for raising this and have expanded the discussion of calibration in the revision.

[1] F. Bigi, M. F. Langer, and M. Ceriotti. The dark side of the forces: assessing non-conservative force models for atomistic machine learning. In ICML, PMLR 267, 2025.

[2] D. Zhang, H. Bi, F.-Z. Dai, W. Jiang, X. Liu, L. Zhang, and H. Wang. Pretraining of attention-based deep learning potential model for molecular simulation. npj Computational Materials, 10:94, 2024.

[3] L. Barroso-Luque, M. Shuaibi, X. Fu, B. M. Wood, M. Dzamba, M. Gao, A. Rizvi, C. L. Zitnick, and Z. W. Ulissi. Open Materials 2024 (OMat24) inorganic materials dataset and models. arXiv:2410.12771, 2024.

[4] A. R. Tan, S. Urata, S. Goldman, J. C. B. Dietschreit, and R. Gómez-Bombarelli. Single-model uncertainty quantification in neural network potentials does not consistently outperform model ensembles. npj Computational Materials, 9:225, 2023.

[5] K. Tran, W. Neiswanger, J. Yoon, Q. Zhang, E. Xing, and Z. W. Ulissi. Methods for comparing uncertainty quantifications for material property predictions. Machine Learning: Science and Technology, 1:025006, 2020.

AuthorsOpenReview ↗

Replies to Reviewer B3Fn (Part 2)

Relation to Bigi et al., and the stability claim in Section 4.1. We agree with and thank the reviewer for bringing up both points.

Bigi et al. [1] do compute the difference between the two forces, as the slow force in their multiple-time-stepping (MTS) scheme to stabilize direct-force dynamics. That use assumes a hybrid model in which both forces are supervised, which at foundation scale means training the conservative force on the full large-scale foundation datasets, and the quantity serves as a correction to unstable dynamics rather than as a measure for monitoring. The novelty of our work is the use and evaluation of UΔU_\Delta as a per-structure signal available from standard direct-force training alone. To our knowledge, this is the first work to introduce and evaluate it as an out-of-the-box metric for direct-force models, showing across multiple settings that it (i) tracks non-conservativity and stability at one instead of the 3N3N backward passes required by the exact Jacobian-based measure λ\lambda, (ii) ranks force error as an uncertainty metric with no training modification, additional architecture, post-hoc fitting or access to the training data, and (iii) generates or selects structures as an acquisition function for data-efficient finetuning that improves both energy drift and force accuracy, without training both forces on the full training set as MTS requires. Bigi et al. [1] never introduce or evaluate UΔU_\Delta in any of these roles. We thank the reviewer again for raising this point and have acknowledged the connection and positioned our work against it explicitly in the revision.

On Section 4.1, we agree that a gradient field conserves energy under symplectic integration regardless of its accuracy, and that the finding is established in prior work [1, 2]. We have accordingly reframed Section 4.1 as a confirmation of this finding, which our setting extends: here F^c\hat{\mathbf F}_{\mathrm c} is never trained and its error is large, unlike the trained F^c\hat{\mathbf F}_{\mathrm c} in [1, 2], so exact conservativity matters more than static accuracy for energy conservation even when the conservative force is substantially less accurate. Section 4.1 has also been presented as the motivation for UΔU_\Delta as a cheap proxy for non-conservativity and stability in the revised version, in place of λ\lambda.

[1] F. Bigi, M. F. Langer, and M. Ceriotti. The dark side of the forces: assessing non-conservative force models for atomistic machine learning. In ICML, PMLR 267, 2025.

[2] D. Zhang, H. Bi, F.-Z. Dai, W. Jiang, X. Liu, L. Zhang, and H. Wang. Pretraining of attention-based deep learning potential model for molecular simulation. npj Computational Materials, 10:94, 2024.

[3] L. Barroso-Luque, M. Shuaibi, X. Fu, B. M. Wood, M. Dzamba, M. Gao, A. Rizvi, C. L. Zitnick, and Z. W. Ulissi. Open Materials 2024 (OMat24) inorganic materials dataset and models. arXiv:2410.12771, 2024.

[4] A. R. Tan, S. Urata, S. Goldman, J. C. B. Dietschreit, and R. Gómez-Bombarelli. Single-model uncertainty quantification in neural network potentials does not consistently outperform model ensembles. npj Computational Materials, 9:225, 2023.

[5] K. Tran, W. Neiswanger, J. Yoon, Q. Zhang, E. Xing, and Z. W. Ulissi. Methods for comparing uncertainty quantifications for material property predictions. Machine Learning: Science and Technology, 1:025006, 2020.

AuthorsOpenReview ↗

Replies to Reviewer B3Fn (Part 3)

Per-trajectory correlations. We agree with the reviewer, and with Reviewer cLka, that UΔU_\Delta is not exactly the magnitude of the non-conservative component of the direct force, since F^c\hat{\mathbf F}_{\mathrm c} is not the conservative projection of F^nc\hat{\mathbf F}_{\mathrm{nc}}. In the revision we have described UΔU_\Delta as the disagreement between the two force predictions and based its use on the measured correlations rather than on that decomposition.

The reviewer is also right that pooling across materials can create correlation through differences in stiffness and force scale, and that a per-system analysis reflects how UΔU_\Delta is used during a simulation of one material. Following this suggestion, we recomputed every correlation within each trajectory and summarize the per-trajectory values by their median and interquartile range, so that these values carry none of the between-material scale differences the reviewer identifies. We also extended the analysis from one model family on one dataset to four architecture/dataset settings covering non-equivariant Orb and equivariant EquiformerV2 on MPtrj and OMat24 [3], extended the NVE simulations from 5 ps to 50 ps for consistency throughout the paper as suggested by Reviewer bi7z, and ran DFT on 100 evenly sampled frames along each trajectory to track the true errors of both forces.

Spearman ρ\rho, median [IQR] over 50 trajectoriesorb-inf-mpa / MPtrjorb-inf-omat / OMat24eqV2-31M-mp / MPtrjeqV2-31M-omat / OMat24
ρ(UΔ,λ)\rho(U_\Delta, \lambda)0.72 [0.59, 0.86]0.67 [0.48, 0.85]0.79 [0.65, 0.94]0.83 [0.67, 0.95]
ρ(UΔ,energy drift rate)\rho(U_\Delta, \text{energy drift rate})0.75 [0.52, 0.84]0.71 [0.53, 0.89]0.84 [0.61, 0.95]0.87 [0.69, 0.92]
ρ(UΔ,εnc)\rho(U_\Delta, \varepsilon_{\mathrm{nc}})0.83 [0.70, 0.92]0.79 [0.67, 0.91]0.83 [0.68, 0.94]0.77 [0.64, 0.92]
ρ(UΔ,εc)\rho(U_\Delta, \varepsilon_{\mathrm c})0.91 [0.76, 0.95]0.88 [0.81, 0.94]0.93 [0.84, 0.98]0.90 [0.78, 0.95]
ρ(λ,εnc)\rho(\lambda, \varepsilon_{\mathrm{nc}})0.66 [0.55, 0.82]0.62 [0.47, 0.81]0.81 [0.68, 0.90]0.75 [0.57, 0.81]
ρ(λ,εc)\rho(\lambda, \varepsilon_{\mathrm c})0.70 [0.57, 0.84]0.69 [0.53, 0.81]0.73 [0.59, 0.89]0.77 [0.63, 0.82]
ρ(εc,εnc)\rho(\varepsilon_{\mathrm c}, \varepsilon_{\mathrm{nc}})0.81 [0.64, 0.89]0.77 [0.63, 0.92]0.84 [0.67, 0.92]0.72 [0.61, 0.90]

The within-trajectory values are lower than the pooled ones, as the reviewer anticipated, but UΔU_\Delta continues to track λ\lambda, the drift rate and both force errors inside a single material, in every setting tested. Within an individual trajectory, non-conservativity and the two force errors move together with energy drift, and UΔU_\Delta captures all of them at one backward pass, with no DFT labels and without the 3N3N passes λ\lambda requires. That λ\lambda and εnc\varepsilon_{\mathrm{nc}} move together may be attributed to εnc\varepsilon_{\mathrm{nc}} measuring the distance between F^nc\hat{\mathbf F}_{\mathrm{nc}} and the DFT forces, which are themselves exactly conservative: the lower the error, the closer the direct force is to the exact gradient field of the DFT energy, and the lower the non-conservativity. The two force errors also correlate with each other here, in the adversarial attack trajectories and in the UQ benchmarks, which is why UΔU_\Delta tracks both well through the disagreement of the two forces, and may be explained by the shared representation discussed in Section 5. This also explains the active learning results above, where finetuning on high-UΔU_\Delta structures improves stability and accuracy together, and considerably more than the MD baseline. UΔU_\Delta can therefore be interpreted as a practical joint indicator of per-material non-conservativity and force error within an MD trajectory, without DFT labels or the expensive Jacobian-based λ\lambda. We have accordingly updated Section 4.2 with this finding in the revision.

On the per-system reporting of the other experiments: Table 1 evaluates pretrained foundation potentials, which are trained and mostly used across many systems, so a multi-system test set reflects their intended use, and the new OMat24 benchmark below spans 500 structures across 82 elements. Per-system evaluation, which the reviewer rightly identifies as the way UΔU_\Delta is used in practice, is covered by multiple other experiments: the MD trajectories above (one material each; 100 systems in total after the rebuttal experiments), the ammonia and SiO2 benchmarks (each a single chemical system), every adversarial attack path (one system per path, 71 paths in total after the rebuttal experiments), and the NVE frames collected during energy drift in Table 4 (one system per row).

Reviewer_cLkaOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes the Force Delta UΔU_\Delta, the magnitude of the difference between the direct force and the energy-gradient force of a direct-force MLIP, as an out-of-the-box signal for both MD instability and force error. The authors argue that MD stability depends more on conservativity than on force accuracy, and that UΔU_\Delta tracks both a Jacobian-based non-conservativity measure and energy drift. They then evaluate UΔU_\Delta as a UQ metric on pretrained Orb and EquiformerV2 models, adversarial OOD structures, and a small benchmark trained from scratch, reporting performance competitive with ensembles and other single-model methods. Finally, they use UΔU_\Delta-maximized structures for finetuning and report reduced drift and force error.

Critical strengths
  • The proposed methods is simple and easy to apply to released models. UΔU_\Delta can be evaluated with no change in training and access to training data, so it can be computed for novel materials without labels.
Critical weaknesses
  • The proposed measure is not new, and the paper does not position itself against its existing uses. The difference between direct and conservative forces is already computed explicitly in the MTS scheme of Bigi et al. [1], and force–energy-gradient consistency is already used as a diagnostic for direct-force models [2, 3]. The paper presents UΔU_\Delta as a newly introduced quantity and "the first" out-of-the-box UQ metric without noticing this connection. The actual contribution reduces to an empirical evaluation of an existing quantity as a UQ signal. That places the full burden on the experiments, which do not carry it.
  • The interpretation of UΔU_\Delta as non-conservativity is incorrect and contradicted by the paper's own results. The conservative force is not the conservative projection of the direct force. So UΔU_\Delta mixes the non-conservative part of the direct force with the error of the unsupervised conservative force. The paper's own analysis indicates that the latter dominates. The stability claim in Sec. 4.1 also holds by construction, since any gradient field conserves energy under symplectic integration regardless of its accuracy [4]. The causal conclusion that instability arises from non-conservativity rests only on correlations pooled across systems. As a result, the first two stated contributions are not supported as claimed.
  • The evidence does not support the generality claimed for any contribution.
    • The stability and non-conservativity results rely essentially on a single model family.
    • For the only equivariant direct-force head, the correlation with direct-force error is weak on most adversarial OOD systems.
    • The OOD evaluation is circular, because structures are generated by maximizing UΔU_\Delta itself.
    • The finetuning result has no equal-budget baseline with randomly or MD-sampled structures, so the gain cannot be attributed to UΔU_\Deltabased selection.

[1] F. Bigi, M. F. Langer, and M. Ceriotti. The dark side of the forces: assessing non-conservative force models for atomistic machine learning. In Proceedings of the 42nd International Conference on Machine Learning (ICML), PMLR 267, 2025.

[2] X. Fu, B. M. Wood, L. Barroso-Luque, D. S. Levine, M. Gao, M. Dzamba, and C. L. Zitnick. Learning smooth and expressive interatomic potentials for physical property prediction. In Proceedings of the 42nd International Conference on Machine Learning (ICML), PMLR 267, 2025.

[3] Y. Chiang, T. Kreiman, C. Zhang, M. C. Kuner, E. Weaver, I. Amin, H. Park, Y. Lim, J. Kim, D. Chrzan, A. Walsh, S. M. Blau, M. Asta, and A. S. Krishnapriyan. MLIP Arena: Advancing fairness and transparency in machine learning interatomic potentials via an open, accessible benchmark platform. In Advances in Neural Information Processing Systems (NeurIPS) Datasets and Benchmarks Track, 2025.

[4] E. Hairer, C. Lubich, and G. Wanner. Geometric Numerical Integration: Structure-Preserving Algorithms for Ordinary Differential Equations. Springer, 2nd edition, 2006.

Questions for authors

On W1

  • How does UΔU_\Delta differ, conceptually and practically, from the force difference already computed in the MTS scheme of Bigi et al. (2025) and from existing force–energy consistency diagnostics? What is claimed as new?

On W2

  • Can you separate how much of UΔU_\Delta comes from the non-conservative component of the direct force versus the error of the conservative force? For example, does UΔU_\Delta still correlate with λ\lambda after controlling for the conservative-force error?
  • Do the correlations with λ\lambda and energy drift hold within individual trajectories, rather than only when pooled across systems?
  • Can you show a setting where accuracy and conservativity vary independently, so that the stability claim is tested rather than holding by construction?

On W3

  • Does UΔU_\Delta remain competitive with ensembles on OOD structures that were not generated by maximizing UΔU_\Delta, evaluated on the same structures for all methods?
  • Do the stability, non-conservativity, and UQ results hold for other direct-force families, in particular for equivariant direct-force heads?
  • Does finetuning on UΔU_\Deltaselected structures outperform finetuning on the same number of randomly or MD-sampled structures, across multiple seeds?
9 comments
AuthorsOpenReview ↗

Replies to Reviewer cLka (W1/Q1)

We thank the reviewer for carefully reading our work and for making many constructive and valuable comments, which have considerably strengthened both the positioning and the generality of our results that support the UΔU_\Delta utility claims in the original submission. We address each point below.

Novelty and relation to prior work

We thank the reviewer for pointing out these related works and agree that the difference between direct and conservative forces has appeared before. Bigi et al. [1] use it as the slow force in a multiple-time-stepping (MTS) integrator to stabilize dynamics driven by F^nc\hat{\mathbf F}_{\mathrm{nc}}; this requires a hybrid model with both forces supervised on the full training set, and the difference serves as a correction rather than as a measure. Chiang et al. [2] report a related but different quantity, the deviation between the bond-projected force and a finite-difference energy derivative, as one benchmark score per model on homonuclear diatomic curves. We could not find the quantity defined or used in Fu et al. [3]; if we have missed it, we would be grateful for a pointer and will position against it accordingly.

What we claim as new is the use and evaluation of UΔU_\Delta as a per-structure signal available from standard direct-force training alone. To our knowledge, this is the first work to introduce and evaluate it as an out-of-the-box metric for direct-force models, showing across multiple settings that it (i) tracks non-conservativity and energy drift at one instead of the 3N3N backward passes required by the exact Jacobian-based measure λ\lambda, (ii) ranks force error as an uncertainty metric with no training modification, additional architecture, post-hoc fitting or access to the training data, and (iii) generates or selects structures as an acquisition function for data-efficient finetuning that improves both stability and force accuracy, without training both forces on the full training set as MTS requires. Neither [1] nor [2] evaluates UΔU_\Delta in any of these roles. We thank the reviewer again for pointing us to these previous works and have revised our paper to acknowledge the connection and have positioned our work against them explicitly in the rebuttal version.

[1] F. Bigi, M. F. Langer, and M. Ceriotti. The dark side of the forces: assessing non-conservative force models for atomistic machine learning. In ICML, PMLR 267, 2025.

[2] Y. Chiang, T. Kreiman, C. Zhang, M. C. Kuner, E. Weaver, I. Amin, H. Park, Y. Lim, J. Kim, D. Chrzan, A. Walsh, S. M. Blau, M. Asta, and A. S. Krishnapriyan. MLIP Arena: Advancing fairness and transparency in machine learning interatomic potentials via an open, accessible benchmark platform. In NeurIPS Datasets and Benchmarks Track, 2025.

[3] X. Fu, B. M. Wood, L. Barroso-Luque, D. S. Levine, M. Gao, M. Dzamba, and C. L. Zitnick. Learning smooth and expressive interatomic potentials for physical property prediction. In ICML, PMLR 267, 2025.

AuthorsOpenReview ↗

Replies to Reviewer cLka (W2/Q2)

We agree with the reviewer that F^c\hat{\mathbf F}_{\mathrm c} is not the conservative projection of F^nc\hat{\mathbf F}_{\mathrm{nc}}, and thank the reviewer for pointing this out. In the revision we have described UΔU_\Delta as the disagreement between the two force predictions and based its use on the measured correlations rather than on that decomposition.

We also agree that any gradient field conserves energy under symplectic integration regardless of its accuracy, and that Bigi et al. [1] and Zhang et al. [4] have shown a conservative force giving more stable dynamics than a more accurate direct force. We have accordingly reframed Section 4.1 as a confirmation of this finding, which our setting extends: here F^c\hat{\mathbf F}_{\mathrm c} is never trained and its error is large, unlike the trained F^c\hat{\mathbf F}_{\mathrm c} in [1, 4], so exact conservativity matters more than static accuracy for energy conservation even when the conservative force is substantially less accurate. Section 4.1 have also been presented as the motivation for UΔU_\Delta as a cheap proxy for non-conservativity and drift, in place of λ\lambda in the revision.

The reviewer is also right that correlations pooled across systems may differ from within-trajectory values. We accordingly computed all correlations within each individual trajectory and extended the study from one model family and one dataset to four architecture/dataset settings, covering non-equivariant Orb and equivariant EquiformerV2 evaluated on MPtrj and OMat24 [5]. We also extended the NVE simulations from 5 ps to 50 ps for consistency throughout the paper, as suggested by Reviewer bi7z, and ran DFT on 100 evenly sampled frames along each trajectory to track the true errors of both forces.

Spearman ρ\rho, median [IQR] over 50 trajectoriesorb-inf-mpa / MPtrjorb-inf-omat / OMat24eqV2-31M-mp / MPtrjeqV2-31M-omat / OMat24
ρ(UΔ,λ)\rho(U_\Delta, \lambda)0.72 [0.59, 0.86]0.67 [0.48, 0.85]0.79 [0.65, 0.94]0.83 [0.67, 0.95]
ρ(UΔ,energy drift rate)\rho(U_\Delta, \text{energy drift rate})0.75 [0.52, 0.84]0.71 [0.53, 0.89]0.84 [0.61, 0.95]0.87 [0.69, 0.92]
ρ(UΔ,εnc)\rho(U_\Delta, \varepsilon_{\mathrm{nc}})0.83 [0.70, 0.92]0.79 [0.67, 0.91]0.83 [0.68, 0.94]0.77 [0.64, 0.92]
ρ(UΔ,εc)\rho(U_\Delta, \varepsilon_{\mathrm c})0.91 [0.76, 0.95]0.88 [0.81, 0.94]0.93 [0.84, 0.98]0.90 [0.78, 0.95]
ρ(λ,εnc)\rho(\lambda, \varepsilon_{\mathrm{nc}})0.66 [0.55, 0.82]0.62 [0.47, 0.81]0.81 [0.68, 0.90]0.75 [0.57, 0.81]
ρ(λ,εc)\rho(\lambda, \varepsilon_{\mathrm c})0.70 [0.57, 0.84]0.69 [0.53, 0.81]0.73 [0.59, 0.89]0.77 [0.63, 0.82]
ρ(εc,εnc)\rho(\varepsilon_{\mathrm c}, \varepsilon_{\mathrm{nc}})0.81 [0.64, 0.89]0.77 [0.63, 0.92]0.84 [0.67, 0.92]0.72 [0.61, 0.90]

UΔU_\Delta continues to track λ\lambda, the drift rate and both force errors within an F^nc\hat{\mathbf F}_{\mathrm{nc}}-driven trajectory in every setting, at one backward pass, with no DFT labels and without the 3N3N passes λ\lambda requires. The bottom three rows show that non-conservativity and the two force errors move together within a trajectory. That λ\lambda and εnc\varepsilon_{\mathrm{nc}} co-vary may be attributed to εnc\varepsilon_{\mathrm{nc}} measuring the distance between F^nc\hat{\mathbf F}_{\mathrm{nc}} and the DFT forces, which are themselves exactly conservative: the lower the error, the closer the direct force is to an exact gradient field, and the lower the non-conservativity. The two force errors also correlate with each other here, in the adversarial attack trajectories and in the other UQ benchmarks in this work, which is why UΔU_\Delta tracks both errors well through the disagreement of the two forces, and may be explained by the shared representation discussed in Section 5.

The partial correlation the reviewer asks for is correspondingly weak: ρ(UΔ,λ∣εc)=0.26\rho(U_\Delta, \lambda \mid \varepsilon_{\mathrm c}) = 0.26 for the eqV2-omat setting. This is expected, since accuracy and conservativity co-vary within F^nc\hat{\mathbf F}_{\mathrm{nc}}-driven dynamics and controlling for one removes most of the shared signal. UΔU_\Delta is therefore best read as a joint indicator of non-conservativity and force error, without DFT labels or the expensive Jacobian-based λ\lambda.

Our finding in Section 4.2 accordingly becomes: within F^nc\hat{\mathbf F}_{\mathrm{nc}}-driven dynamics, force error — which also measures the distance to an exact DFT gradient field — and non-conservativity move together with instability, and UΔU_\Delta captures all of them without expensive DFT labels or λ\lambda. This is what allows a single quantity to serve both as a monitor during simulation and as an acquisition function for active learning, as the finetuning results show. We thank the reviewer again for bringing up these points and have updated Sections 4.1 and 4.2 accordingly in the revision.

AuthorsOpenReview ↗

Replies to Reviewer cLka (W3/Q3 Part 1)

OOD structures not generated by maximizing UΔU_\Delta, and equivariant direct-force heads. Tables 1 and 2 of the original submission already evaluate UΔU_\Delta against ensembles and other UQ methods on the same structures, none generated by maximizing UΔU_\Delta: Table 1 on a held-out set spanning bulk solids, a surface with adsorbate and molecules, and Table 2 on the ammonia test set of Tan et al. [7], designed for the extrapolative OOD regime. To further strengthen generality, we added four UQ benchmarks on non-adversarial OOD settings and extended the adversarial study to seven additional models, covering non-equivariant (Orb) and equivariant (MACE, eqV2) architectures. UΔU_\Delta remains superior or competitive against ensembles and other UQ methods while being the only single-model method available out of the box from standard direct-force training, and its utility extends to hybrid models in which both forces are trained.

  • Pretrained foundation models, on an OMat24 [5] rattled subset spanning 82 elements as OOD test set. ρ(UΔ,εnc)\rho(U_\Delta, \varepsilon_{\mathrm{nc}}), averaged over models in each family:

    MethodOrb (5 models)EquiformerV2, MPtrj-trained (4 models)
    UΔU_\Delta0.84 ± 0.020.86 ± 0.01
    Heterogeneous ensemble UvarU_{\mathrm{var}}0.88 ± 0.040.87 ± 0.02
    Confidence head0.76 ± 0.01—
    MC dropout—0.72 ± 0.01
    ∣F^nc∣‾\overline{\lvert\hat{\mathbf F}_{\mathrm{nc}}\rvert} (baseline suggested by Reviewer B3Fn)0.73 ± 0.020.71 ± 0.02

    UΔU_\Delta outperforms both the confidence head and MC dropout, rivals ensembles while using a single model, and is the only single-model metric available for both families.

  • MACE with an equivariant direct-force head on ammonia. As suggested by Reviewer bi7z, we replaced the Orb-style MLP head with one that reads out the l=1l=1 features directly, so that F(Rx)=RF(x)\mathbf F(R\mathbf x) = R\mathbf F(\mathbf x) holds exactly (rotation violation 4×10−154 \times 10^{-15}, against 1.5×10−11.5 \times 10^{-1} for the MLP head). Every method improves from force equivariance; UΔU_\Delta remains competitive with the deep ensemble and outperforms GMM, while MVE remains unstable due to its hard-to-optimize specialized loss. Submitted results are included for comparison. ρ\rho (AUROC):

    MethodOrbMACE, equivariant F^nc\hat{\mathbf F}_{\mathrm{nc}}MACE, MLP head
    UΔU_\Delta0.832 ± 0.026 (0.969 ± 0.007)0.874 ± 0.031 (0.982 ± 0.002)0.861 ± 0.055 (0.966 ± 0.003)
    GMM0.819 ± 0.070 (0.962 ± 0.023)0.841 ± 0.042 (0.979 ± 0.006)0.898 ± 0.030 (0.982 ± 0.003)
    MVE0.692 ± 0.120 (0.897 ± 0.076)0.702 ± 0.157 (0.862 ± 0.124)0.675 ± 0.241 (0.791 ± 0.129)
    Deep ensemble0.825 (0.982)0.933 (0.987)0.873 (0.976)
    ∣F^nc∣‾\overline{\lvert\hat{\mathbf F}_{\mathrm{nc}}\rvert}0.723 ± 0.024 (0.891 ± 0.005)0.731 ± 0.072 (0.882 ± 0.100)0.773 ± 0.050 (0.902 ± 0.001)
  • Condensed-phase SiO2 [7], large periodic bulk structures whose OOD test set has high configurational complexity from high-deformation-rate trajectories. ρ\rho (AUROC):

    MethodOrbMACE, equivariant F^nc\hat{\mathbf F}_{\mathrm{nc}}MACE, MLP head
    UΔU_\Delta0.917 ± 0.012 (0.986 ± 0.006)0.935 ± 0.031 (0.975 ± 0.008)0.908 ± 0.029 (0.982 ± 0.004)
    GMM0.784 ± 0.030 (0.967 ± 0.010)0.775 ± 0.018 (0.914 ± 0.009)0.796 ± 0.022 (0.921 ± 0.012)
    MVE0.635 ± 0.183 (0.827 ± 0.125)0.712 ± 0.243 (0.851 ± 0.243)0.681 ± 0.103 (0.867 ± 0.098)
    Deep ensemble0.951 (0.990)0.953 (0.988)0.932 (0.985)
    ∣F^nc∣‾\overline{\lvert\hat{\mathbf F}_{\mathrm{nc}}\rvert}0.762 ± 0.014 (0.831 ± 0.003)0.744 ± 0.004 (0.865 ± 0.002)0.722 ± 0.012 (0.867 ± 0.003)
  • Multi-system training from scratch, MACE with the equivariant head trained on an MPtrj subset, OMat24 rattled OOD test set:

    Methodρ\rhoAUROC
    UΔU_\Delta0.864 ± 0.0230.898 ± 0.002
    Deep ensemble0.9020.926
    ∣F^nc∣‾\overline{\lvert\hat{\mathbf F}_{\mathrm{nc}}\rvert}0.718 ± 0.0320.810 ± 0.021

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.