acceptodds
Under review as a conference paper at ICLR 2027

Derivative-Aware Uncertainty Quantification for Equivariant MLIPs

Abstract

Uncertainty quantification (UQ) for machine-learned interatomic potentials (MLIPs) is essential considering potential system distribution shift and data scarcity. It faces unique challenges as MLIPs have to respect physics principles: energies must respect symmetry, forces are derivatives of energy, and posterior samples should ideally remain physically valid potentials. We propose derivative-aware subnetwork Laplace, a post-hoc Bayesian UQ framework for equivariant MLIPs. Starting from a pre-trained deterministic model, we select to Bayesianize a compact force-informed subnetwork by learning a Gaussian posterior which incorporates both energy and force Jacobians. The resulting global posterior induces coherent sampled energy surfaces and force fields, while preserving invariance, equivariance, and conservative structure when applied within an admissible equivariant parameterization. We evaluate DASL on controlled and molecular benchmarks under distribution shift, showing that derivative-aware posterior geometry improves force uncertainty without retraining or ensemble.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
uncertainty quantification, machine-learned interatomic potentials

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_Mjo7OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

DASL is a post-hoc Bayesian uncertainty method for pretrained equivariant machine-learned interatomic potentials (MLIPs). It keeps the trained network fixed and places a Gaussian Laplace posterior over a small subset of 64-512 weights. The posterior curvature uses both energy and force Jacobians. Weights are selected by their joint energy-force sensitivity on the training data, which reverses the max-variance rule of prior subnetwork Laplace (MaxVar). The uncertainty at a test input is the induced output variance. The method is tested on a 1D toy potential, on ammonia and 3BPA with a PaiNN model trained by the authors, and on the released MACE-OFF23-small model over five molecular splits. It is compared with last-layer Laplace, MaxVar and random subnetworks on the same frozen model, and with BLIP, deep ensembles and Bayes by Backprop. DASL has the highest force Spearman correlation between uncertainty and error on ammonia (0.905) and MACE (0.612), but not on 3BPA. My expertise is generative modeling and ML evaluation, not atomistic simulation, so I focus on the method and the evaluation.

Critical strengths
  1. Practical post-hoc setting with controlled comparisons. The pretrained predictor stays unchanged and no retraining is needed. All Laplace variants share the same frozen mean, so differences come only from the uncertainty method. The paper uses matched budgets, readout controls and budget ablations, and it is tested on a released pretrained MACE model.
  2. Informed selection beats random selection everywhere. DASL improves over random subnetworks on every benchmark and the paper is also open about where DASL does not win.
Critical weaknesses
  1. Gains over simple baselines are unclear and the evidence is thin. DASL has the highest Spearman on ammonia and MACE but loses on 3BPA at both temperatures, where the deep ensemble is best. The authors state that DASL vs MaxVar on MACE is not statistically significant. Ammonia and 3BPA are single runs with no confidence intervals. The evaluation data is small: 64 training and 129 test configurations for ammonia, and 64 adaptation and 128 test molecules per MACE split. NLL and CRPS are nearly identical across the Laplace variants because a large fixed noise floor dominates the predictive variance. On MACE, DASL’s force CRPS (0.0433) is slightly worse than the frozen model with noise only (0.0428). The 3BPA dihedral split is mentioned but not reported.
  2. Limited novelty and cost/usefulness are not shown. Symmetry preservation and energy-force consistency, presented as core properties, hold for any Laplace posterior over an equivariant energy model. So last-layer Laplace, MaxVar and random subnetworks have them as well. All subnetwork baselines in the experiments also use the joint energy-force curvature. Only the selection rule distinguishes DASL. Wall time and memory are not reported, although scoring all 694320 MACE weights needs force second derivatives. The motivation is MD trajectories drifting out of distribution, but no downstream use of the uncertainty is tested.
Questions for authors
  1. Report wall time and memory for subnetwork selection, posterior construction and prediction, vs. last-layer Laplace, MaxVar (SWAG) and the ensembles, especially for the 694320-parameter MACE model.
  2. Report 1) confidence intervals over seeds for ammonia and 3BPA 2) the 3BPA results on all four test sets, including the dihedral split 3) ammonia NLL and CRPS with a validation-selected noise floor, so the scores reflect the epistemic part.
  3. Can you show the uncertainty is useful downstream, for example in active learning or in detecting failures in an MD run that drifts out of distribution?
Reviewer_RvasOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper proposes using subnetwork Laplace approximation for post hoc uncertainty estimation in MLIPs. The authors motivate Laplace approximation by noting that it can be applied to an existing foundation model to produce force uncertainties that are both equivariant and compatible with energy uncertainties. The paper then proposes a method to select a subset of parameters, different from the original subnetwork Laplace, to make posterior estimation tractable. Empirical evaluation shows superiority to other variable selection methods for subnetwork Laplace and some baselines for uncertainty quantification of MLIPs.

Critical strengths

The presentation is well-structured, clear and polished.

The paper addresses an important problem and does a good job of motivating some of its design choices.

Critical weaknesses

Overall, the paper has a nice flow, with motivated derivation and a clear presentation. The main weakness is the empirical evaluation, which is limited compared to similar papers in this area in terms of baselines, scale, datasets, and metrics.

My main problem is with the paper’s empirical evaluation, which is quite limited. Given the range of Laplace-based methods in the past (see the list below), and since this is an application of an existing variant, the paper needs an extensive empirical analysis clearly showing some advantage.

While the motivation through the connection between conservative forces, symmetry and uncertainty is nicely explained here, it is not unique to this method; They hold automatically for any method that puts a distribution over the weights of an equivariant and conservative model (e.g., full, diagonal or last-layer Laplace, SWAG, or deep ensembles.)

The main novelty on the theory side is the variable selection method, but in my current understanding of it, the theoretical justification isn’t meaningful: when lambda (prior precision) is small relative to c, all variables will have a similar score – e.g., for the paper’s own toy problem it seems like all the scores for selected weights are all > .998 (am I correct?)

Weaknesses in empirical evaluation:

Baselines: Laplace approximation and its relatives appear in many prior works in MLIPs; for some examples, see the list below. The paper does not adequately contrast with such earlier works in either the presentation or the experiments. What distinguishes this work is producing full covariance over a small subset of weights. The experiments should show that this choice is meaningfully better than the alternatives, with the simplest examples being full-weight diagonal or KFAC Laplace. Other important related baselines include GP-based methods, last-layer ensembles, SWAG, GMMs (mentioned but missing from the table), or ensembles of existing pretrained models.

Metrics: training and inference costs (runtime and memory) are not reported, so it is difficult to assess whether the model is worth the cost and would remain practical at larger scales. For example, the posterior is constructed using fewer than 100 structures, several orders of magnitude fewer than in related work (e.g., BLIP).

Scale: given the motivation is post-hoc UQ for foundation models, the paper could apply its methodology to a large foundation model. Frontier foundation models for MLIPs are three to four orders of magnitude larger than the largest model used in the experiments.

Chong, S., Grasselli, F., Ben Mahmoud, C., Morrow, J. D., Deringer, V. L., & Ceriotti, M. (2023). Robustness of local predictions in atomistic machine learning models. Journal of Chemical Theory and Computation, 19, 8020–8031.

Bigi, F., Chong, S., Ceriotti, M., & Grasselli, F. (2024). A prediction rigidity formalism for low-cost uncertainties in trained neural networks. Machine Learning: Science and Technology, 5(4), 045018.

Chong, S., Bigi, F., Grasselli, F., Loche, P., Kellner, M., & Ceriotti, M. (2025). Prediction rigidities for data-driven chemistry. Faraday Discussions, 256, 322–344.

Mazitov, A., Bigi, F., Kellner, M., Pegolo, P., Tisi, D., Fraux, G., Pozdnyakov, S., Loche, P., & Ceriotti, M. (2025). PET-MAD as a lightweight universal interatomic potential for advanced materials modeling. Nature Communications, 16, 10653.

Ho, C. H., Ortner, C., & Wang, Y. (2026). Flexible uncertainty calibration for machine-learned interatomic potentials. npj Computational Materials; arXiv:2510.00721.

Nitchi, D., Levy, D., Côté, M., & Ravanbakhsh, S. (2026). Bayesian last layer for neural force fields. ICML 2026 Workshop on AI for Science.

Willow, S. Y., Park, T. H., Sim, G. B., et al. (2025). Bayesian E(3)-equivariant interatomic potential with iterative restratification of many-body message passing. arXiv:2510.03046.

Zaverkin, V., & Kästner, J. (2021). Exploration of transferable and uniformly accurate neural network interatomic potentials using optimal experimental design. Machine Learning: Science and Technology, 2, 035009.

Zaverkin, V., Holzmüller, D., Steinwart, I., & Kästner, J. (2022). Exploring chemical and conformational spaces by batch mode deep active learning. Digital Discovery, 1(5), 605–620.

Questions for authors

I don't have a critical question at the moment but please identify any misunderstanding in the list of weaknesses above.

Reviewer_2XpmOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes DASL (Derivative-Aware Subnetwork Laplace), a post-hoc Bayesian UQ method for pretrained equivariant MLIPs. It starts from a function-space formulation in which the uncertain potential is a GP E∼GP(mE,kE)E \sim \mathrm{GP}(m_E, k_E) whose covariance must be (i) symmetry-consistent (KΦ(g ⁣⋅ ⁣x,g ⁣⋅ ⁣x′)=TgKΦTg⊤K_\Phi(g\!\cdot\! x, g\!\cdot\! x') = T_g K_\Phi T_g^\top) and (ii) derivative-consistent, i.e., the energy–force covariance blocks are obtained by differentiating the energy kernel (F=−∇REF = -\nabla_R E). To realize these properties post hoc, DASL freezes the pretrained parameters θ⋆\theta_\star, selects a compact set SS of KK stochastic parameters, and places a full-covariance GGN–Laplace posterior q(θS)=N(θ⋆S,ΛS−1)q(\theta_S) = \mathcal N(\theta_{\star S}, \Lambda_S^{-1}) with ΛS=P0,S+CE,S+CF,S\Lambda_S = P_{0,S} + C_{E,S} + C_{F,S}, where both energy and force Jacobians enter the curvature. The subnetwork is selected by a minimum marginal-variance criterion (vj=(λ+cj)−1v_j = (\lambda + c_j)^{-1}, cjc_j the whitened joint energy–force curvature), reversing the usual MaxVar subnetwork rule; Proposition 1 shows this is equivalent (under the diagonal approximation) to maximizing the retained output-space variance, since tr(KS)=∑j∈Scjvj\mathrm{tr}(K_S) = \sum_{j\in S} c_j v_j is increasing in cjc_j. The linearized posterior is shown (App. B) to induce a finite-rank tangent kernel kE(x,x′)=JE,S(x)ΛS−1JE,S(x′)⊤k_E(x,x') = J_{E,S}(x)\Lambda_S^{-1}J_{E,S}(x')^\top of rank ≤∣S∣\le |S| that inherits invariance/equivariance and derivative consistency — provided the underlying model is invariant for every admissible parameter value. Validation: a 1-D controlled potential (energy-only vs energy+force curvature; subnetwork choice vs last-layer/MaxVar/BLIPs), the ammonia inversion benchmark (PaiNN trained on 64 configurations, 129-configuration OOD test), 3BPA at 300 K/600 K (PaiNN trained on 100 configurations), and a post-hoc evaluation on the released MACE-OFF23-small checkpoint (694K parameters, five molecule-disjoint splits, K=512K = 512). Results: DASL gives the best force uncertainty–error Spearman correlation on ammonia (0.905 vs 0.895 last-layer, 0.817 MaxVar, 0.863 random) and MACE-OFF23 (ρF=0.612\rho_F = 0.612 vs 0.601 MaxVar, 0.536 BBB, 0.439 BLIP), but is behind last-layer Laplace and MaxVar on 3BPA ranking (0.515 vs 0.598/0.586 at 300 K), has higher force CRPS than BLIP/BBB on MACE, and the authors themselves state the MACE difference from MaxVar is not statistically significant.

Critical strengths
  1. A clean, correct, and well-organized formulation of what physics-principled MLIP UQ should be. The two function-space requirements (symmetry-consistent, derivative-consistent covariance), the tangent-kernel interpretation of subnetwork selection, and the variance-reversal criterion give the field a useful organizing principle, and the theory in Appendix B is internally consistent (I checked Prop. 1 and the kernel properties). This is consequential because it turns "uncertainty should respect physics" from a slogan into a verifiable covariance condition.
  2. Unusually honest, controlled evaluation. The frozen-mean matched-budget design (Laplace variants share the exact predictive mean; KK-matched; random-selection control; readout-location controls in Table 8) isolates the uncertainty approximation from mean quality, and the paper explicitly reports where MinVar loses (3BPA) and where it is not significant (MACE). This makes the supported conclusions credible even though they are smaller than the abstract implies.
Critical weaknesses
  1. The headline mechanism — MinVar subnetwork selection — is only weakly, sometimes negatively, supported. On 3BPA DASL ranks worse than both last-layer Laplace and MaxVar (0.515 vs 0.598/0.586 at 300 K; 0.622 vs 0.642/0.637 at 600 K); on MACE it ties MaxVar within the stated significance; and on ammonia random selection reaches 0.863 against DASL's 0.905. The more robust ingredient — joint energy+force curvature (CE+CFC_E + C_F) — is close to standard practice (BLIP, ensembles also use forces). Consequence: the paper's distinctive contribution (the reversed selection criterion and its theory) does not demonstrably drive the reported gains, so the central claim of "derivative-aware geometry improves force uncertainty" is not convincingly established over a simpler recipe.
  2. The validation scope and the efficiency claim are under-supported for what the introduction promises. All benchmarks are small organic molecules trained from scratch on 64–100 configurations except the single MACE post-hoc run (694K parameters, 64 adaptation structures, one checkpoint); there are no crystals/periodic systems (e.g., OC20) despite the general MLIP framing, no larger foundation-potential scale, and — despite computational overhead being the stated motivation — no wall-clock, memory, or cost comparison versus ensembles, and no coverage/calibration analysis beyond nominal 90% coverage of 0.89 in Table 4. Additionally, the induced posterior is a finite-rank (≤K\le K), unimodal GP, and the authors' own toy result shows it narrowing near x=1.5x=1.5 despite large extrapolation error — so the symmetry-structured Gaussian is a local object whose OOD behavior is limited by construction.
Questions for authors
  1. (Clarification — symmetry of samples) The kernel inherits invariance only if the model is invariant for every admissible parameter value, but MinVar selection is by curvature only. Does the selected subnetwork SS ever contain symmetry-breaking parameters (e.g., energy biases, biases in the equivariant stream of PaiNN/MACE)? Can you provide a numerical check that posterior samples remain (approx.) invariant/equivariant under random rotations and translations, e.g., max⁡∣Eθ(g ⁣⋅ ⁣x)−Eθ(x)∣\max |E_\theta(g\!\cdot\!x) - E_\theta(x)| across samples?
  2. (Clarification — why a Gaussian, and is it adequate here) The weight-space Gaussian induces a finite-rank, unimodal GP in function space; your toy figure shows underconfidence near the sparse region (x≈1.5x \approx 1.5) despite large extrapolation error. (i) Is the Gaussian choice meant to be a statement about the rotation group SO(3), or purely a local Laplace choice given that symmetry is structural in the architecture? (ii) How do you assess multimodality — e.g., does the ranking metric ρF\rho_F or any coverage metric degrade relative to ensembles in regions where the true posterior may be multimodal?
  3. (Clarification — attribution of gains) Since MinVar selection loses on 3BPA, ties on MACE, and faces a strong random baseline on ammonia, can you ablate selection rule vs joint E+F curvature separately, and identify any setting where MinVar decisively beats MaxVar and last-layer at matched KK? If not, what is the minimal recipe that retains most of the reported benefit?
  4. (Clarification — efficiency) The introduction motivates DASL by UQ compute cost, but no cost numbers are reported. What are the wall-clock/memory costs of building J^\hat J (energy and force Jacobians) and ΛS\Lambda_S for the MACE checkpoint, and of inference (sampling θS\theta_S), versus a 5-member deep ensemble at equivalent accuracy?
  5. (Suggestion, future work) All benchmarks are molecules (ammonia, 3BPA, MACE-OFF23); would you extend to crystalline/periodic benchmarks (e.g., OC20, SPIKE) or to a larger foundation potential (e.g., MACE-MP / NequIP-10M scale)? This would materially strengthen the generality and scalability claims.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.