acceptodds
Under review as a conference paper at ICLR 2027

Risk-Controlled DFT Selection for Machine-Learned Interatomic Potentials

Abstract

Density functional theory (DFT) gives accurate energies and forces but is expensive. Pretrained machine-learned interatomic potentials (MLIPs) are widely used for their cheap costs, but their errors vary widely across molecules. So how do we choose which configurations to trust with MLIP and which to send to DFT? We cast this as risk-controlled selection and solve it with a conformal risk control (CRC) framework. A finite CRC rule turns any error score fixed before calibration into a routing decision, whose expected capped per-molecule error left to the MLIP is at most a user-chosen fraction of its fitting-set mean. Because every score receives the same guarantee, scores can be compared by the DFT cost or call. With this comparison across eight dataset–MLIP pairs, we find that committee disagreement often overlooks errors that the models share. We also show that an analytical-chemistry-informed lightweight error predictor ranks errors more accurately than existing error-score baselines. At it sends 7% and 14% fewer than two committee-based baselines while trailing by 3.4 percentage points from oracle. The advantage is emphasized when the user's DFT reference differs from the MLIPs' training reference and is less apparent when the references match. We also show that replacing only some MLIP energies with DFT can break error cancellation in energy differences, so guarantees on relative energies must be calibrated directly.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Machine Learning Interatomic Potentials, Conformal Risk Control, Density Functional Theory, Data selection

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_NQPLOpenReview ↗
Rating
2(+1)
Confidence
2
Informativeness
2(+1)
Presentation
2
Soundness
2(+1)
Read review
Paper summary

The paper applies conformal risk control (CRC) to MLIPs and proposes using it to inform routing to the ground-truth (typically KS-DFT) backend when the MLIP output is classified as unreliable. Starting from a fully trained MLIP, it is proposed to label a small subset of the targeted evaluation / distribution (by "test" I mean w.r.t the MLIP, not their classifiers / the data split seems intact) They use handcrafted /predefined molecular point cloud features as input to an MLP to predict the energy MAE, Euclidean mean force error, and maximum force error, from which an uncertainty-type score is assigned. They calibrate the decision boundary using CRC on a small hold-out set.

The experiments contain three organic chemistry datasets at three levels of theory...

  • COMP6: ωB97X/6-31G(d) (no dispersion correction / smaller basis than MLIPs)
  • QCML: PBE0 (non-range-separated, no dispersion /some non-gaussian basis set)
  • MACE-OFF23: ωB97M-D3(BJ)/def2-TZVPPD combined with three MLIPs trained on three different levels of theory (all range-separated hybrids with dispersion corrected / large gaussian basis sets):
  • MACE-OFF23(M): ωB97M-D3(BJ)/ def2-TZVPPD
  • AIMNet2: ωB97M-D3 / def2-TZVPP
  • AceFF-2: ωB97M-V / def2-TZVPP
Critical strengths

S1: The method is a lot cheaper than the compared baselines, albeit both are negligible compared to DFT.

  • The committees need 3 MLIP evaluations per configuration, the proposed method only 1 (Section 3.1).
Critical weaknesses

W1: In the most important scenario the proposed method underperforms compared to the baselines.

  • The "error" being routed is mostly a level-of-theory offset.
  • It seems wierd to ask a higher level of theory trained MLIP to predict lower quality energies.
    • All MLIPs considered are trained on large basis sets at a range seperated hybrid level of theory that includes dispersion correction.
    • The experiments on COMP6 essentially ask to match numerical basis set errors since it uses a much smaller basis set.
    • The experiments on QCML remove dispersion corrections and the range seperation evaluating on lower quality hybrid DFT data
    • The only remaining evaluations on MACE-OFF at matched theory show that the baseline is better
      • Figure 2 excludes exactly this scenario.
      • Figure 3a) baselines are better to equal at matched levels of theory
      • Figure 3b) baselines save more DFT calls.
      • Figure 22: the baselines achieve better relative energies with similar selection success.
      • Figure 23: at all values alpha, the baselines achieve better relative energy MAE at similar success rates.
  • Given the data, the numbers quoted in the abstract seem to be missleading, and should be clearly state that these improvements are limited to settings where the MLIPs are trained on a higher level of theory and asked to match lower fidelity data.

W2: The proposed routing metrics are miss-aligned with core observables of interest.

  • Appendix J Figure 23
  • Appendix K shows that the method fails to adapt to physical observables tolerances of interest.

W3: Baselines

  • Missing: Δ\Delta-learning or fine-tuning on the 40% fitting set, then routing what remains
  • The supervised committee baseline seems handicapped
    • Single seed-0 network with L1 loss, whereas the proposed score is a three seed ensemble with an additional ranking loss.
    • On QCML × MACE-OFF23 it gets G = -0.28 (G = share of the random-to-oracle DFT savings, 0 = random, 1 = oracle) and the native variant (committee of MACE-OFF23 S/M/L models) G = -1.07, identical to sending everything to DFT.
    • A log-transformed refit was excluded (see Q3).
Questions for authors

Q1: The prior work section is very thin.

Q2: Regarding the reference mismatch

  • Your gains come from reference mismatch, and you already label 40% at the target reference. Why not correct the MLIP first with Δ\Delta-learning or finetuning on the 40% fitting set?
  • Could you please repeat the experiments with exactly equal levels of theory of MLIPs and deployment, in for example a size extrapolation or chemical space extrapolation setting on a standard high fidelity dataset like OpenMolecules25?

Q3: I would be interested in the rationality behind the exclusion of some results mentioned in the supplementary material.

The "VERIFIED_RESULTS.md" file suggests that negative results might have been excluded from the final manuscript. "Transition1x campaigns [...] were excluded." How does the proposed model perform on transition state geometries? Also please provide the log1p results variants for the supervised committee baselines. Good scientific practice demands a complete reporting of all tested variant at least in the appendix.

7 comments

[W1/Q2] We appreciate the reviewer's concern, as it helped us rethink the entire scope and core principles of the paper. The reviewer is right on the main point, and we therefore changed the experiments and the claims accordingly.

1. What the error contains.

In all eight original pairs, the MLIP and the dataset used different DFT references, so the measured error mixed MLIP approximation error with differences between electronic-structure methods. Our composition-linear alignment removes per-element offsets but not geometry-dependent differences between methods. Table 6 now lists the training and evaluation reference of every pair.

2. Why is the mismatched setting still useful, and what is its new scope?

Practitioners often have a fixed reference (for example, to stay consistent with a specific dataset like ∇2\nabla^2DFT) while the best available pretrained MLIP was trained elsewhere. The question is when that MLIP can stand in for their reference. We agree this is not a test of uncertainty relative to the MLIP's own training reference.

The reviewer also noted that every MLIP was trained at a higher level than the data it had to match. The 16 pairs now include the reverse case: on OMol25-core (ω\omegaB97M-V/def2-TZVPD), AIMNet2 and ANI-2x were trained at lower or different levels (Table 6).

3. Matched references.

Following Q2, we evaluated OMol25 with UMA-S at the matched reference. We added strengthened native committees wherever a native ensemble exists, and we added Transition1x.

Matched settingOursNative committeeBetter
MACE-OFF23 ×\times MACE-OFF23(M)53.75 (GG 0.763)52.35 (GG 0.815)Committee
OMol25-core ×\times UMA-S41.54 (GG 0.832)39.03 (GG 0.899)Committee
COMP6 ×\times ANI-2x57.93 (GG 0.663)57.13 (GG 0.695)Committee
Transition1x ×\times ANI-2x42.13 (GG 0.867)40.42 (GG 0.911)Committee

DFT calls (% of pool) at α=0.30\alpha=0.30 (from Tables 15 and 16).

When the references match and a native ensemble exists, the native committee is better. This is clearly stated in the main text (Table 1, Section 4.1, Section 5). On OMol25 a latent-mean predictor is better still (37.91%, Table 15). Our score still closes most of the random-to-oracle gap in these settings (G=0.66G=0.66–0.830.83), so it remains useful when no native ensemble exists. On MACE-OFF23 it also beats the plain native disagreement score (GG 0.763 vs. 0.730). We state clearly throughout the manuscript when our structural predictor outperforms others, and the specific cases where committees or latent predictors outperform ours.

4. Size extrapolation (further discussed in Q2).

On OMol25 with UMA-S, fitting on molecules with ≤150\le 150 atoms and testing on molecules with >200>200 atoms, our predictor reaches G=0.802G=0.802, close to the native committee (G=0.819G=0.819) (Appendix E.4). A matched nominal reference does not imply unseen data, since UMA-S was trained on OMol25 (this is stated in the manuscript).

5. Correcting the MLIP first (Δ\Delta-learning, further discussed in Q2).

We trained kernel and neural energy corrections on the fitting set, then routed the corrected model with separate CRC calibration (Table 18, Appendix E.6).

PairUncorrectedKernelNeural
COMP6 / MACE-OFF23(M)60.2455.2154.24
QCML-6950-9 / MACE-OFF23(M)46.8742.5339.51
QCML-6950-9 / AceFF-253.2744.7942.82
∇2\nabla^2DFT-core / UMA-S61.1156.8855.56
OMol25-core / AIMNet257.5054.6053.45
OMol25-core / UMA-S (matched)41.5441.8241.61

DFT calls (% of pool) at α=0.30\alpha=0.30 (from Table 18).

Correction lowers DFT by 5.00 (kernel) and 6.68 (neural) points on average in the mismatched pairs, and gives no benefit on the matched control. This supports the reviewer's view that much of the correctable error comes from the reference mismatch. Correction and routing can work together: a corrected MLIP is another frozen model, and CRC routes its remaining errors with the same guarantee.

6. Mismatched references.

On the 16 mismatched pairs, our predictor and the fairly trained supervised committee need nearly the same DFT: 57.54% and 57.44% (Table 1). The advantage of our method is that it needs only one MLIP instead of three, and no model internals.

7. Chemical-space shift.

We also moved fitted predictors between datasets with target recalibration. Transfer is poor (G=−0.26G=-0.26 to 0.500.50 over 18 transfers, against 0.730.73–0.950.95 in domain; Appendix E.5). The predictor must be refit per pair, while the guarantee holds whenever calibration uses the target population.

[W1/Q2]-continued

Changes to the claims.

  • The old abstract's claim of 7% and 14% fewer DFT calls than committees is removed. The abstract now says that, on reference-mismatched pairs, our predictor matches a supervised three-MLIP committee (which was also improved and is to be mentioned in W3/Q3).
  • We no longer claim that our predictor outperforms committees or model-internal predictors in general.
  • The CRC guarantee is separated from score quality (Remark 1). When references match, a user can plug the native committee into the same rule and keep the same guarantee.
  • The relative-energy comparisons the reviewer cited (old Figures 22–23) are addressed in our response to W2. Error-based routing is no longer presented as a method for relative energies.

[W2]

We thank the reviewer for this criticism. It led to the main new contribution of the revision: a rule that controls the chemical observable directly.

1. Physical tolerances.

We now certify physical tolerances directly across all 16 pairs: total energy error above 1 kcal/mol (V1), per-atom energy error above 0.05 kcal/mol/atom (V2), and maximum force error above 0.10 eV/Å (V3). At 1% and 5% violation budgets our score needs 98.75–98.82% and 94.41–94.52% DFT, and even the oracle needs 96.02–96.44% and 91.18–91.56% (Table 17, Figure 11a). We show that strict per-geometry accuracy is expensive for every score, even for the oracle, and this is stated in Section 4.1.

2. Why error-based routing fails for relative energies.

Replacing only some energies with DFT breaks error cancellation and creates new violations in 0.8–5.9% of previously accurate conformer pairs (Figure 16, Appendix F.2.3). Absolute-error scores also predict selection failure poorly, because they mostly track the error shared by all conformers of a molecule, and that error cancels.

MLIPMLIP energy marginOur absolute-error score
UMA-S0.8680.564
AIMNet20.8540.540
ANI-2x0.7990.552

AUROC for predicting conformer-selection failure (0.5 is random); 8,000 held-out ∇2\nabla^2DFT molecules (from Table 26).

3. A new rule that controls the decision (Section 2.4).

We use the same fundamental CRC principle, but calibrate an energy window around the MLIP minimum, evaluate the candidates inside it with DFT, and return their DFT minimum. Widening the window only adds candidates, so the failure loss always decreases (Lemma 1). The guarantee applies directly, without the conservative monotone envelope used in the old Appendix K.

At a 1% failure budget, the window needs 4.6%, 12.6% and 27.2% DFT for UMA-S, AIMNet2 and ANI-2x, respectively. Meanwhile, calibrated top-kk needs 30.4–42.9%, and error-based routing needs 63.1–86.7% (Table 2, Figure 2). The observed failure rates are 0.96%, 1.03% and 0.59%. All policies, tolerances and budgets were fixed in a dated protocol before any held-out energy was read (Appendix G.1).

4. Reported limitations.

  • Cost grows with the error spread within a molecule (Figure 18c), and calibrated top-kk is cheaper on dense SPICE snapshots (Table 32).
  • The guarantee is an average over molecules, so molecules with more than 45 atoms tend to fail more often (2.15% for UMA-S, Table 34).
  • The rule controls the chosen conformer, not every pairwise energy. Controlling all pairs at a 5% budget still needs 73–94% DFT (Table 27).

In summary, we identified why error-based routing misses the observables, built a rule that targets the observable directly, and validated it on a pre-registered held-out set.

[W3/Q3]

We thank the reviewer for pointing this out. We also realized that the supervised committee was handicapped and that all excluded variants should be reported. We fixed the committee, restored all excluded results, and selected the strongest committee as the comparison for the entire paper.

1. Strengthening the supervised committee.

We retrained the supervised committee with the same fitting data, three seeds, architecture, ℓ1\ell_1-plus-ranking objective and training budget as our predictor. We also added logarithmic input preprocessing. On QCML-6950-9, single MLIP predictions reach energy errors of about 10810^8 eV, which collapsed the original network to a nearly constant score.

Supervised committee variantGGDFT at α=0.30\alpha=0.30
Raw inputs, 1 seed (original)−0.278-0.27886.90%
log⁡(1+d)\log(1+d) inputs0.93643.26%
Median-scaled log⁡(1+d/a)\log(1+d/a) inputs0.93743.26%
Fair: log inputs + our training setup0.97842.31%
Ours0.82046.87%

QCML-6950-9 ×\times MACE-OFF23(M) (from Table 14).

On this pair every log-input variant beats our predictor, and we report that.

2. All variants over 16 pairs (including the log⁡(1+d)\log(1+d) results).

Supervised committee variantMean GG (16 pairs)
Raw inputs, 1 seed (original)0.520
Log inputs, 1 seed0.633
Fair: log inputs + our training setup0.785
Ours0.822

From Figure 9 and Table 12. Per-pair values of the log-input variant are in Table 8.

The fair committee is now used as the main comparison. Against it, our predictor needs nearly the same DFT: 57.54% vs. 57.44% (Table 1). We therefore no longer claim an advantage over the committee, as the two are statistically tied. However, ours uses one MLIP instead of three (about 136 fewer CPU core-hours per pair, Appendix E.7). The native MACE-OFF23 S/M/L committee is also reported with strengthened training (Table 15).

3. Δ\Delta-learning baseline.

Correcting the MLIP on the fitting set and then routing lowers DFT by 5.00–6.68 points in mismatched pairs, and does not help on the matched control (Table 18). Details are in our response to W1/Q2.

4. Transition1x was restored.

We added the Transition1x experiments: 19,930 configurations from 1,993 reactions (Table 16, Appendix E.4).

MLIPOursFair sup. committeeNative ensemble
ANI-2x0.867 / 42.130.669 / 51.870.911 / 40.42
AIMNet20.864 / 63.520.624 / 66.93–
MACE-OFF23(M)0.917 / 51.330.894 / 51.61–
AceFF-20.821 / 59.590.636 / 63.47–

GG / DFT (%) at α=0.30\alpha=0.30 (from Table 16).

Our predictor beats the supervised committee for all four MLIPs. The native ANI-2x ensemble is better than ours, and we state this in Section 5. These configurations come from reaction paths but are not Hessian-confirmed transition states, and some large force errors remain among the kept configurations. We therefore make no claim about transition-state or barrier accuracy.

5. Additional scores.

Beyond the requested baselines, we added nine error scores, including gradient, rigidity, latent and PROBE-style scores (Table 12; please see our response to Q1). Two scores that use MLIP internals rank slightly better than ours, and we report them as better choices when internals are available.

6. Complete reporting.

The appendix now contains the result tables behind every reported number, including all baseline variants and the restored experiments.

Reviewer_a6RhOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This work proposes a framework for deciding when to use a trained machine learning interatomic potential (MLIP) and when to use DFT. A structure-based error predictor ranks configurations, while conformal risk control calibrates a selection threshold to bound the expected capped loss combining energy and force errors, configuration judged unreliable are routed to DFT. Experiemnts show that the proposed socoring method can reduce DFT usage.

Critical strengths
  • The paper is well organized and clearly describes the components of the proposed framework. This makes the methodological choices and underlying assumptions easier to follow and evaluate.
  • The work addresses an important practical problem: reducing DFT costs while controlling prediction error. This gives the study practical relevance to atomistic modeling, where DFT often constitute a major computational bottleneck.
Critical weaknesses
  • The controlled loss seems to have no direct chemical meaning. The authors proposed an expected, normalized, and capped composite loss, rather than a fixed physical accuracy requirement or the reliability of retained predictions. Although the appendices examine more direct physical targets, stricter tolerances can substantially reduce the DFT savings.
  • The evaluation is restricted to energy and force predictions on fixed configurations, without testing geometry optimization or MD. It therefore remains unclear whether the proposed method can deliver comparable accuracy and computational savings in practical atomistic simulations.
Questions for authors
  • If I understand correctly, each new chemical population-MLIP combination requires appropriate DFT-labelled calibration data, which may limit the practical savings when applying the method to new chemical systems. Could the authors clarify how the risk-budget fraction α\alpha should be chosen in practice, what motivated the highlighted value of 0.3, and how this choice relates to a desired physical accuracy
  • Many of the chemcially meaning energies are relative energies, for example reaction energies. If different states are assigned to different evaluators with different MLIP/DFT calcualtions, this could trigger addtional source of error.
  • Although the present work focuses on DFT routing with frozen pretrained MLIPs, it would be interesting to investigate whether the proposed score could also support data acquisition for MLIP fine-tuning or active learning.
5 comments

[W1] We appreciate the reviewer's concern. We agree that the current loss has no practical meaning, and we understand why the reviewer gave us a low informativeness score. Based on the reviewer's comments, we made the following changes to address this question and to improve the informativeness and overall soundness of the paper.

The routing loss of Section 2.3 is a normalized, capped composite of energy and force errors. Its budget ϵ0=αℓˉfit\epsilon_0=\alpha\bar\ell_{\mathrm{fit}} is a fraction of the mean error on the fitting set, not a physical tolerance. For these reasons, we made two changes.

1. Physical tolerances are now controlled directly.

Proposition 1 holds for any bounded loss that is monotone in the threshold (Remark 3), so physical tolerances receive the same guarantee with no change to the method. Across all 16 dataset–MLIP pairs, we certify three such losses (Table 17, Figure 11a, Appendix E.6):

  • V1: total energy error above 1 kcal/mol;
  • V2: per-atom energy error above 0.05 kcal/mol/atom;
  • V3: maximum atomic force error above 0.10 eV/Å.
LossBudgetOursRaw comm.Sup. comm.Oracle (test labels)
V11%98.8298.8898.9196.41
V15%94.5294.5894.5491.54
V21%98.8198.9098.9296.44
V25%94.5294.5994.5591.56
V31%98.7598.8498.8796.02
V35%94.4194.3594.4491.18

DFT calls (% of pool), mean over 16 pairs (from Table 17).

Even the oracle needs more than 96% DFT at a 1% budget. The cost therefore reflects how rarely pretrained MLIPs reach chemical accuracy on individual geometries, not the quality of our score. We agree that strict tolerances remove most of the savings, and this is now stated in the main text (Section 4.1).

2. A new rule whose loss is a chemical decision.

Many tasks do not need every geometry to be accurate. In particular, relative energy errors matter significantly more than absolute energy errors in practical chemical settings. We therefore chose the problem of selecting the lowest-energy conformer, which depends on relative energies, and errors shared by all conformers of a molecule cancel each other. On held-out ∇2\nabla^2DFT, these shared offsets (4.2–6.7 kcal/mol) are much larger than the spread within a molecule (0.59–1.10 kcal/mol) (Table 26).

We therefore added a second rule (Section 2.4, Proposition 2). It calibrates an energy window around the MLIP minimum, evaluates the candidates inside the window with DFT, and returns the one with the lowest DFT energy. Its loss has a direct chemical meaning: the probability that the returned conformer lies more than 1 kcal/mol above the best DFT candidate.

MLIPAll-MLIP failWindow failWindow DFTTop-kk DFTError-based DFT
UMA-S2.300.964.630.463.1
AIMNet26.661.0312.642.984.6
ANI-2x9.830.5927.242.986.7

Held-out ∇2\nabla^2DFT, 8,000 molecules, 1% failure budget; all values in % (from Table 2).

If the DFT energy of the returned conformer is also needed, the window costs 17.6–32.8% (Table 2).

The choice of loss therefore decides the cost. Requiring every MLIP prediction to be chemically accurate needs nearly all-DFT. Controlling the chemical decision directly gives a guaranteed result with a small fraction of the DFT. Going beyond reporting physical tolerances, we built this decision rule, and the paper is now updated and organized around this point (Figure 1, Table 4).

[W2] We appreciate the reviewer's concern that the current evaluation is applied to fixed configurations, and we also wondered whether this method can provide practical computational savings in MD simulations. We therefore added an MD study and a reaction-path study, along with end-to-end costs reported in the manuscript.

1. MLIP molecular-dynamics snapshots (Appendix E.8).

  • ANI-2x and AIMNet2 ran Langevin NVT at 300 and 600 K on 300 COMP6 and 300 SPICE molecules (0.5 fs step, 1 ps equilibration, 10 ps production).
  • One production snapshot per molecule, model and temperature was labeled with DFT at the source reference.
  • The descriptor, predictor, loss and CRC rule were unchanged, and the budget α=0.20\alpha=0.20 was fixed before any new label was read.
MLIPTTForce RMSE (eV/Å)DFT, oursDFT, randomRisk ratio
ANI-2x300 K0.10575.0%86.7%1.037
ANI-2x600 K0.12080.0%95.0%0.808
AIMNet2300 K0.19775.8%85.8%0.975
AIMNet2600 K0.20482.5%90.0%0.733

COMP6 MD snapshots, α=0.20\alpha=0.20 (from Table 22).

Our score sends 7.5–15.0 points fewer snapshots to DFT than random selection, and the realized risk ratio stays near or below 1. Against the native eight-member ANI-2x committee, averaged over all budgets, ours needs 73.4, 70.2 and 74.6% DFT, compared to 74.0, 71.6 and 77.2% for the committee. The three settings are: (1) calibrate and test at 300 K, (2) calibrate and test at 600 K, and (3) calibrate at 300 K and test at 600 K (Figure 14).

2. Stability over longer runs.

From the final NVT states of eight molecules, both MLIPs ran 50 ps of NVE. None of the 32 runs failed, the largest energy drift was 1.095 meV/atom, and DFT force errors at 1, 25 and 50 ps did not grow (Figure 13).

3. Off-equilibrium geometries.

On Transition1x reaction paths (19,930 configurations from 1,993 reactions), our score reaches G=0.821G=0.821–0.9170.917 across four MLIPs (Table 16).

4. Measured cost with setup (Appendix E.7).

One DFT calculation takes 11 s to 28 min, while our descriptor and predictor take under 0.02 s per configuration (Table 19).

Scenario (α=0.30\alpha=0.30, median cell)Speedup over all-DFT
New pool, rule already fitted1.62$\times$
One pool, counting fitting and calibration labels0.47$\times$
Four pools1.01$\times$
Ten pools1.30$\times$

From Table 20.

For conformer selection, which needs no fitted predictor, the window costs 30.0–52.6% of full DFT even for a single pool with calibration included (Table 35).

To fully answer the reviewer's concern, we tested a temperature shift, compared against the native ANI-2x committee, checked 50 ps NVE stability, and replaced call counts with measured wall time.

[Q1] The reviewer is correct that each population–MLIP pair needs its own labeled data. We now measure this cost instead of leaving it implicit, and then explain how to choose the budget.

1. Calibration data.

  • A predictor moved to a new population keeps only G=−0.26G=-0.26 to 0.500.50 over 18 transfers, against 0.730.73–0.950.95 in domain (Appendix E.5). Each pair therefore needs its own fitting and calibration labels.
  • The fitting set can be small. Over the eight UMA-S-committee cells, our predictor beats both committees once 5% of molecules are used for fitting (Figure 10, Table 10).
  • Setup labels pay back after about four comparable pools (Table 20). If the fitting and calibration molecules belong to the set the user must label anyway, routing still saves about 17% of all DFT (Table 21, case B1).
  • The conformer window (Section 2.4) has no fitted model and only needs calibration labels. It costs 30.0–52.6% of full DFT for a single pool and 7.2–29.8% per pool over ten pools (Table 35). We also note that at a 1% budget, it needs at least 99 calibration molecules (Appendix C.2).

2. Choosing the budget in practice.

The budget must be fixed before the calibration labels are used. The guarantee does not hold if it is tuned on calibration or test results (Remark 2). We thus recommend choosing the loss first, according to how the result will be used:

If the user needs…Use this lossThe budget means
Every kept prediction within a physical tolerancePhysical violation loss (Remark 3, Table 17)Allowed violation rate
A correct chemical decision, e.g. the lowest conformerSelection-failure loss (Section 2.4)Allowed failure probability
A relative trade-off between kept error and DFT costNormalized proxy with α\alphaFraction of fitting-set error left to the MLIP

For the proxy, the user may choose α\alpha by inspecting the DFT-versus-α\alpha curve on the fitting set alone. Please note that this does not affect the guarantee (Appendix D).

3. Why α=0.30\alpha=0.30?

We report every α\alpha from 0.05 to 0.50 (Figures 4–7). Our main comparison metric GG integrates over the whole grid, so no comparison depends solely on α=0.30\alpha=0.30. It is a mid-grid point used for tables. At this point, the sign of every paired DFT difference agrees with the ordering by GG in all 16 cells (Table 8). We rewrote this explanation more clearly in Appendix D.

4. Relation to physical accuracy.

α\alpha is relative to the fitting population's own mean error, so the same α\alpha corresponds to different physical accuracies in different datasets. We do not suggest reading it as a physical tolerance. When physical accuracy is the requirement, it should be controlled directly with the physical loss above, which uses the same code.

In addition, the new conformer-selection rule does not need a fitted predictor, which removes the fitting-label cost for that task.

Reviewer_GYy1OpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper combines structure-based error prediction with molecule-level conformal risk control (CRC) for selective DFT evaluation, assessing DFT savings under a controlled proxy risk across eight dataset–MLIP pairs.

Critical strengths
  1. Separating error ranking from molecule-level risk calibration enables different reliability scores to be compared by their DFT costs under the same risk guarantee.

  2. The broad ablation study and the finding that partial DFT replacement can disrupt error cancellation and reduce conformer-selection success provide useful insights for hybrid MLIP–DFT workflows.

Critical weaknesses
  1. The correspondence between the latent baselines and the closest prior methods remains unclear, making it difficult to assess the added value over existing error predictors coupled with the same CRC procedure.

2.The existing ablations do not directly establish whether predicting and combining three error channels improves ranking and DFT-routing efficiency over directly regressing the composite error score.

  1. The evaluation reports call counts and cost proxies rather than measured runtime savings that account for upfront labeling, training, and inference overhead.
Questions for authors
  1. Under matched data splits, labeling budgets, and controlled losses, what additional benefits in ranking quality or practical cost does the structure-based predictor offer over existing predictors such as P-DRUM coupled with the same molecule-level CRC?

  2. Could the authors compare joint three-channel prediction with direct regression of the composite error score, holding structural inputs, training budgets, and CRC settings fixed, to quantify this design’s contribution to DFT savings?

  3. Could the authors report end-to-end wall-clock speedups over an all-DFT baseline separately for deployment alone and for the full workflow including upfront labeling and training?

3 comments

As the authors have not submitted the rebuttal, I will keep the initial score.

Thank you for your patience, and we apologize for the delay. The additional experiments requested across the reviews took longer than we expected. We have now uploaded the revised manuscript and posted our point-by-point responses below.

[W1, Q1] We thank the reviewer for this point and agree the original text did not make the correspondence clear. We have revised Appendix E.3 accordingly.

Our latent baselines pool the target MLIP's per-atom features (mean, max, or both) and regress the error with the same model, objective, seeds, and fitting labels as our structural predictor. They represent post-hoc predictors built on model internal representations, but they are not P-DRUM, which applies per-atom heads before pooling. We therefore also ran P-DRUM itself (its MACE descriptors and published heads) inside our pipeline, on the two datasets, COMP6 and QCML-6950-9, with a MACE target. All scores use identical splits, alignment, normalization, and molecule-level CRC, so they share the same risk guarantee and differ only in the error score used for calibration. We report the normalized gain GG in the table (revised Table 13 in Appendix E.3) below.

ScoreNeedsCOMP6 / MACEQCML / MACE
Structural (ours)ZZ, coordinates0.8790.819
Latent meanMLIP internals0.8730.837
Latent maxMLIP internals0.6540.731
Latent mean + maxMLIP internals0.8490.843
P-DRUM, norm / normMLIP internals0.5450.058
P-DRUM, norm / vectorMLIP internals0.4330.415
P-DRUM, signed / normMLIP internals0.9380.068
P-DRUM, signed / vectorMLIP internals0.8070.446
Structural + latent meanboth0.9080.865

The published P-DRUM variants are each strong on at most one cell (e.g., signed/norm reaches 0.9380.938 on COMP6 but 0.0680.068 on QCML-6950-9), while the pooled latent predictors and ours are strong on both. Table 12 in Appendix E.3 extends the comparison to all 16 cells and to further scores under the same CRC (PROBE-, LLPR-, and gradient-based). Over all 16 cells, latent mean ranks errors better on average (G=0.859G=0.859 vs.\ 0.8220.822), while ours is better on all three COMP6 cells. On the two MACE cells, our structural predictor is within 0.0250.025 of the best pooled latent predictor on both cells and above all of them on COMP6 (0.8790.879 vs. at most 0.8730.873). It also avoids the instability of the published P-DRUM variants. It achieves this using only atomic numbers and coordinates, so it applies when internals are unavailable or the MLIP is replaced, and it needs a single MLIP evaluation rather than a committee.

On the two MACE cells, our structural predictor is within 0.020.02 of the best pooled latent predictor on both cells and above all of them on COMP6 (0.8790.879 vs.\ at most 0.8730.873). It also avoids the instability of the published P-DRUM variants: its weaker cell (0.8190.819) is well above the weaker cell of every P-DRUM variant (≤0.415\le 0.415). It achieves this using only atomic numbers and coordinates, so it applies when internals are unavailable or the MLIP is replaced, and it needs a single MLIP evaluation rather than a committee.

[W2, Q2] We thank the reviewer for requesting this ablation, which we have added to Appendix E.2 (revised Table 11) and summarize below. We held the structural inputs, network, seeds, training budget, and CRC rule fixed and varied only the predictor's target and training loss. The target is either the three normalized errors summed into the score (our default), the composite score itself, or the capped error min⁡{s,Bcfg}\min\{s,B_{\mathrm{cfg}}\} that CRC controls. The loss is either ℓ1\ell_1 with a pairwise ranking term (our default) or ℓ1\ell_1 alone.

PredictorTargetLossMean GGDFT (%) at α=0.3\alpha=0.3
Three heads (ours)3 errors, summedℓ1\ell_1 + ranking0.82357.54
Direct compositecomposite scoreℓ1\ell_1 + ranking0.82057.58
Direct compositecomposite scoreℓ1\ell_10.81857.65
Capped-loss scalarcapped errorℓ1\ell_1 + ranking0.82157.51

Means over the 16 primary cells.

Predicting the three errors separately and summing them gives the highest GG at a comparable DFT cost, but all variants differ by less than 0.0050.005 in GG and 0.120.12 percentage points in DFT. The three-head design therefore does not substantially improve routing efficiency. Its benefit is flexibility: a user can reweight the energy and force errors (wtw_t in Eq.~3) for a new application without refitting the predictor.

[W3, Q3] We agree that DFT call counts do not by themselves show savings in computation time. We have added Appendix E.7, which compares the time of running DFT on every configuration with the time of our workflow. We timed each step separately (Table 19): one DFT calculation takes 11 seconds to 28 minutes depending on the dataset, while every step our workflow adds (MLIP prediction, descriptor, error predictor, threshold calibration) takes at most 2~seconds per configuration or a few minutes per dataset. Total time is therefore set almost entirely by the number of DFT calculations.

As requested, Table 20 reports the speedup over running DFT on every configuration separately for deployment alone and for the full workflow, including labeling the fitting and calibration sets and training the predictor (median over 14 dataset--MLIP pairs at α=0.3\alpha=0.3):

ScopeSpeedup over all-DFT
Deployment only (predictor already trained and calibrated)1.62×
Full workflow, one pool0.47×
Full workflow, reused for 4 pools1.01×
Full workflow, reused for 10 pools1.30×

Deployment alone is 1.62 times faster, because DFT runs on only about 57% of the pool. Including the labeling of the fitting and calibration sets, a single pool is slower than running DFT on every configuration, because these sets are larger than the pool. The method therefore saves time only when the trained and calibrated predictor is reused across several pools that resemble the calibration data; we now state this limitation explicitly. Committee-based routing gives nearly the same values, since DFT dominates every workflow. These totals are built from the measured time of each step rather than from one end-to-end run, and OMol25 could not be timed.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.