acceptodds
Under review as a conference paper at ICLR 2027

ExFiT-Bench: What Transfers When Universal Interatomic Potentials are Fine-Tuned on Experiments?

Abstract

Universal machine learning interatomic potentials (uMLIPs) are now widely adopted to enable simulations across diverse materials, which makes their evaluation increasingly important. Most benchmarks measure agreement with density functional theory (DFT), while some compare properties predicted by pretrained models with experiment. When measurements are available, however, a potential can be fine-tuned to match them before further use. We introduce ExFiT-Bench to evaluate uMLIPs in this setting. The benchmark focuses on ensemble-derived properties, for which experimental measurements provide no labels for individual configurations. Under a shared protocol, we fine-tune six uMLIPs on five tasks spanning elastic constants, a phase transition and a liquid structure factor in titanium, diamond and MgO. We evaluate the target observable and transfer to held-out temperatures, another polymorph, other observables and compositions. Fine-tuning reduces the on-target error for every model on every task, but the pretrained ranking does not consistently carry over. The correction often transfers to new conditions of the target observable, but transfer across observables and compositions is less reliable and can reduce accuracy. We release the benchmark data, code and a leaderboard for evaluating new uMLIPs and fine-tuning methods.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
MLIP, machine learning potential, Fine-tuning, Experimental data, benchmark

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_hisWOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
3(+1)
Read review
Paper summary

The paper introduces a benchmark for fine-tuning universal machine learning interatomic potentials (uMLIPs) on experimental data. The benchmark consists of five tasks: titanium elastic constants, titanium hcp-bcc phase transition, titanium liquid structure factor, diamond elastic constants, and MgO elastic constants. The work finetunes six uMLIPs on the five tasks using differentiable trajectory reweighting (DiffTRe) then evaluates the models on the fitted observables and on other temperatures, polymorphs, properties, and compositions. The fine-tuned models consistently improve fitted observables while the transfer to other observables or compositions is inconsistent. The paper also finds that fine-tuning changes the ranking of pretrained models.

Critical strengths

Well Motivated: Most MLIP benchmarks evaluate pretrained models against DFT or experiment. Evaluating models after experimental fine-tuning is a meaningful extension.

Potential for standardization: Consolidating experiments from different works into a common framework provides reproducible, consistent, and fair comparison across potentials. The authors will provide code and benchmark data further supporting this.

Empirical evaluation across multiple potentials: Six models with different architectures and pretraining datasets are evaluated using a shared experimental fine-tuning procedure. The work examines model performance on the fine-tuning observable and other observables.

Critical weaknesses

Limited benchmark diversity. Although the paper presents five tasks, they involve only three compositions and three of the five tasks are elastic constants. A benchmark for uMLIPs, which are intended to represent a much broader range of elements and chemical environments, should include a more diverse set of compositions, material classes, and physical properties.

Limited novelty of the benchmark dataset. Three of the five tasks (diamond elastic constant, titanium elastic constants, and titanium hcp-bcc phase transition) reproduce previously published experimental MLIP fine-tuning applications.

Limited diversity of experimental fine-tuning methods. The benchmark uses one previously published experimental fine-tuning method. Including alternative fine-tuning strategies would strengthen the benchmark and determine whether the observed transferability, catastrophic forgetting, and changes in ranking depend on the fine-tuning method.

Questions for authors

Could the authors provide more materials, observables, and fine-tuning strategies to strengthen the benchmark for uMLIPs? In Section 3.2 when fine-tuning on diamond, the authors observe that corrections transfer to other cubic materials more reliably than the hexagonal one. Could the authors provide more examples to determine whether this observation depends on symmetry, chemistry or other factors? How much of the observed change in model ranking is due to the chosen strategy of fine-tuning instead of fine-tuning on experimental data in general? For all uMLIPs fine-tuned, can the authors report how the fine-tuning affects energy and force errors relative to the DFT datasets used to train the uMLIPs? Why are the regularization studies in Appendix A.5 only reported for the titanium elastic constant and for the MACE and SevenNet models?

4 comments

Thank you for the concrete and constructive requests. We quote each point and answer it below. The revised paper shows every change in blue, and its new Appendix C holds the new figures and tables.

W1, Q1.

W1. Limited benchmark diversity. Although the paper presents five tasks, they involve only three compositions and three of the five tasks are elastic constants. A benchmark for uMLIPs, which are intended to represent a much broader range of elements and chemical environments, should include a more diverse set of compositions, material classes, and physical properties.

Q1. Could the authors provide more materials, observables, and fine-tuning strategies to strengthen the benchmark for uMLIPs?

The benchmark's breadth lies in what each fine-tuned model is scored on, and we have now added five new compositions as held-out test targets. Each model fine-tuned on an elastic task is also scored on the phonons and the other compositions, and the titanium models on a held-out polymorph, the transition line and the liquid structure factor. We now also score the paper's models fine-tuned on the diamond, MgO and titanium elastic constants, with no retraining, on the metals Cu and Mg, the covalent Si and the ionic NaCl and ZnO, shown in Table A. We did not add a new fitted observable, because each one needs its own measurements and an MD estimator checked against them, and the five fitted tasks already span elastic, thermodynamic and structural observables. We also scored three more fine-tuning strategies on all six models on the titanium elastic task, as W3.2 below shows.

Table A. Transfer to five new held-out compositions. Rows group the 15 pairs of a training and a new composition by what they share, counted once per model. SRME is the symmetric relative mean error of the elastic constants against experiment. A change counts only beyond the measurement floor, the disagreement between independent experiments, plus two MD standard errors, with all three seeds agreeing in sign. A negative mean ΔSRME means the error on the new composition fell.
Shared with the training compositionPairsBetter / worse / within, over 6 models per pairMean ΔSRME
structure and bonding (diamond→Si, MgO→NaCl, Ti→Mg)39 / 2 / 7−0.017
structure only53 / 4 / 23+0.0003
bonding only (MgO→ZnO, Ti→Cu)22 / 2 / 8−0.002
neither52 / 6 / 22+0.006

Q2.

In Section 3.2 when fine-tuning on diamond, the authors observe that corrections transfer to other cubic materials more reliably than the hexagonal one. Could the authors provide more examples to determine whether this observation depends on symmetry, chemistry or other factors?

We added five compositions that separate these factors, and they show that neither symmetry nor chemistry alone explains the transfer, as corrections transfer most reliably when source and target share both crystal system and bonding. Sharing only the crystal system leaves the elastic-constant error unchanged on average, helping and hurting about equally often, while sharing both lowers the error in half of the cases and rarely raises it, as Table A shows. The three pairs that share both, diamond to Si, MgO to NaCl and titanium to Mg, share no element, so the transfer does not rest on chemistry. Section 3.2 now states this close-analogue rule in place of the symmetry explanation.

W2.

Limited novelty of the benchmark dataset. Three of the five tasks (diamond elastic constant, titanium elastic constants, and titanium hcp-bcc phase transition) reproduce previously published experimental MLIP fine-tuning applications.

We agree that three tasks reuse published applications, but this is deliberate, and the novelty lies in benchmarking uMLIPs on them, which the source studies did not do. Reuse brings checked measurements and MD estimators of each observable, so differences between uMLIPs reflect the models rather than the task, and it gives the system-specific potential of each source study as a baseline. Each source study trained one specialised model. The benchmark instead compares six uMLIPs under one protocol, scores them away from the fitted target, and adds MgO elastic constants and the liquid Ti structure factor as new tasks. We changed "we reproduce" to "we adopt the targets and protocols of".

W3.1.

Limited diversity of experimental fine-tuning methods. The benchmark uses one previously published experimental fine-tuning method.

The benchmark fixes the experimental targets and how each fine-tuned model is scored, not the fine-tuning method, so any method can enter the leaderboard by declaring its training data, loss and schedule. The reference results use differentiable trajectory reweighting, the method of the source studies, as one fixed protocol, and we tested three more strategies on top of it, as W3.2 below reports.

W3.2.

Including alternative fine-tuning strategies would strengthen the benchmark and determine whether the observed transferability, catastrophic forgetting, and changes in ranking depend on the fine-tuning method.

Transfer, forgetting and the change in ranking all depend on the strategy. The paper fine-tunes on the experimental loss alone, which we call EXP-ONLY, and we added three regularized strategies, DFT-REPLAY, PL-REPLAY and L2-SP, on all six models on the titanium elastic task. A strategy that fits the target less tends to keep the pretrained order more, as Table B shows. On average over the six models, the replay strategies raise the diamond and MgO errors, mostly through PET-MAD, while L2-SP leaves them almost unchanged. DFT-REPLAY also raises the force error on the pretraining data, as Q4 shows.

Table B. Pretrained against fine-tuned order under each strategy, titanium elastic task. Kendall τb\tau_b between the pretrained and fine-tuned ranks of the six models, with pairs closer than the measurement floor counted as ties and 95% intervals over seeds and simulation blocks. A τb\tau_b of +1 means an unchanged order. The titanium block leaves out ΔG, which only EXP-ONLY measured, so its EXP-ONLY value differs from the full titanium block with ΔG. The block of Table 4 of the revised paper adds diamond and MgO. The on-target SRME is the mean over the six models. DFT-REPLAY and PL-REPLAY replay titanium DFT structures under DFT labels or under labels from the pretrained model, and L2-SP penalizes distance from the pretrained weights. This is Table 20 of the revised paper.
strategyon-target SRMEτb\tau_b, titanium block [95% CI]τb\tau_b, block of Table 4 [95% CI]
pretrained0.213––
EXP-ONLY0.022+0.45 [+0.28, +0.93]+0.47 [+0.41, +0.60]
L2-SP0.055+0.41 [+0.33, +0.69]+0.60 [+0.47, +0.73]
DFT-REPLAY0.088+0.60 [+0.60, +0.89]+0.47 [+0.33, +0.60]
PL-REPLAY0.154+1.00 [+0.83, +1.00]+0.47 [+0.47, +0.60]

Q3.

How much of the observed change in model ranking is due to the chosen strategy of fine-tuning instead of fine-tuning on experimental data in general?

Changing the strategy reorders the models about as much as fine-tuning does. On average, the orders from two different strategies agree with a τb\tau_b of +0.53 on the block of Table 4, against +0.50 between the pretrained order and each strategy's order. The strengths come from MACE and SevenNet in Appendix A.5, so these runs show trade-offs, and tuning each strategy for the leaderboard is future work.

Q4.

For all uMLIPs fine-tuned, can the authors report how the fine-tuning affects energy and force errors relative to the DFT datasets used to train the uMLIPs?

The revised paper now reports the energy and force errors on each model's own pretraining data, and under EXP-ONLY both barely change for a typical model and task, as Figure 18 shows. The force error rises by a median 3.5% and at most 1.69 times over the 25 pairs of five models and five tasks, and the energy error by a median 6% and at most 1.68 times. We compare energies after removing a constant shift per element, which the experimental loss leaves free and which changes no force, stress or reaction energy. We exclude MatterSim because its pretraining data are not public. DFT-REPLAY, one of the strategies in W3.2, replays only a titanium DFT set and more than doubles the force error in MACE, while L2-SP leaves it almost unchanged.

Q5.

Why are the regularization studies in Appendix A.5 only reported for the titanium elastic constant and for the MACE and SevenNet models?

The regularization study is an analysis of trade-offs and not part of the benchmark ranking, which uses EXP-ONLY on all six models. We swept its strength on two models, MACE and SevenNet, and on the titanium elastic task, whose source study released DFT data to replay, as far as our compute budget allowed. We have now run the three strategies at those strengths on all six models, as W3.2 shows.

Thank you to the authors for the detailed response and additional experiments.

W1, Q1.

The authors have added five new held-out compositions, but no new fine-tuning tasks and all are evaluated on elastic constants. This effort works to address limitations of the benchmark diversity but its training-task diversity is still limited.

Q2.

The additional transfer experiments work to understand how corrections transfer between materials sharing crystal structure and bonding type. However, these conclusions are based on only three source-target material pairs which have similar local atomic environments or coordination. The results support a limited empirical trend on the tested materials, not an established relationship.

W2.

I agree with the authors that the benchmark is still important to the community even with limited novelty.

W3, Q3.

The results confirm the concern that model rankings depend on the fine-tuning strategy. Consequently, the reported rankings characterize how uMLIPs perform under a particular experimental fine-tuning strategy rather than general ability to adapt to experimental data.

Q4.

Figure 18 shows that experimental fine-tuning can substantially increase energy and force errors on pretrained models. Could the authors discuss whether preserving the pretrained potentials accuracy should be incorporated into the benchmark? Would this be interesting for distinguishing models that achieve similar experimental accuracy but differ in how well they retain their original predictive capabilities?

Q5.

The authors now evaluate three regularization strategies on all six models for titanium elasticity partially addressing the question.

Overall:

The authors have included more experiments to work towards addressing weaknesses and questions. My remaining concerns are the diversity of the experimental fine-tuning tasks and that the benchmark characterizes uMLIP performance under one experimental fine-tuning strategy rather than general ability to adapt to experimental data. The benchmark would be interesting for the uMLIP community so I have increased my weak reject to a weak accept.

Thank you for the follow-up. We agree that both points are worth extending. The benchmark fine-tunes on five tasks and ranks the models under one shared protocol, so that every model is compared on equal terms. Models fine-tuned on more tasks or with other strategies can be scored on the leaderboard, which uses the same evaluation tasks.

Q4.

Could the authors discuss whether preserving the pretrained potentials accuracy should be incorporated into the benchmark? Would this be interesting for distinguishing models that achieve similar experimental accuracy but differ in how well they retain their original predictive capabilities?

Yes. We think the benchmark should report how much pretrained accuracy each model keeps after fine-tuning. This would distinguish models that match experiment equally well but lose different amounts of pretrained accuracy.

Reviewer_bNoQOpenReview ↗
Rating
3(+1)
Confidence
4(+1)
Informativeness
3
Presentation
2
Soundness
3(+1)
Read review
Paper summary

The paper introduces ExFiT-Bench. It evaluates universal machine-learning interatomic potentials (uMLIPs) after they are fine-tuned to experimental ensemble observables, using differentiable trajectory reweighting (DiffTRe). Six uMLIPs (MACE, MatterSim, PET-MAD, SevenNet, Orb, UMA) are fine-tuned on five tasks:

  • titanium elastic constants
  • the hcp–bcc phase transition line
  • the liquid structure factor S(q)
  • diamond elastic constants
  • MgO elastic constants

Models are scored on target and off target: held-out temperatures, the bcc polymorph, other titanium observables, phonons, and other compositions. Fine-tuning lowers on-target error for every model. Transfer to new conditions of the same observable is generally good, while transfer across observables and compositions is selective and sometimes harmful. Rankings after fine-tuning differ from pretrained rankings.

Critical strengths

Strengths

  1. It asks an important question no one has addressed. Existing uMLIP benchmarks score pretrained models only. In practice, though, researchers often correct a model against measurements before using it. This paper is the first systematic attempt to ask which model performs best after that correction. It runs six uMLIPs through a shared protocol, uses three seeds, sets explicit held-out conditions, and reproduces prior DiffTRe studies, which makes the benchmark credible. The release of code, machine-readable experimental data and a leaderboard gives the community a concrete reference point.

  2. The evaluation yields genuinely useful insights beyond the headline numbers. Learned-symmetry models (Orb, PET-MAD) lose rotational equivariance under fine-tuning (Table 2). A pooled S(q) error can improve while the second peak, which carries the short-range order, moves away from experiment (Table 12). The replay set determines which distribution regularization preserves (Figs. 5–6). Each of these is a practical warning for anyone fine-tuning on experiment, and none would be visible from on-target errors alone.

Critical weaknesses

Weaknesses

  1. The paper shows what transfers but does not explain why. Without an explanation, the results cannot be extended to a new model or task. The clearest example is the finding that corrections transfer between diamond and MgO but not to titanium. The paper attributes this to cubic versus hexagonal symmetry. With only three compositions, however, crystal system, bonding and chemistry are confounded. The paper's own data suggest a testable alternative. Most pretrained models underestimate C₁₁, C₃₃ and C₄₄, consistent with the systematic softening of the potential energy surface that Deng et al. (2025) report. If fine-tuning mainly corrects this shared bias, transfer should follow wherever the bias is shared. Two analyses would turn the benchmark from a catalogue of outcomes into a predictive tool:

    • a curvature analysis on off-target configurations;
    • a diagnostic such as gradient alignment between target and off-target observables.

    Similarly, the paper attributes forgetting to weight directions the experiments leave unconstrained, but never measures how many directions each task actually constrains. One joint fine-tuning run on the three titanium tasks would show whether these observables are compatible or in conflict. The conclusion itself names this as the main open problem.

  2. The central ranking claim is not statistically supported. The claim that pretrained rankings do not carry over rests on mean ranks across heterogeneous metrics. Many of the differences behind those ranks fall within noise or below the experimental floor. Two examples:

    • Fine-tuned bcc errors of 0.025–0.067 sit around a 0.050 disagreement between independent measurements.
    • Fine-tuned ΔG errors of 0.5–1.1 meV/atom are comparable to the 0.1–0.7 meV/atom reproducibility of thermodynamic integration.

    The reported standard deviations capture seed variation only, often 0.000. They omit the uncertainty of the MD estimators, and pretrained values carry no uncertainty at all. Changes such as 0.090 → 0.093 therefore cannot be interpreted. Rank correlations with bootstrap intervals, and a null showing how much ranks move from noise alone, are needed before the ranking result can be treated as a finding.

Questions for authors

Questions for the Authors

  1. Mechanism of transfer. Does transfer across compositions follow the shared softening of the potential energy surface rather than crystal symmetry? For example, do curvature changes on off-target configurations predict which off-target errors improve? So does gradient alignment between target and off-target observables?

  2. Ranking robustness. Please report rank correlations between pretrained and fine-tuned models, with bootstrap intervals over seeds and MD estimator noise. Do the rankings still differ once differences within noise are accounted for?

  3. Orientation dependence. Orb and PET-MAD show about 3% orientation dependence after fine-tuning. That is comparable to their on-target errors. How do the rankings change when elastic constants are averaged over orientations?

  4. Table 10. Sec. 3.2 says MgO fine-tuning improves diamond "for all six models." Yet PET-MAD's held-out diamond error rises from 0.160 to 0.178. Could you reconcile these?

6 comments

Thank you for the careful and constructive review. We quote each point and answer it below. The revised paper shows every change in blue, and its new Appendix C holds the new figures and tables.

W1.1.

The paper shows what transfers but does not explain why. Without an explanation, the results cannot be extended to a new model or task. The clearest example is the finding that corrections transfer between diamond and MgO but not to titanium. The paper attributes this to cubic versus hexagonal symmetry. With only three compositions, however, crystal system, bonding and chemistry are confounded.

We agree that the symmetry explanation was confounded, and five new held-out compositions show that corrections transfer most reliably to close analogues that share both crystal system and bonding, not by symmetry alone. We scored the models fine-tuned on the three elastic tasks against the elastic constants of Cu, Si, NaCl, Mg and ZnO, which separate crystal system from bonding, as Table A shows. A shared crystal system alone rarely changes the error. The three pairs that share both crystal system and bonding, diamond to Si, MgO to NaCl and titanium to Mg, share no element, so the transfer does not rest on chemistry. Section 3.2 now replaces the cubic versus hexagonal explanation with "Five held-out compositions show that this is not an effect of symmetry alone, as corrections transfer most reliably to close analogues that share both crystal system and bonding."

Table A. Transfer to five new held-out compositions. Rows group the 15 pairs of a training and a new composition by what they share, counted once per model. A change counts only beyond the measurement floor, the disagreement between independent experiments, plus two MD standard errors, with all three seeds agreeing in sign. A negative mean ΔSRME means the error on the new composition fell.
Shared with the training compositionPairsBetter / worse / within, over 6 models per pairMean ΔSRME
structure and bonding (diamond→Si, MgO→NaCl, Ti→Mg)39 / 2 / 7−0.017
structure only53 / 4 / 23+0.0003
bonding only (MgO→ZnO, Ti→Cu)22 / 2 / 8−0.002
neither52 / 6 / 22+0.006

W1.2, W1.3, Q1.

W1.2. The paper's own data suggest a testable alternative. Most pretrained models underestimate C₁₁, C₃₃ and C₄₄, consistent with the systematic softening of the potential energy surface that Deng et al. (2025) report. If fine-tuning mainly corrects this shared bias, transfer should follow wherever the bias is shared.

W1.3. Two analyses would turn the benchmark from a catalogue of outcomes into a predictive tool:

  • a curvature analysis on off-target configurations;
  • a diagnostic such as gradient alignment between target and off-target observables.

Q1. Mechanism of transfer. Does transfer across compositions follow the shared softening of the potential energy surface rather than crystal symmetry? For example, do curvature changes on off-target configurations predict which off-target errors improve? So does gradient alignment between target and off-target observables?

Yes, in part, transfer follows the shared bias. The softening sits mostly in C44, which is too soft in all six pretrained models on Ti, diamond and MgO. In the median case, 51% of the off-target change scales all constants by one common factor, and the rest corrects each constant by its own amount. We counted 126 cases, each a model, a source–target pair and one of C11, C12 and C44. A constant improves on the target in 65% of the 89 cases where its pretrained error has the same sign on source and target, against 41% of the other 37, as Figure 16 shows.

Both diagnostics predict which off-target errors improve, as Figure 17 shows. The curvature change, the log change of the mean squared phonon frequency, which equals the mean curvature of the energy surface at the crystal, predicts which models improve their phonon error on another composition, partly by construction since both read the same phonon bands. The gradient alignment, the cosine between the source-loss gradient and the gradient of each off-target constant, predicts which elastic constants of another composition improve. Constants it pushes toward experiment improve in 71% of cases, those it pushes away in 28%. The curvature change does not predict the change in a composition's overall elastic error, the one the paper reports, and the gradient alignment predicts only its direction, weakly. The revised paper states this scope.

W1.4.

Similarly, the paper attributes forgetting to weight directions the experiments leave unconstrained, but never measures how many directions each task actually constrains.

Each task constrains only one to four effective directions out of 2.9 to 291 million weights, so almost every direction stays free. We counted them at the pretrained weights by the participation ratio of the singular values of the gradients of the fitted values, three cubic constants or five Ti constants at four temperatures.

W1.5.

One joint fine-tuning run on the three titanium tasks would show whether these observables are compatible or in conflict. The conclusion itself names this as the main open problem.

The elastic and liquid structure-factor tasks are neither fully compatible nor fully in conflict, as the conflict runs in one direction and depends on the model. We fine-tuned the two jointly for all six models, as adding the phase-transition task did not fit in the rebuttal period. The joint run of each model comes close to its single-task runs on both targets. Adding the structure factor makes the elastic constants less accurate than training on them alone in four of six models. This loss of accuracy exceeds the 0.008 adiabatic bound, the largest change the adiabatic correction can make to the Ti elastic error. Adding the elastic constants leaves the structure factor as accurate as training on it alone in five models and improves it in PET-MAD.

W2.1, Q2.

W2.1. The central ranking claim is not statistically supported. The claim that pretrained rankings do not carry over rests on mean ranks across heterogeneous metrics.

Rank correlations with bootstrap intervals, and a null showing how much ranks move from noise alone, are needed before the ranking result can be treated as a finding.

Q2. Ranking robustness. Please report rank correlations between pretrained and fine-tuned models, with bootstrap intervals over seeds and MD estimator noise. Do the rankings still differ once differences within noise are accounted for?

We adopted the suggested statistics, and the pretrained order still changes beyond noise on the Ti elastic block of Table 2 of the revised paper and on its ΔG column, as Table B shows. The revised paper now reports the Kendall τb\tau_b between pretrained and fine-tuned orders for each block and its single-observable columns. The 95% intervals come from a bootstrap over seeds and MD trajectory blocks. Pairs closer than the noise floor count as ties. The noise floor is the disagreement between independent measurements, or the 0.008 adiabatic bound on Ti. On the phase-transition, diamond and MgO blocks, the fine-tuned order stays close to the pretrained one. We do not count two cases as evidence. Noise alone reorders the S(q) block as much as fine-tuning does, as the last column of Table B shows, and every pair in the bcc column lies within the bcc measurement floor.

Table B. Kendall τb\tau_b between pretrained and fine-tuned orders of the six models, 95% intervals from a bootstrap over training seeds and MD blocks. A block is the models fine-tuned on one task, ranked by mean rank over all their off-target columns. A τb\tau_b of +1 means an unchanged order, 0 no relation and −1 a reversed order. The third column ties pairs closer than the 0.008 adiabatic bound on Ti or the measurement floor elsewhere. The last column is the τb\tau_b that noise alone would give. Each model keeps its pretrained rank and takes the fine-tuned error of that rank, and seed and MD noise is added. Table 14 of the revised paper calls these two columns noise τ and compression null.
Block or columnτb\tau_b [95% CI]τb\tau_b, ties below floorτb\tau_b from noise alone
Ti elastic block0.00 [−0.28, +0.14]0.00+0.93
its ΔG column−0.73 [−0.87, −0.47]−0.59+0.88
its bcc column−0.28 [−0.55, +0.07]all pairs tied+0.83
phase-transition block+0.87 [+0.73, +1.00]+0.83+0.98
S(q) block+0.33 [+0.20, +0.73]+0.73+0.74
diamond block+0.41 [+0.33, +0.73]+0.55+0.95
MgO block+0.60 [+0.47, +0.97]+0.60+0.89

Giving each model the fine-tuned error of its pretrained rank and adding the seed and MD noise leaves the Ti elastic order almost unchanged, and none of the 4000 bootstrap draws of this comparison reorders the models as much as fine-tuning does. The abstract now replaces "the pretrained ranking does not consistently carry over" with "Fine-tuning reduces the on-target error for every model on every task, but the pretrained ranking is not a reliable proxy for the ranking after fine-tuning."

W2.2.

Many of the differences behind those ranks fall within noise or below the experimental floor. Two examples:

  • Fine-tuned bcc errors of 0.025–0.067 sit around a 0.050 disagreement between independent measurements.
  • Fine-tuned ΔG errors of 0.5–1.1 meV/atom are comparable to the 0.1–0.7 meV/atom reproducibility of thermodynamic integration.

We agree that the bcc differences and the ΔG values you cite are too small to interpret. Every pair of fine-tuned models lies within the 0.050 disagreement between the bcc measurements, so the revised rank statistics treat the bcc pairs as ties, and the order over all titanium properties still differs from the pretrained one, as Table B shows. The ΔG errors of 0.5 to 1.1 meV/atom come from the models fitted to ΔG itself and lie within the reproducibility, so we do not rank them. Our ranking claim uses the ΔG of the models fitted to the titanium elastic constants, which ranges from 1.5 to 10.9 meV/atom. There, 13 of the 15 pairs of models differ by more than the reproducibility, and with the other two tied the order is still reversed.

W2.3.

The reported standard deviations capture seed variation only, often 0.000. They omit the uncertainty of the MD estimators, and pretrained values carry no uncertainty at all. Changes such as 0.090 → 0.093 therefore cannot be interpreted.

Tables 16 to 19 of the revised paper now include the MD estimator error in the uncertainty of every elastic-constant and S(q) value, so no uncertainty is zero, unlike the seed-only 0.000 the paper printed, although a few intervals look like a single value at the printed precision. Pretrained values now also have intervals, from the same procedure. Table C shows the cases you raise. The intervals of 0.090 and 0.093 just touch, and the change is smaller than the 0.008 adiabatic bound, so we agree it cannot be interpreted.

Table C. Selected values with 95% intervals that include the MD estimator error, next to the seed-only standard deviation that the paper printed. Tables 16 to 19 of the revised paper give every elastic-constant and S(q) value in this form.
ValuePaper, mean (seed sd)95% interval with MD error
UMA, pretrained, Ti elastic SRME at the fitted temperatures0.090 (none)[0.088, 0.092]
UMA, MgO fine-tuned, Ti elastic SRME at the fitted temperatures (off target)0.093 (0.000)[0.092, 0.094]
SevenNet, Ti elastic fine-tuned, Ti SRME at held-out T0.040 (0.000)[0.039, 0.042]
SevenNet, Ti elastic fine-tuned, S(q) RMSE0.042 (0.000)[0.041, 0.043]
MACE, Ti phase fine-tuned, S(q) RMSE0.052 (0.000)[0.051, 0.053]

Q3.

Orientation dependence. Orb and PET-MAD show about 3% orientation dependence after fine-tuning. That is comparable to their on-target errors. How do the rankings change when elastic constants are averaged over orientations?

Averaging changes the order of the fine-tuned models only between models whose averaged errors differ by less than the 0.008 adiabatic bound on Ti or the measurement floor on diamond and MgO, as Figure 14 shows for six cell orientations at the fitted temperature. The revised paper reports these averaged values alongside the single-orientation ones.

Q4.

Table 10. Sec. 3.2 says MgO fine-tuning improves diamond "for all six models." Yet PET-MAD's held-out diamond error rises from 0.160 to 0.178. Could you reconcile these?

Thank you for catching this, and we have narrowed the sentence. MgO fine-tuning lowers the diamond error beyond the measurement floor for four models at the fitted temperature, but raises PET-MAD's error from 0.160 to 0.178 at the held-out temperature, so Section 3.2 now states the improvement for four of the six models.

Reviewer_EE9oOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

There is a lot of interest in ML for chemistry at the moment, but this area is not generally widely known in the ML community, so some explaining is necessary.

Universal Interatomic Potentials are energy-force molecular networks. They are trained on large datasets representing Density Functional Theory (DFT) and cover a large proportion of the periodic table of the elements. These are essentially the foundation models of chemistry. Another ingredient in the paper is Molecular Dynamics (MD) simulations, where learned forces are integrated and sampled through the Boltzmann distribution. Further, uMLIPs are learned energy functions, the Boltzmann distribution is the probability measure that energy defines, MD is the sampler for that measure. Finally, the paper studies how fine-tuning adjusts the energy so that expectations under the measure match experiment. In computer vision terms, this is similar to a Gibbs/MRF model (Geman & Geman) whose energy is provided by a neural network.

In the terms of the abstract, the paper compares 6 uMLIPs: MACE, MatterSim, PET-MAD, SevenNet, Orb and UMA, ranging from 3M to 290M parameters, by fine tuning them over 5 different tasks. The fine-tuning method is called DiffTRe. It uses importance sampling when possible and MD when degeneracies occur.

Errors are scored on-target and off-target:

  • On target, equivalent to the training error: the fitted observable itself
  • Off target:
    • held-out temperatures;
    • another crystal phase (bcc Ti);
    • other observables;
    • other materials.

The paper's finding are as follows:

  • Fine-tuning always reduces the on-target error, which is akin to a training error, so is expected.

  • The "off-target" errors are various generalization:

    • Held-out temperature between the fitted one (similar to interpolation): transfers well
    • Same observables outside the fitted range (mild extrapolation): mostly transfers
    • Same property (eg another crystal of the same element, equivalent to distribution shift): mostly improves
    • Other properties of the same element: (equivalent to a transfer to a different task): results are mixed
    • Other materials (equivalent to checking on catastrophic forgetting): no gain, sometimes worse.

Furthermore, every model improves on the fitted target. However, they improve by very different amounts, so the order of the six models by error gets reshuffled by comparing before and after fine-tuning.

The author's take on it is that if one plans to fine-tune on experimental data, choosing the model that tops a standard pretrained leaderboard is not reliable. How well a model absorbs the correction is a separate property, so a separate benchmark is needed. That is the paper’s main contribution, in addition to the benchmark and the methodology.

Critical strengths

1- The question makes sense. Practitioners already fine-tune foundation potentials on experimental data, and AFAIK, no one has compared models systematically on what that does elsewhere. Evaluating off-target effects is a real and significant contribution.

2- The physics is careful. It uses:

  • proper estimators (stress-fluctuation elastic constants, Frenkel–Ladd thermodynamic integration);
  • an adiabatic-vs-isothermal bound;
  • polar correction for MgO phonons;
  • two independent experimental measurements as an error floor, which is excellent practice;
  • reproductions of three published studies, including their DimeNet++ baselines.

3- The rotation check (Table 2) is good. It shows that fine-tuning degrades the rotation symmetry of the two models that learn it from data instead of building it in.

4- It is reproducible: 3 seeds, released code with a conformance test, a leaderboard, and honest limitations stated in §5.

Critical weaknesses

1- The scope is somewhat narrow

  • There are three materials, one each of metallic, covalent and ionic bonding.
  • Three of the five tasks are elastic constants.
  • Only titanium has several properties.
  • “Transfer across compositions is unreliable” rests on 3 materials, i.e. 6 directed pairs.

The general conclusions are plausible but thinly supported. At the same time, the leaderboard computations are already expensive.

2- Hyperparameters were chosen on the target alone, but the conclusions are about off-target behaviour.

  • With so many free directions, what happens off target is decided largely by implicit regularisation: the learning rate, only 40 epochs, and Adam with an unusual β = (0.1, 0.4) [please comment]
  • The learning rate is picked per model and task by the lowest on-target training loss, from 3–4 values, with no validation on anything else.
  • So the off-target rankings, and the headline “ranking does not carry over”, may reflect optimiser settings as much as the models.

3- Some ranking differences are smaller than known systematic errors.

  • The adiabatic correction bounded at an SRME of 0.008 on Ti (Appendix A.3). The fine-tuned on-target Ti errors are 0.011–0.033.
  • For Orb and PET-MAD, the elastic constants depend on cell orientation by up to 3.1% after fine-tuning (Table 2), and they are reported in a single orientation without averaging. That is the same size as the errors being ranked.
  • On diamond and MgO, several fine-tuned models already fall within the disagreement between experiments, so ranking them is meaningless.

Using mean ranks hide all of this, so confidence intervals should be reported.

5- Related to this, performance compression makes reshuffling almost unavoidable. After fine-tuning, all six errors are squeezed into a narrow band (0.011–0.033, versus 0.090–0.387 before). When values are that close, small differences decide the order, so ranks become unstable whether or not there is a real effect. “The ranking changes” is partly what one would expect from compression alone. Given this, it would make sense to recommend a model taking into account computation costs and not just performance.

6- The models differ widely in architectures, size and data for pre-training. Is there a link between the findings ("which models fine-tunes best") and the model design choices?

7- The leaderboard is expensive: A full leaderboard entry needs 5 tasks × 3 seeds × a learning-rate sweep, which translates from hundreds to thousands of GPU-hours. Is the leaderboard realistic for the community? This should. be reported in the paper.

Questions for authors
  • how do the off-target errors vary across the learning-rate sweep? presumably you already have those runs?
  • Can you give rank stability with uncertainty, e.g. Kendall τ between the pretrained and fine-tuned rankings with bootstrap intervals, treating differences below the experimental disagreement or orientation spread as ties?
  • Can the systematic-softening explanation be tested directly? For example: does the transferred correction act mainly as a global stiffening, and does transfer success track the sign pattern of the pretrained errors?
  • Why not average the elastic constants over orientations for Orb and PET-MAD, given the cost is small next to MD?
  • What is the total compute for one full leaderboard entry?
  • Do you expect the conclusions to hold beyond three materials? Which additions (alloys, molecular liquids, oxides) would test them most?
  • Can you provide an accuracy-versus-GPU-hours plot (ie a Pareto front) per task, separating fine-tuning cost from production MD cost, and a cost column in the leaderboard. That would make the benchmark much more useful to practitioners, who choose under a compute budget. It would also show whether the paper’s compressed on-target differences are worth the 6× spread in cost.
3 comments

Thank you for the careful and constructive review. We quote each point and answer it below. The revised paper shows every change in blue, and its new Appendix C holds the new figures and tables.

W1.1, Q6.

W1.1. The scope is somewhat narrow

  • There are three materials, one each of metallic, covalent and ionic bonding.
  • "Transfer across compositions is unreliable" rests on 3 materials, i.e. 6 directed pairs.

The general conclusions are plausible but thinly supported.

Q6. Do you expect the conclusions to hold beyond three materials? Which additions (alloys, molecular liquids, oxides) would test them most?

We agree that more compositions were needed, and we added five held-out compositions, which show that corrections transfer most reliably to close analogues that share both crystal system and bonding. We therefore expect the conclusions to hold beyond these materials in this narrower form. We scored the six uMLIPs fine-tuned on the diamond, MgO and titanium elastic constants on five new compositions, Cu, Si, NaCl, Mg and ZnO. These separate crystal system from bonding. Table A shows the result.

Table A. Transfer to five new held-out compositions. Rows group the 15 pairs of a training and a new composition by what they share, counted once per model. A change counts only beyond the measurement floor, the disagreement between independent experiments, plus two MD standard errors, with all three seeds agreeing in sign. A negative mean ΔSRME means the error on the new composition fell.
Shared with the training compositionPairsBetter / worse / within, over 6 models per pairMean ΔSRME
structure and bonding (diamond→Si, MgO→NaCl, Ti→Mg)39 / 2 / 7−0.017
structure only53 / 4 / 23+0.0003
bonding only (MgO→ZnO, Ti→Cu)22 / 2 / 8−0.002
neither52 / 6 / 22+0.006

Section 3.2 now replaces its symmetry explanation with "Five held-out compositions show that this is not an effect of symmetry alone, as corrections transfer most reliably to close analogues that share both crystal system and bonding."

Molecular liquids would test the conclusions most, as their bonding lies furthest from the current tasks. Alloys would extend the transfer test to mixed metallic compositions, and oxides would add fewer new cases, since MgO already represents them.

W1.2.

  • Three of the five tasks are elastic constants.
  • Only titanium has several properties.

We agree that three fitted targets are elastic constants and that titanium alone is fitted to several properties, but the benchmark gets its breadth from what each model is scored on, and diamond and MgO models are also scored on several properties. Their fine-tuned models are scored on their phonons and on the elastic constants of the other compositions.

W2, Q1.

W2. Hyperparameters were chosen on the target alone, but the conclusions are about off-target behaviour.

  • With so many free directions, what happens off target is decided largely by implicit regularisation: the learning rate, only 40 epochs, and Adam with an unusual β = (0.1, 0.4) [please comment]
  • The learning rate is picked per model and task by the lowest on-target training loss, from 3–4 values, with no validation on anything else.
  • So the off-target rankings, and the headline "ranking does not carry over", may reflect optimiser settings as much as the models.

Q1. how do the off-target errors vary across the learning-rate sweep? presumably you already have those runs?

Selecting the rate on the target alone is deliberate, because a practitioner fine-tuning on one experiment has no measurement of the off-target properties. This choice does not drive the off-target rankings. The rate that fits the target best in the sweep also gives the lowest off-target error in 16 of the 18 combinations of six models and three elastic tasks. The two exceptions are UMA on titanium and PET-MAD on diamond. The off-target error here is the geometric mean of the ratio of fine-tuned to pretrained error over every off-target property, such as held-out temperatures, other compositions and phonons.

The transfer to other compositions also barely depends on the number of epochs or on β. Retraining all six models with Adam's default β = (0.9, 0.999) leaves 29 of the 36 errors on other compositions within the measurement floor, one for each of six models on six pairs of a training and another composition. Training MACE, MatterSim and SevenNet on diamond and MgO for 80 instead of 40 epochs leaves 11 of 12 such errors unchanged, one for each of three models on four pairs.

We took β = (0.1, 0.4) from the released code of the diamond and titanium source studies, Thaler & Zavadlav (2021) and Röcken & Zavadlav (2024). They also suit runs of only 40 to 160 updates, far fewer than the roughly 1000 steps that the default β₂ = 0.999 averages over.

W3.1.

Some ranking differences are smaller than known systematic errors.

  • The adiabatic correction bounded at an SRME of 0.008 on Ti (Appendix A.3). The fine-tuned on-target Ti errors are 0.011–0.033.

We agree that differences below 0.008 cannot order the models physically, and our ranking claim does not rely on them. The fine-tuned order of the Ti elastic models on the other titanium properties stays unrelated to the pretrained order even when every Ti difference below 0.008 counts as a tie, with τb\tau_b of 0.00, as Table B below shows. A lower on-target error below the bound still means a closer fit to the experimental values, though not a more accurate model.

W3.2, Q4.

W3.2. For Orb and PET-MAD, the elastic constants depend on cell orientation by up to 3.1% after fine-tuning (Table 2), and they are reported in a single orientation without averaging. That is the same size as the errors being ranked.

Q4. Why not average the elastic constants over orientations for Orb and PET-MAD, given the cost is small next to MD?

Averaging over orientations changes the order of the fine-tuned models only between models whose averaged errors differ by less than the 0.008 adiabatic bound on Ti or the measurement floor on diamond and MgO. We averaged over six cell orientations at the fitted temperature, as you suggested, and Figure 14 shows the result. The revised paper reports these averaged values alongside the single-orientation ones.

W3.3.

On diamond and MgO, several fine-tuned models already fall within the disagreement between experiments, so ranking them is meaningless.

Ranking the fine-tuned models on target remains meaningful as a measure of how closely each model fits the chosen experiment, although not as physical accuracy. Our claim that the ranking changes concerns off-target errors. On diamond and MgO, the fine-tuned order stays close to the pretrained one once pairs within the floor are tied, so this claim rests on titanium, as Table B below shows.

W3.4, Q2.

W3.4. Using mean ranks hide all of this, so confidence intervals should be reported.

Q2. Can you give rank stability with uncertainty, e.g. Kendall τ between the pretrained and fine-tuned rankings with bootstrap intervals, treating differences below the experimental disagreement or orientation spread as ties?

We adopted the suggested statistics, and the pretrained order still changes beyond noise on the Ti elastic block of Table 2 of the revised paper and on its ΔG column, as Table B shows. The revised paper now reports the Kendall τb\tau_b between pretrained and fine-tuned orders for each block and its single-observable columns. The 95% intervals come from a bootstrap over seeds and MD trajectory blocks. Pairs closer than the noise floor count as ties. The noise floor is the disagreement between independent measurements, or the 0.008 adiabatic bound on Ti. On the phase-transition, diamond and MgO blocks, the fine-tuned order stays close to the pretrained one. We do not count two cases as evidence. Noise alone reorders the S(q) block as much as fine-tuning does, as W5.1 explains, and every pair in the bcc column lies within the bcc measurement floor.

Table B. Kendall τb\tau_b between pretrained and fine-tuned orders, 95% intervals from 4000 bootstrap replicates over seeds and MD blocks. A block is the models fine-tuned on one task, ranked by mean rank over all their off-target columns. A τb\tau_b of +1 means an unchanged order, 0 no relation and −1 a reversed order. The last column ties pairs closer than the 0.008 adiabatic bound on Ti or the measurement floor elsewhere.
Block or columnτb\tau_b [95% CI]τb\tau_b, ties below floor
Ti elastic block0.00 [−0.28, +0.14]0.00
its ΔG column−0.73 [−0.87, −0.47]−0.59
its bcc column−0.28 [−0.55, +0.07]all pairs tied
phase-transition block+0.87 [+0.73, +1.00]+0.83
S(q) block+0.33 [+0.20, +0.73]+0.73
diamond block+0.41 [+0.33, +0.73]+0.55
MgO block+0.60 [+0.47, +0.97]+0.60

The abstract now replaces "the pretrained ranking does not consistently carry over" with "Fine-tuning reduces the on-target error for every model on every task, but the pretrained ranking is not a reliable proxy for the ranking after fine-tuning."

W5.1.

Related to this, performance compression makes reshuffling almost unavoidable. After fine-tuning, all six errors are squeezed into a narrow band (0.011–0.033, versus 0.090–0.387 before). When values are that close, small differences decide the order, so ranks become unstable whether or not there is a real effect. "The ranking changes" is partly what one would expect from compression alone.

Compression alone does not explain the change in ranking. We gave each model the fine-tuned error of its pretrained rank, so the order is kept but the errors are as close as after fine-tuning, and added the seed and MD noise. This keeps the pretrained order almost intact on the other titanium properties, with τb\tau_b of +0.93, while the actual fine-tuned models give 0.00. The narrowing on target is the intended effect of fitting one experiment, and the order there still measures how closely each model fits it. The off-target errors narrow mainly at the held-out temperatures of the fitted observable. The gaps between models on ΔG exceed the reproducibility of the free-energy calculation in 13 of the 15 pairs, and the order still reverses with the other two pairs tied.

W5.2, Q7.

W5.2. Given this, it would make sense to recommend a model taking into account computation costs and not just performance.

Q7. Can you provide an accuracy-versus-GPU-hours plot (ie a Pareto front) per task, separating fine-tuning cost from production MD cost, and a cost column in the leaderboard. That would make the benchmark much more useful to practitioners, who choose under a compute budget. It would also show whether the paper's compressed on-target differences are worth the 6× spread in cost.

We have added both. Figure 15 plots on-target error against fine-tuning cost and, separately, production-MD cost, with the Pareto front of each task, and the leaderboard now lists both costs for each entry. Figure 15 also gives a recommendation. When on-target differences are within the noise floor, the models fit the chosen experiment differently but are equally accurate physically, so we recommend the cheapest one after checking its off-target errors on the leaderboard. Otherwise we choose from the front in Figure 15.

W6.

The models differ widely in architectures, size and data for pre-training. Is there a link between the findings ("which models fine-tunes best") and the model design choices?

Apart from one link, design does not predict the fine-tuning outcome. Orb and PET-MAD, which learn rotational symmetry from data rather than building it in, become orientation dependent after fine-tuning. We compared four design traits, built-in symmetry, parameter count, pretraining-set size and cutoff radius, with nine fine-tuning outcomes. Only parameter count tracks one of them, the transfer to other compositions, and about two such matches are expected by chance among these 36 comparisons.

W1.3, W7, Q5.

W1.3. At the same time, the leaderboard computations are already expensive.

W7. The leaderboard is expensive: A full leaderboard entry needs 5 tasks × 3 seeds × a learning-rate sweep, which translates from hundreds to thousands of GPU-hours. Is the leaderboard realistic for the community? This should. be reported in the paper.

Q5. What is the total compute for one full leaderboard entry?

We believe the leaderboard is realistic, because it fixes the evaluation and leaves training to the submitter. Each entry declares its training data, loss, optimiser and schedule on a card with its code. A full entry with three seeds and the rate sweep takes 1242 to 2108 H200 GPU-hours, one to two weeks on one 8-GPU node. An entry may also report one seed and only some tasks. One seed of all five tasks takes two to three days, and one seed of one task well under a day, as Table C shows. Table 13 of the revised paper reports these costs.

Table C. H200 GPU-hours of one leaderboard entry, measured on an idle card.
PathWhat it includesPer modelAll six models
Full5 tasks × 3 seeds, learning-rate sweep1242–21089400
One seed1 seed per task at the declared rate318–5562549
One task, one seed1 seed of 1 task at the declared rate30–123—

Q3.

Can the systematic-softening explanation be tested directly? For example: does the transferred correction act mainly as a global stiffening, and does transfer success track the sign pattern of the pretrained errors?

Yes, and the direct test supports the explanation in part. A median share of 0.51 of the off-target change in the constants lies along one common scaling of all of them. We counted 126 cases, each a model, a source–target pair and one of C11, C12 and C44. A constant improves on the target in 65% of the 89 cases where its pretrained error has the same sign on source and target, against 41% of the other 37, as Figure 16 shows.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General response

We thank the three reviewers for detailed and constructive reviews. We have uploaded a revised paper with every change in blue, and its new Appendix C collects the new results. We summarize the main changes below and answer each point in full under the reviewer who raised it.

Five new held-out compositions show what transfers. Corrections transfer most reliably to close analogues that share both crystal system and bonding. Please see EE9o W1.1 and Q6, bNoQ W1.1, and hisW W1, Q1 and Q2.

We tested why corrections transfer. Corrections partly follow the pretrained bias that source and target share, and two diagnostics, the curvature change and the gradient alignment, predict which phonon and elastic-constant errors improve. Please see EE9o Q3 and bNoQ W1.2 to W1.4.

The change in ranking is statistically supported. After fine-tuning on the titanium elastic constants, the pretrained order of the models changes beyond noise on the other titanium properties and reverses on the free energy. Every regularized strategy that fits the target still changes the pretrained order. The abstract now states the claim as "the pretrained ranking is not a reliable proxy for the ranking after fine-tuning". Please see EE9o W3.4, Q2 and W5.1, bNoQ W2.1, Q2 and W2.2, and hisW W3.

Values now carry uncertainties that include the MD estimator error, and averaging over cell orientations changes the order only between models within noise. Please see bNoQ W2.3 and Q3, and EE9o W3.2 and Q4.

We now report cost and recommend models with it. A leaderboard entry can be a single task at one seed, which takes 30 to 123 H200 GPU-hours per model, and Figure 15 of the revised paper recommends models by accuracy against cost. Please see EE9o W1.3, W5.2, W7, Q5 and Q7.

Revisions to the paper.

  • The abstract, contributions and conclusion state the ranking claim as above.
  • Section 3.2 replaces the symmetry explanation with the close-analogue rule.
  • A new Appendix C adds the rank statistics, new compositions, uncertainties, orientation averages, retention, transfer diagnostics, costs and the cost-aware recommendation.
  • Section 3.2 now checks each transfer against the measurement floor, which narrows three statements. Diamond fine-tuning improves MgO only for MACE and PET-MAD, MgO fine-tuning moves the titanium error beyond the bound only for Orb, and phase-transition fine-tuning changes the phonon error by less than its floor.
  • The revision renumbers the tables, so the old Tables 1, 2, 3 and 4 are now Tables 2, 3, 4 and 1.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.