- Rating
- 3(+2)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2(+1)
Read review
Paper summary
The paper bundles CCSD(T)-quality reaction-energy targets from pre-existing benchmarks (GMTKN55, GSCDB137, RDB7, MSR-ACC/TAE25, and DES15K; spanning main-group and transition-metal thermochemistry, kinetics, barrier heights, atomization reactions, and non-covalent interactions) into a unified reaction representation and augment every molecular state with standarized PBE0/def2-QZVP KS-DFT auxiliary data (electron densities, density matrices, Hamiltonians, etc.). On top of this dataset, they train and evaluate several models:
- fine-tuning the non-local neural XC functional Skala (including a seperate small-scale transition-metal run),
- MACE-based delta-learning and adaptive-hybrid models on top of a B3LYP baseline,
- XC enhancement-factor models and a DeePHF baseline trained from scratch.
Critical strengths
S1: Highly relevant and very useful
Highly relevant contribution with a lot of practical utility for a very active field that is currently lacking high quality easy-to-use datasets.
Critical weaknesses
W1: Claims about the model comparison do not quiet match actual experiments
The dataset is a strong core contribution, but the framing of the experiments should be scoped more carefully.
- The conclusion claims a side-by-side comparison of three neural DFT paradigms "under exactly the same data and protocol". Only Skala is a neural XC functional.
- Delta-MACE is an additive energy correction on B3LYP and alpha-MACE predicts a mixing coefficient.
- The protocols differ (Skala is trained post-SCF on frozen PBE0 densities, the MACE models sit on a B3LYP baseline, the TM run is a separate fine-tuning with 221 training reactions).
- Fig. 3 shows a single run per model without seeds or uncertainty estimates, so differences between models cannot be assessed.
- The claim that it is possible "for the first time" to compare neural DFT architectures head-to-head is too strong. Multi-seed, multi-architecture ML-XC functional comparisons under a shared training protocol already exist, albeit on lower-level range seperated hybrid targets rather than coupled-cluster data. The novelty here is having much better CCSD(T)-quality reaction energy datasets from several established benchmarks paired with standardized DFT inputs, and the claims should be scoped to that. See more in Q5.
W2: Sometimes key information is completely missing.
- Dataset table (see also Q4)
- I coudn't find a clear definition of (page 4) which is part of the core "NER" metric that large parts of the paper are based on, is it just the standard deviation of the energy differences. Given that interpretation see Q6.
- Have the reference methods also been homogenized? (See W3 under the assumption that the authors have done this.)
- Appendix B might benefit from just including pseudo-code, but if you release that code together with the dataset, then just linking it for the final submission is fine.
- Which version of Skala are you using? V1.0 or V1.1? (based on your citation I would guess V1.0?)
- the update they released this year seems to be outperforming range seperated hybrids the one from previous year does not.
- the difference is a small architecture tweak leading to longer range, and a larger training set afaik.
W3: Reference methods have not been homogenized. (?)
This is my most pressing concern The references come from different protocols:
- GMTKN55 mixed CCSD(T)/CBS-type,
- GSCDB137, RDB7 higher-level than most of the others, RDB7 is valence only with frozen cores?
- DES15K: CCSD(T)/CBS relative energies, no absolute energies available?
- MSR-ACC/TAE25: CCSD(T)/CBS level W1-F12-type protocol.
W4: No code in supplementary material for review.
Although the Reproducibility Statement links to the dataset and checkpoints, the training and evaluation code promised in the Abstract/Conclusion is not provided. Given that the training pipeline is dressed as a core contribution, it would be nice to be able to test the performance characteristics and througput of the pipeline.
W5: Strong-baseline claim of the second contribution seems not sufficiently supported.
A systematic empirical study of several neural DFT model variants, including nonlocal neural XC functionals [...]. We establish strong baselines.
The NN-XC models are trained and evaluated on the full holdout only post-SCF on a precomputed donated PBE0 density (with self-consistency only evaluated once on the 300-reaction slim subset, where NER actually degrades from 1.29 to 1.41). Training solely on fixed densities leaves the XC potential unconstrained and can introduce large density-driven errors (or require a costly hybrid PBE0 SCF calculation at inference time to donate the density) compared to self-consistent training/evaluation at scale. See e.g.
- Kim, M.-C.; Sim, E.; Burke, K. Understanding and Reducing Errors in Density Functional Calculations. Phys. Rev. Lett. 2013, 111 (7), 073003. https://doi.org/10.1103/PhysRevLett.111.073003
- Li, L. et al. Kohn-Sham Equations as Regularizer: Building Prior Knowledge into Machine-Learned Physics. Phys. Rev. Lett. 2021, 126 (3), 036401. https://doi.org/10.1103/PhysRevLett.126.036401
Questions for authors
I am willing to substantially increase my score if my questions below are adequately addressed and some of the claims are better scoped. In general, I enjoyed the paper but it seems very unpolished as is.
Q1: Absolute reference energy labels
Are absolute energies also released with the dataset or really only energy differences? -> This would extending the dataset easier.
Q2: Level of theory
Could you expand upon the rational behind PBE0 / def2-QZVP level of theory choice? Why not limit to triple zeta and add diffuse long range functions? Why PBE0 vs. other hybrids, or maybe even range seperated hybrids with dispersion corrections like wB97M-V?
In addition, we store the semilocal DFA part of the B3LYP Becke (1993) exchange-correlation energy EB3LYP xc [ρ] = ∫︁ εB3LYP xc [ρ(r), ∇ρ(r)] dr and the D3 dispersion corrections evaluated for B3LYP functional.
Do you perform this as a second DFT calculation or do you reuse the density from your PBE0 features? What about the reference methods, I would expect there to systemic errors in the computed energies that might hinder joint training on the entire dataset.
Q3: Unrestricted KS vs. Restricted KS
Why are you using UKS throughout and not just for open-shell systems? Woudn't RKS be both more efficient and more accurate for closed-shell systems?
Q4 Could you please add a Table in the appendix that gives a strucured summary of section 2.2. and figure 1a?
What is stored per sample?
- Point cloud (R, Z)
- ground state descriptors: , what about the KS-Hamiltoinan
- (which ?)
- high accuracy (CCSD(T) and beyond) relative energies. What about absolute energies?
- Grid properties (Which grid level? Is it the standard pyscf grid level 3 quadrature grid?)
- ...
Q5: Validation set design
By pruning filtering density grids from GSCDB137, arent you effectively filtering out large molecules? Recent works suggest, that size-extrapolation of learned functionals might be an issue especially for more expressive non-local architectures. Could you please add a small size extrapolation subset or is this to expensive to compute with wave function based methods even for just a handful of molecules?
Q6: Definition of
Is the distribution of the energy differences heavy tailed and are the mean errors typically dominated by individual samples? Also it might be worth here to compare to
Dick, S.; Fernandez-Serra, M. Highly Accurate and Constrained Density Functional Obtained with Differentiable Programming. Phys. Rev. B 2021, 104 (16), L161109. https://doi.org/10.1103/PhysRevB.104.L161109
where I first saw such a construction but they seem to normalize by the entire range.
Q7: Intercompatible are these reference calculation protocols
Do you have an idea how much the different reference calculation protocols disagree with each other? Is the relative-energy MAE on the order of mHa or Ha etc.?
Q8: Could you also add the mGGA-level NN-XC functionals to the comparison?
Q9: Self-consistent training and evaluation of Skala
Section 4 notes that main training and holdout evaluations are performed post-SCF on frozen PBE0/def2-QZVP densities (please also state this explicitly in Section 3.1), while self-consistency is only mentioned once on the 300 reaction slim subset. Do you only evaluate self-consistently on the slim subset, or do you also train self-consistently (e.g., via the implicit function theorem or differentiating through the SCF solver)?
General remarks (did not negatively influence my recommendation)
- Please mind the correct application of citet vs citep.
- I would not characterise DM21 as state of the art and rather refer to the Nagai et al 2020 and Dick et al 2021 as early works on the topic.
- On page 3 ". or noncovalent" you are missing an "F" (let a model proof-read for typos).
- On page 4 (Please add numbers to equations) you define but dont use it in the sum.
- section 3 has exactly one subsection 3.1, you might wanna remove the subsection devision
14 comments
Responce. Part 2
W3. Thank you for your point. CAREB retains the reference energy differences from the original benchmarks and standardizes the DFT inputs and reaction representation. Using different reference protocols is a deliberate choice - the objective is to preserve accurate reference values across different chemical regimes. Recomputing everything with one method would require demonstrating that the replacement maintains or improves that accuracy. Moreover, this approach has direct precedents. GMTKN55 combines W4, W1-F12, W2-F12, CCSD(T)/CBS and other reference protocols [1, Table 1]. GSCDB137 likewise selects and updates references by subset, including core-valence and beyond-CCSD(T) corrections where available [2, Section 3]. These choices reflect the different requirements of atomization energies, reaction barriers and noncovalent interactions. A common method name alone would not ensure comparable errors across these quantities. The same principle applies to training. For example, ωB97M(2) was fitted using the training partition of MGCDB84, a collection assembled from numerous independently developed benchmark datasets [3]. Thus, combining curated reference sources is an established approach to functional development, as well as evaluation. DES15K provides a particularly concrete example. Its interaction energies combine MP2/CBS with a CCSD(T) correction evaluated in a system-dependent basis. The original authors tested this protocol against independent S66 and S66x8 reference values at the same geometries, obtaining mean unsigned errors of 0.07 and 0.05 kcal/mol, respectively [4, Technical Validation]. This is evidence that different reference protocols can yield closely agreeing targets; identical computational settings are not a prerequisite. Regarding absolute energies, the DES15K target is the interaction energy, E(AB) − E(A) − E(B). Our model predicts this same difference from the constituent states. Absolute high-level energies are therefore unnecessary for this training objective: the published interaction energy directly supplies the required label. Regarding RDB7, it uses frozen-core correlated reference calculations, and CAREB retains that published protocol. There are independent precedents for using this reference level in learned electronic-structure methods. The DeePHF reaction-modeling study in JACS Au (2025, DOI: 10.1021/jacsau.5c00541, Section 2.2.1) trains density-matrix-based energy corrections against CCSD(T)-F12a/cc-pVDZ-F12 energies, the same method and basis used in the original RDB7. Separately, the study 10.1002/sstr.70504 directly uses RDB7 reference energies to train corrections to methods including r²SCAN-3c. Our inclusion of RDB7 follows this established use of high-level reaction data for learning energy corrections. It adds extensive coverage of reaction barriers and reaction energies while preserving the source reference protocol. The rationale for including this component is therefore its demonstrated relevance to learned electronic-structure models and its coverage of reaction chemistry. We therefore distinguish reference accuracy from protocol uniformity. The appropriate test is whether the reference approximations are sufficiently accurate and compatible for the target quantities. Retaining the curated source references preserves their established accuracy assessments; imposing a single replacement protocol would not, by itself, improve the benchmark. [1] Goerigk et al. A look at the density functional theory zoo with the advanced GMTKN55 database for general main group thermochemistry, kinetics and noncovalent interactions (2017): https://doi.org/10.1039/C7CP04913G [2] Liang and Head-Gordon. Gold-Standard Chemical Database 137 (GSCDB137): A Diverse Set of Accurate Energy Differences for Assessing and Developing Density Functionals (2025): https://doi.org/10.1021/acs.jctc.5c01380 [3] Mardirossian and Head-Gordon. Survival of the most transferable at the top of Jacob’s ladder: Defining and testing the ωB97M(2) double hybrid density functional (2018): https://doi.org/10.1063/1.5025226 [4] Donchev et al. Quantum chemical benchmark databases of gold-standard dimer interaction energies (2021): https://doi.org/10.1038/s41597-021-00833-x [5] Liu et al., “Revisiting a large and diverse data set for barrier heights and reaction energies: best practices in density functional theory calculations for chemical kinetics,” Phys. Chem. Chem. Phys. 27, 13326–13339 (2025). https://doi.org/10.1039/D5CP01181G
W4. The training and evaluation scripts are included in the public release at https://dataset.chemdft.workers.dev/code/. The Reproducibility Statement links to the release root, which may have made the code less apparent. We have added the direct code link to the statement.
Responce. Part 3
W5. The main experiment evaluates models using the same type of precomputed PBE0 inputs supplied by CAREB. This is a useful benchmark setting in its own right: it allows model development without repeating the electronic-structure preprocessing for every experiment. Accuracy in this setting is the result being measured; self-consistent performance is assessed separately in Section 4. The reported SCF calculation converged for all required states and changed MAE from 0.84 to 1.00 kcal/mol (“1.84” instead of “0.84” was typo in the submitted manuscript and we changed the number in present text), while NER changed from 1.29 to 1.41. These results concern the slim validation subset. The cited density-corrected DFT literature distinguishes functional errors from density-driven errors; it does not establish that fixed-density evaluation is inherently less accurate. Energy fitting alone does not uniquely determine the XC potential, but the model’s parametrization and pretraining also constrain it. This distinction matters when developing a self-consistent functional, while leaving the value of the fixed-input benchmark intact. The PBE0 calculation is part of the cost of generating those inputs.
Q1 . Thank you for your question! CAREB releases high-level relative (reaction) energies as given by the source benchmarks. Absolute energies are not uniformly available across sources. Therefore, absolute energies are not included. Our training/evaluation label is the reaction energy difference as in other papers in the field.
Q2. We chose PBE0/def2-QZVP to generate reusable electronic-structure inputs, with particular attention to the density representation. The high-level reference energies provide the training targets independently of this choice. Our use of PBE0 is supported by density benchmarks. Medvedev et al. [1] compared 128 functionals on atomic and ionic systems and included PBE0 in their L1 group of methods producing the most accurate densities according to their criteria (Table 1). Their density-based analysis of PBE hybrids also yielded an optimal exact-exchange fraction of 26.3%, close to the standard 25% in PBE0. These results support PBE0 as a density donor. The stronger reaction-energy performance of ωB97M-V does not, by itself, establish that its density would be a better input for the learned models. Our choice concerns the input representation rather than selecting the most accurate standalone energy functional. The quadruple-zeta basis was selected for the same reason. Jakobsen and Jensen [2] showed that representing accurate reference densities using single-determinant orbitals can incur significant errors in double- and triple-zeta basis sets, principally because of insufficient higher-order polarization functions. In their tests, these errors became practically insignificant at quadruple-zeta quality or above. This provides a rationale for using a polarized quadruple-zeta basis in a resource intended for density-functional development. Diffuse functions improve the description of extended density tails, particularly for anions and weakly bound systems, but do not provide the same increase in polarization flexibility. An augmented triple-zeta basis therefore addresses a different aspect of basis completeness. We prioritized reducing representation errors through quadruple-zeta quality, while recognizing the value of diffuse augmentation for particularly diffuse systems. Since preprocessing is performed during dataset construction and reused across subsequent training runs, its additional computational cost is amortized over model development. The semilocal B3LYP XC contribution is evaluated on the converged PBE0 density. Accordingly, an additional B3LYP SCF calculation was not performed. Exact exchange is stored separately and included with the B3LYP mixing coefficient of 0.20. The density used for each term in the B3LYP-based baseline is PBE0, which we have clarified in Section 2.2. D3(BJ) is evaluated separately from the molecular geometry using the B3LYP damping parameters and requires no additional density calculation. Finally, the reference-protocol question is separate from input generation. CAREB retains the published reference energy differences from its source benchmarks; it does not construct targets by combining absolute energies from unrelated protocols. GMTKN55 and GSCDB137 themselves use subset-specific reference methods [3,4]. Retaining these curated references preserves their original accuracy assessments. Recomputing all labels with one method would introduce a new reference approximation whose accuracy would need to be established. The specific differences between source protocols and their relevance to joint training are addressed in W3.
[1] https://doi.org/10.1126/science.aah5975
[2] https://doi.org/10.1021/acs.jctc.0c01029
Responce. Part 1
Dear reviewer GfaL, thank you for your thoughtful review! Below we address all your points.
W1. The main experiment covers three distinct uses of CAREB: learning an XC functional with Skala, learning an additive energy correction with Δ-MACE, and learning an exchange-mixing coefficient with α-MACE. Appendices D–E specify the same main-group training splits and reaction-energy objective. However, the energy expressions and baselines are method-specific, and fine-tuning pretrained Skala differs from training the MACE models. We have therefore replaced “under exactly the same data and protocol” with an explicit description of these shared and method-specific elements, and described the experiment as a comparison of modeling strategies rather than three neural XC architectures. To assess variability across training runs, we have launched repeated Skala fine-tuning and training of both MACE models using three random seeds per method, keeping the main-group training, validation, and holdout reactions fixed. Once these runs are complete, we will update Figure 3 with the mean and standard deviation of the test-subset results. We have also removed the broad “for the first time” claims about head-to-head architecture comparisons from Sections 5 and 7. Our contribution is the open release of 130053 high-level reaction-energy targets from established benchmarks, paired with standardized DFT-derived inputs and organized in a unified reaction representation. The experiments demonstrate how this resource supports neural XC, Δ-learning and adaptive-hybrid approaches.
W2. Dataset structure. Table 1 summarizes the source datasets and target counts, while Section 2.2 describes the stored quantities. We have summarized this information together in a schema table in revised manuscript that separates molecular-state records from reaction records. The table specifies, for each molecular state, the nuclear charges and coordinates, total charge and spin multiplicity, spin-resolved PBE0 density matrices, the energy components Enuc, E1, EH, and ExHF, the spin-resolved densities on the numerical grid with grid coordinates and weights, the semilocal B3LYP XC energy evaluated on the PBE0 density, the B3LYP D3(BJ) dispersion correction, and basic metadata (number of atoms and grid points); and, for each reaction, the reaction ID, stoichiometry over molecular states, and the high-level relative reference energy. Definition of σ_d. We use the fixed normalization constants from GSCDB137, supplied in Standard_errors.csv. In that benchmark, the “standard error” for subset d is defined as σ_d = (e_d,(2) + e_d,(3) + e_d,(4)) / 3, where e_d,(k) is the kth-lowest subset error among the hybrid functionals evaluated in the original GSCDB137 study [1]. It is an empirical reference error scale, not the standard deviation of the reference energies or the standard error of their mean. These constants are fixed across the models being compared. Our overall NER averages MAE_d/σ_d over the evaluated subsets. We have included this definition and the source citation explicitly in Section 2.3 and use D consistently in the equation. Reference methods. The high-level reference energies retain the protocols of their source benchmarks. Standardization applies to the DFT-derived inputs and reaction representation; we did not recompute all reference labels using one method. The rationale and specific protocol differences are addressed in our response to W3. Subset-selection algorithm. Appendix C specifies the quota-preserving genetic algorithm and its main settings. We have expanded the description in Appendix C to cover population initialization, fitness evaluation, tournament selection, category-wise crossover, within-category mutation, and stopping criteria. Skala version. We used Skala 1.0, initialized from microsoft/skala: Skala exchange-correlation functional We have identified this checkpoint explicitly in Section 3.1 and distinguish it from the randomly initialized models in Appendix F. The reported fine-tuning results concern this specific checkpoint and evaluation protocol.
[1] Liang and Head-Gordon, Gold-Standard Chemical Database 137 (GSCDB137): A Diverse Set of Accurate Energy Differences for Assessing and Developing Density Functionals, Section 4.1: https://doi.org/10.1021/acs.jctc.5c01380