acceptodds
Under review as a conference paper at ICLR 2027

CAREB: Chemical Accuracy Reaction Energy Dataset and Benchmark for Neural Density Functionals

Abstract

We introduce the largest openly available dataset for neural density functional theory, coupling 130 053 highly accurate reaction-energy targets to a unified reaction representation with DFT-level features for every molecular state. The dataset compiles and standardizes decades of DFA benchmarking across thermochemistry, kinetics, noncovalent interactions, barrier heights, atomization reactions, and DES15K dimer profiles, and is designed to support development of accurate DFT methods including neural functionals and ML -learning on top of DFA baselines. In this work, we instantiate this setting by finetuning the Skala neural XC functional and training -learning models in our benchmark, establishing strong baselines and highlighting chemical regimes where neural DFT still falls short of high-level accuracy. Additionally, a small representative subset of Gold Standard Chemistry Database was constructed by an evolutionary algorithm allowing model selection to be accelerated by orders of magnitude. By releasing all reference data, DFT-derived inputs, training code, and pretrained models, our work opens the way to fully reproducible development and systematic stress-testing of neural DFT architectures.

Primary area:
datasets and benchmarks
Keywords:
density functional theory, machine learning for quantum chemistry, reaction-energy benchmark, exchange-correlation functional, CCSD(T), Δ-learning, electronic-structure datasets

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_GfaLOpenReview ↗
Rating
3(+2)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2(+1)
Read review
Paper summary

The paper bundles CCSD(T)-quality reaction-energy targets from pre-existing benchmarks (GMTKN55, GSCDB137, RDB7, MSR-ACC/TAE25, and DES15K; spanning main-group and transition-metal thermochemistry, kinetics, barrier heights, atomization reactions, and non-covalent interactions) into a unified reaction representation and augment every molecular state with standarized PBE0/def2-QZVP KS-DFT auxiliary data (electron densities, density matrices, Hamiltonians, etc.). On top of this dataset, they train and evaluate several models:

  • fine-tuning the non-local neural XC functional Skala (including a seperate small-scale transition-metal run),
  • MACE-based delta-learning and adaptive-hybrid models on top of a B3LYP baseline,
  • XC enhancement-factor models and a DeePHF baseline trained from scratch.
Critical strengths

S1: Highly relevant and very useful

Highly relevant contribution with a lot of practical utility for a very active field that is currently lacking high quality easy-to-use datasets.

Critical weaknesses

W1: Claims about the model comparison do not quiet match actual experiments

The dataset is a strong core contribution, but the framing of the experiments should be scoped more carefully.

  • The conclusion claims a side-by-side comparison of three neural DFT paradigms "under exactly the same data and protocol". Only Skala is a neural XC functional.
    • Delta-MACE is an additive energy correction on B3LYP and alpha-MACE predicts a mixing coefficient.
    • The protocols differ (Skala is trained post-SCF on frozen PBE0 densities, the MACE models sit on a B3LYP baseline, the TM run is a separate fine-tuning with 221 training reactions).
  • Fig. 3 shows a single run per model without seeds or uncertainty estimates, so differences between models cannot be assessed.
  • The claim that it is possible "for the first time" to compare neural DFT architectures head-to-head is too strong. Multi-seed, multi-architecture ML-XC functional comparisons under a shared training protocol already exist, albeit on lower-level range seperated hybrid targets rather than coupled-cluster data. The novelty here is having much better CCSD(T)-quality reaction energy datasets from several established benchmarks paired with standardized DFT inputs, and the claims should be scoped to that. See more in Q5.

W2: Sometimes key information is completely missing.

  • Dataset table (see also Q4)
  • I coudn't find a clear definition of σd\sigma_d (page 4) which is part of the core "NER" metric that large parts of the paper are based on, is it just the standard deviation of the energy differences. Given that interpretation see Q6.
  • Have the reference methods also been homogenized? (See W3 under the assumption that the authors have done this.)
  • Appendix B might benefit from just including pseudo-code, but if you release that code together with the dataset, then just linking it for the final submission is fine.
  • Which version of Skala are you using? V1.0 or V1.1? (based on your citation I would guess V1.0?)
    • the update they released this year seems to be outperforming range seperated hybrids the one from previous year does not.
    • the difference is a small architecture tweak leading to longer range, and a larger training set afaik.

W3: Reference methods have not been homogenized. (?)

This is my most pressing concern The references come from different protocols:

  • GMTKN55 mixed CCSD(T)/CBS-type,
  • GSCDB137, RDB7 higher-level than most of the others, RDB7 is valence only with frozen cores?
  • DES15K: CCSD(T)/CBS relative energies, no absolute energies available?
  • MSR-ACC/TAE25: CCSD(T)/CBS level W1-F12-type protocol.

W4: No code in supplementary material for review.

Although the Reproducibility Statement links to the dataset and checkpoints, the training and evaluation code promised in the Abstract/Conclusion is not provided. Given that the training pipeline is dressed as a core contribution, it would be nice to be able to test the performance characteristics and througput of the pipeline.

W5: Strong-baseline claim of the second contribution seems not sufficiently supported.

A systematic empirical study of several neural DFT model variants, including nonlocal neural XC functionals [...]. We establish strong baselines.

The NN-XC models are trained and evaluated on the full holdout only post-SCF on a precomputed donated PBE0 density (with self-consistency only evaluated once on the 300-reaction slim subset, where NER actually degrades from 1.29 to 1.41). Training solely on fixed densities leaves the XC potential vxc=dExc/dρv_{\text{xc}} = d E_{\text{xc}}/ d\rho unconstrained and can introduce large density-driven errors (or require a costly hybrid PBE0 SCF calculation at inference time to donate the density) compared to self-consistent training/evaluation at scale. See e.g.

Questions for authors

I am willing to substantially increase my score if my questions below are adequately addressed and some of the claims are better scoped. In general, I enjoyed the paper but it seems very unpolished as is.

Q1: Absolute reference energy labels

Are absolute energies also released with the dataset or really only energy differences? -> This would extending the dataset easier.

Q2: Level of theory

Could you expand upon the rational behind PBE0 / def2-QZVP level of theory choice? Why not limit to triple zeta and add diffuse long range functions? Why PBE0 vs. other hybrids, or maybe even range seperated hybrids with dispersion corrections like wB97M-V?

In addition, we store the semilocal DFA part of the B3LYP Becke (1993) exchange-correlation energy EB3LYP xc [ρ] = ∫︁ εB3LYP xc [ρ(r), ∇ρ(r)] dr and the D3 dispersion corrections evaluated for B3LYP functional.

Do you perform this as a second DFT calculation or do you reuse the density from your PBE0 features? What about the reference methods, I would expect there to systemic errors in the computed energies that might hinder joint training on the entire dataset.

Q3: Unrestricted KS vs. Restricted KS

Why are you using UKS throughout and not just for open-shell systems? Woudn't RKS be both more efficient and more accurate for closed-shell systems?

Q4 Could you please add a Table in the appendix that gives a strucured summary of section 2.2. and figure 1a?

What is stored per sample?

  • Point cloud (R, Z)
  • ground state descriptors: PupPBE0P^{PBE_0}_\textrm{up}, PdownPBE0P^{PBE_0}_\textrm{down} what about the KS-Hamiltoinan
  • EXCB3LYPE^\textrm{B3LYP}_\textrm{XC} (which PP?)
  • high accuracy (CCSD(T) and beyond) relative energies. What about absolute energies?
  • Grid properties (Which grid level? Is it the standard pyscf grid level 3 quadrature grid?)
  • ...

Q5: Validation set design

By pruning filtering density grids from GSCDB137, arent you effectively filtering out large molecules? Recent works suggest, that size-extrapolation of learned functionals might be an issue especially for more expressive non-local architectures. Could you please add a small size extrapolation subset or is this to expensive to compute with wave function based methods even for just a handful of molecules?

Q6: Definition of σd\sigma_d

Is the distribution of the energy differences heavy tailed and are the mean errors typically dominated by individual samples? Also it might be worth here to compare to

Dick, S.; Fernandez-Serra, M. Highly Accurate and Constrained Density Functional Obtained with Differentiable Programming. Phys. Rev. B 2021, 104 (16), L161109. https://doi.org/10.1103/PhysRevB.104.L161109

where I first saw such a construction but they seem to normalize by the entire range.

Q7: Intercompatible are these reference calculation protocols

Do you have an idea how much the different reference calculation protocols disagree with each other? Is the relative-energy MAE on the order of mHa or μ\muHa etc.?

Q8: Could you also add the mGGA-level NN-XC functionals to the comparison?

Q9: Self-consistent training and evaluation of Skala

Section 4 notes that main training and holdout evaluations are performed post-SCF on frozen PBE0/def2-QZVP densities (please also state this explicitly in Section 3.1), while self-consistency is only mentioned once on the 300 reaction slim subset. Do you only evaluate self-consistently on the slim subset, or do you also train self-consistently (e.g., via the implicit function theorem or differentiating through the SCF solver)?

General remarks (did not negatively influence my recommendation)

  • Please mind the correct application of citet vs citep.
  • I would not characterise DM21 as state of the art and rather refer to the Nagai et al 2020 and Dick et al 2021 as early works on the topic.
  • On page 3 ". or noncovalent" you are missing an "F" (let a model proof-read for typos).
  • On page 4 (Please add numbers to equations) you define DactiveD_{active} but dont use it in the sum.
  • section 3 has exactly one subsection 3.1, you might wanna remove the subsection devision
14 comments
AuthorsOpenReview ↗

Responce. Part 1

Dear reviewer GfaL, thank you for your thoughtful review! Below we address all your points.

W1. The main experiment covers three distinct uses of CAREB: learning an XC functional with Skala, learning an additive energy correction with Δ-MACE, and learning an exchange-mixing coefficient with α-MACE. Appendices D–E specify the same main-group training splits and reaction-energy objective. However, the energy expressions and baselines are method-specific, and fine-tuning pretrained Skala differs from training the MACE models. We have therefore replaced “under exactly the same data and protocol” with an explicit description of these shared and method-specific elements, and described the experiment as a comparison of modeling strategies rather than three neural XC architectures. To assess variability across training runs, we have launched repeated Skala fine-tuning and training of both MACE models using three random seeds per method, keeping the main-group training, validation, and holdout reactions fixed. Once these runs are complete, we will update Figure 3 with the mean and standard deviation of the test-subset results. We have also removed the broad “for the first time” claims about head-to-head architecture comparisons from Sections 5 and 7. Our contribution is the open release of 130053 high-level reaction-energy targets from established benchmarks, paired with standardized DFT-derived inputs and organized in a unified reaction representation. The experiments demonstrate how this resource supports neural XC, Δ-learning and adaptive-hybrid approaches.

W2. Dataset structure. Table 1 summarizes the source datasets and target counts, while Section 2.2 describes the stored quantities. We have summarized this information together in a schema table in revised manuscript that separates molecular-state records from reaction records. The table specifies, for each molecular state, the nuclear charges and coordinates, total charge and spin multiplicity, spin-resolved PBE0 density matrices, the energy components Enuc​, E1​, EH​,​ and ExHF​, the spin-resolved densities on the numerical grid with grid coordinates and weights, the semilocal B3LYP XC energy evaluated on the PBE0 density, the B3LYP D3(BJ) dispersion correction, and basic metadata (number of atoms and grid points); and, for each reaction, the reaction ID, stoichiometry over molecular states, and the high-level relative reference energy. Definition of σ_d. We use the fixed normalization constants from GSCDB137, supplied in Standard_errors.csv. In that benchmark, the “standard error” for subset d is defined as σ_d = (e_d,(2) + e_d,(3) + e_d,(4)) / 3, where e_d,(k) is the kth-lowest subset error among the hybrid functionals evaluated in the original GSCDB137 study [1]. It is an empirical reference error scale, not the standard deviation of the reference energies or the standard error of their mean. These constants are fixed across the models being compared. Our overall NER averages MAE_d/σ_d over the evaluated subsets. We have included this definition and the source citation explicitly in Section 2.3 and use D consistently in the equation. Reference methods. The high-level reference energies retain the protocols of their source benchmarks. Standardization applies to the DFT-derived inputs and reaction representation; we did not recompute all reference labels using one method. The rationale and specific protocol differences are addressed in our response to W3. Subset-selection algorithm. Appendix C specifies the quota-preserving genetic algorithm and its main settings. We have expanded the description in Appendix C to cover population initialization, fitness evaluation, tournament selection, category-wise crossover, within-category mutation, and stopping criteria. Skala version. We used Skala 1.0, initialized from microsoft/skala: Skala exchange-correlation functional We have identified this checkpoint explicitly in Section 3.1 and distinguish it from the randomly initialized models in Appendix F. The reported fine-tuning results concern this specific checkpoint and evaluation protocol.

[1] Liang and Head-Gordon, Gold-Standard Chemical Database 137 (GSCDB137): A Diverse Set of Accurate Energy Differences for Assessing and Developing Density Functionals, Section 4.1: https://doi.org/10.1021/acs.jctc.5c01380

AuthorsOpenReview ↗

Responce. Part 2

W3. Thank you for your point. CAREB retains the reference energy differences from the original benchmarks and standardizes the DFT inputs and reaction representation. Using different reference protocols is a deliberate choice - the objective is to preserve accurate reference values across different chemical regimes. Recomputing everything with one method would require demonstrating that the replacement maintains or improves that accuracy. Moreover, this approach has direct precedents. GMTKN55 combines W4, W1-F12, W2-F12, CCSD(T)/CBS and other reference protocols [1, Table 1]. GSCDB137 likewise selects and updates references by subset, including core-valence and beyond-CCSD(T) corrections where available [2, Section 3]. These choices reflect the different requirements of atomization energies, reaction barriers and noncovalent interactions. A common method name alone would not ensure comparable errors across these quantities. The same principle applies to training. For example, ωB97M(2) was fitted using the training partition of MGCDB84, a collection assembled from numerous independently developed benchmark datasets [3]. Thus, combining curated reference sources is an established approach to functional development, as well as evaluation. DES15K provides a particularly concrete example. Its interaction energies combine MP2/CBS with a CCSD(T) correction evaluated in a system-dependent basis. The original authors tested this protocol against independent S66 and S66x8 reference values at the same geometries, obtaining mean unsigned errors of 0.07 and 0.05 kcal/mol, respectively [4, Technical Validation]. This is evidence that different reference protocols can yield closely agreeing targets; identical computational settings are not a prerequisite. Regarding absolute energies, the DES15K target is the interaction energy, E(AB) − E(A) − E(B). Our model predicts this same difference from the constituent states. Absolute high-level energies are therefore unnecessary for this training objective: the published interaction energy directly supplies the required label. Regarding RDB7, it uses frozen-core correlated reference calculations, and CAREB retains that published protocol. There are independent precedents for using this reference level in learned electronic-structure methods. The DeePHF reaction-modeling study in JACS Au (2025, DOI: 10.1021/jacsau.5c00541, Section 2.2.1) trains density-matrix-based energy corrections against CCSD(T)-F12a/cc-pVDZ-F12 energies, the same method and basis used in the original RDB7. Separately, the study 10.1002/sstr.70504 directly uses RDB7 reference energies to train corrections to methods including r²SCAN-3c. Our inclusion of RDB7 follows this established use of high-level reaction data for learning energy corrections. It adds extensive coverage of reaction barriers and reaction energies while preserving the source reference protocol. The rationale for including this component is therefore its demonstrated relevance to learned electronic-structure models and its coverage of reaction chemistry. We therefore distinguish reference accuracy from protocol uniformity. The appropriate test is whether the reference approximations are sufficiently accurate and compatible for the target quantities. Retaining the curated source references preserves their established accuracy assessments; imposing a single replacement protocol would not, by itself, improve the benchmark. [1] Goerigk et al. A look at the density functional theory zoo with the advanced GMTKN55 database for general main group thermochemistry, kinetics and noncovalent interactions (2017): https://doi.org/10.1039/C7CP04913G [2] Liang and Head-Gordon. Gold-Standard Chemical Database 137 (GSCDB137): A Diverse Set of Accurate Energy Differences for Assessing and Developing Density Functionals (2025): https://doi.org/10.1021/acs.jctc.5c01380 [3] Mardirossian and Head-Gordon. Survival of the most transferable at the top of Jacob’s ladder: Defining and testing the ωB97M(2) double hybrid density functional (2018): https://doi.org/10.1063/1.5025226 [4] Donchev et al. Quantum chemical benchmark databases of gold-standard dimer interaction energies (2021): https://doi.org/10.1038/s41597-021-00833-x [5] Liu et al., “Revisiting a large and diverse data set for barrier heights and reaction energies: best practices in density functional theory calculations for chemical kinetics,” Phys. Chem. Chem. Phys. 27, 13326–13339 (2025). https://doi.org/10.1039/D5CP01181G

W4. The training and evaluation scripts are included in the public release at https://dataset.chemdft.workers.dev/code/. The Reproducibility Statement links to the release root, which may have made the code less apparent. We have added the direct code link to the statement.

AuthorsOpenReview ↗

Responce. Part 3

W5. The main experiment evaluates models using the same type of precomputed PBE0 inputs supplied by CAREB. This is a useful benchmark setting in its own right: it allows model development without repeating the electronic-structure preprocessing for every experiment. Accuracy in this setting is the result being measured; self-consistent performance is assessed separately in Section 4. The reported SCF calculation converged for all required states and changed MAE from 0.84 to 1.00 kcal/mol (“1.84” instead of “0.84” was typo in the submitted manuscript and we changed the number in present text), while NER changed from 1.29 to 1.41. These results concern the slim validation subset. The cited density-corrected DFT literature distinguishes functional errors from density-driven errors; it does not establish that fixed-density evaluation is inherently less accurate. Energy fitting alone does not uniquely determine the XC potential, but the model’s parametrization and pretraining also constrain it. This distinction matters when developing a self-consistent functional, while leaving the value of the fixed-input benchmark intact. The PBE0 calculation is part of the cost of generating those inputs.

Q1 . Thank you for your question! CAREB releases high-level relative (reaction) energies as given by the source benchmarks. Absolute energies are not uniformly available across sources. Therefore, absolute energies are not included. Our training/evaluation label is the reaction energy difference as in other papers in the field.

Q2. We chose PBE0/def2-QZVP to generate reusable electronic-structure inputs, with particular attention to the density representation. The high-level reference energies provide the training targets independently of this choice. Our use of PBE0 is supported by density benchmarks. Medvedev et al. [1] compared 128 functionals on atomic and ionic systems and included PBE0 in their L1 group of methods producing the most accurate densities according to their criteria (Table 1). Their density-based analysis of PBE hybrids also yielded an optimal exact-exchange fraction of 26.3%, close to the standard 25% in PBE0. These results support PBE0 as a density donor. The stronger reaction-energy performance of ωB97M-V does not, by itself, establish that its density would be a better input for the learned models. Our choice concerns the input representation rather than selecting the most accurate standalone energy functional. The quadruple-zeta basis was selected for the same reason. Jakobsen and Jensen [2] showed that representing accurate reference densities using single-determinant orbitals can incur significant errors in double- and triple-zeta basis sets, principally because of insufficient higher-order polarization functions. In their tests, these errors became practically insignificant at quadruple-zeta quality or above. This provides a rationale for using a polarized quadruple-zeta basis in a resource intended for density-functional development. Diffuse functions improve the description of extended density tails, particularly for anions and weakly bound systems, but do not provide the same increase in polarization flexibility. An augmented triple-zeta basis therefore addresses a different aspect of basis completeness. We prioritized reducing representation errors through quadruple-zeta quality, while recognizing the value of diffuse augmentation for particularly diffuse systems. Since preprocessing is performed during dataset construction and reused across subsequent training runs, its additional computational cost is amortized over model development. The semilocal B3LYP XC contribution is evaluated on the converged PBE0 density. Accordingly, an additional B3LYP SCF calculation was not performed. Exact exchange is stored separately and included with the B3LYP mixing coefficient of 0.20. The density used for each term in the B3LYP-based baseline is PBE0, which we have clarified in Section 2.2. D3(BJ) is evaluated separately from the molecular geometry using the B3LYP damping parameters and requires no additional density calculation. Finally, the reference-protocol question is separate from input generation. CAREB retains the published reference energy differences from its source benchmarks; it does not construct targets by combining absolute energies from unrelated protocols. GMTKN55 and GSCDB137 themselves use subset-specific reference methods [3,4]. Retaining these curated references preserves their original accuracy assessments. Recomputing all labels with one method would introduce a new reference approximation whose accuracy would need to be established. The specific differences between source protocols and their relevance to joint training are addressed in W3.

[1] https://doi.org/10.1126/science.aah5975

[2] https://doi.org/10.1021/acs.jctc.0c01029

[3] https://doi.org/10.1039/C7CP04913G

[4] https://doi.org/10.1021/acs.jctc.5c01380

Reviewer_hjngOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces CAREB, a large-scale open-source dataset and benchmarking platform for neural density functional theory (Neural DFT). It aggregates over 130,000 highly accurate CCSD(T) reaction energy targets and pairs them with standardized, low-level DFT physical features (e.g., electron densities on numerical grids, density matrices, Hamiltonians) for each molecular state. The authors benchmarked multiple neural DFT paradigms on this platform—including a fine-tuned Skala neural functional, MACE-based Δ\Delta-learning, and adaptive hybrid models—aiming to address the critical bottleneck of closed-source data and the lack of unified, reproducible standards for fair comparison in the field.

This paper also has some limitations, such as limited limit improvement (especially for transition metals), and no new model of its own.

Critical strengths
  • Excellent Contribution to Data Infrastructure: The paper successfully bridges the gap between the high accuracy demands of quantum chemistry (CCSD(T) energy labels) and the standardized low-level input features required by machine learning models, filling a major void in the field.
  • High Value for Reproducibility and Open Science: By providing a fully open-source and standardized testbed, it breaks the barriers created by proprietary datasets of previous advanced neural functionals (e.g., DM21, Skala). This allows different neural DFT architectures to be rigorously compared head-to-head under identical conditions.
Critical weaknesses
  • Limited improvement over baseline models, particularly failing on Transition Metals:The experimental results reveal that feeding high-quality data does not universally transform model performance. For instance, the MACE-based Adaptive Hybrid model failed to improve absolute accuracy on main-group elements, even showing degradation (MAE worsened to 6.046 kcal/mol). More critically, in tests focusing on complex transition metal (TM) chemistry, even the powerful Skala model, after targeted fine-tuning on this dataset, still failed to outperform traditional strong hybrid functionals such as ω\omegaB97M-V and B3LYP-D3(BJ). This indicates a severe physical generalization barrier in current neural DFT approaches that cannot be overcome merely by data accumulation.
  • Lacks architectural innovation, contributing only as a dataset:The core positioning of this work is strictly confined to dataset construction and benchmark integration. All experiments demonstrated in the paper simply repurpose existing model architectures (e.g., Skala, MACE, DeePHF). The authors do not propose any novel network architectures, physical inductive biases, or mathematical mechanisms designed to solve the inherent fundamental challenges of neural DFT (such as delocalization errors, fractional charge problems, or better capturing multi-reference electron correlation). As a pure benchmarking paper, its contribution to algorithmic innovation is notably thin.
Questions for authors

Nothing

2 comments

Dear reviewer hjng, thank you for recognizing CAREB’s contribution to data infrastructure, reproducibility and open science. Below, we clarify the role of the baseline experiments and the contribution of this dataset-and-benchmark submission.

W1. We note that our paper is in the “Datasets and benchmarks” section. Fine-tuning reduces Skala’s MAE from 1.544 to 1.245 kcal/mol on main-group reactions and from 5.13 to 4.36 kcal/mol on TM reactions. It still trails the strongest conventional hybrids on TM, as explicitly reported in Section 4. However, the TM experiment uses 221 training and 28 test reactions and is not a data-scaling study. It therefore does not establish that the remaining accuracy gap “cannot be overcome merely by data accumulation.”

The failure of the tested adaptive-hybrid model to improve MAE is also a useful benchmark result. CAREB makes these limitations measurable and provides data for investigating them. Its value does not depend on every baseline outperforming existing functionals.

W2. CAREB was submitted under the primary area “datasets and benchmarks”. Its novelty lies in pairing high-level reaction-energy targets from established sources with standardized electronic-structure inputs and a unified reaction representation, and releasing these together for reuse. The experiments provide reference results and demonstrate applications of the resource across several modeling strategies.

Developing a new architecture or resolving the fundamental limitations of neural DFT is a different research objective. Using existing architectures here allows us to demonstrate the dataset’s utility without introducing an additional methodological contribution.

The “Objective of the work” point in the ICLR 2027 Reviewer Guidelines asks reviewers to consider the work’s objective when assessing its potential value and impact. With CAREB’s objective in mind, we ask that it be assessed on the quality, usefulness, and reproducibility of the dataset and benchmark it provides.

Thank you again for recognizing the value of CAREB as an open dataset and benchmark. If our clarifications address your concerns about the baseline results and the scope of the contribution, would you consider raising your score?

Reviewer_EuX5OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces CAREB, a dataset combining 130,053 high-level reaction-energy targets with standardized DFT electronic-structure features. It brings together established chemistry benchmarks and demonstrates their use through Skala fine-tuning and other neural models. A compact validation subset also supports more efficient model development.

Critical strengths

The dataset addresses a practical need by pairing accurate reference energies with the electronic-structure inputs needed to train neural density functionals. This could make future research easier to reproduce and compare.

The experiments demonstrate usefulness across several modeling approaches. Skala fine-tuning improves the main-group holdout MAE from approximately 1.54 to 1.25 kcal/mol. The authors also acknowledge cases where conventional methods remain stronger.

Critical weaknesses

The handling of overlapping reactions and shared molecular structures across data sources and splits needs clearer explanation. This matters for interpreting generalization.

Most evaluations use fixed PBE0 densities. The self-consistent results on the validation subset are encouraging, but independent test results would strengthen the evidence for practical use.

Questions for authors

How are overlapping reactions and shared molecular structures handled across training, validation, and test splits?

Are self-consistent results available on an independent test subset beyond the slim validation set?

2 comments
AuthorsOpenReview ↗

Official Comment by Authors

Thank you for recognizing the value of pairing high-accuracy reaction energies with standardized electronic-structure inputs for developing and evaluating neural density functionals.

W1 / Q1. We used curated source databases (GSCDB, MSR, RDB7, DES15K). As described for GSCDB, overlaps are actively resolved in source construction: “All identity reactions (with zero reaction energy) in MGCDB84 are removed” and “Some data sets are restructured to avoid overlap with other sets” [1]. We used the geometry-matching criterion described below to check for reaction overlap between the GSCDB, MSR, and RDB7 training subsets and the GSCDB validation and holdout sets.

For every molecular geometry we build a covalent graph: atoms are nodes labeled by element Z; an undirected edge (bond) is drawn if the interatomic distance satisfies rij ≤1.3(R i+Rj), with Cordero covalent radii. We matched the molecular graphs using VF2 [2] and required an atomic RMSD of at most 0.01 Å after Kabsch alignment [3]. Five of the 2,428 GSCDB training reactions matched entries in the GSCDB validation or holdout sets. Under the same criterion, we found no reaction-level matches between either GSCDB evaluation set and the MSR or RDB7 training subsets (2,428 reactions each). Excluding the matched holdout reactions changes the reported holdout metrics by less than 0.1% in MAE: Skala 1.245 to 1.245 kcal/mol, Δ-MACE 3.098 to 3.100 kcal/mol, and α-MACE 4.518 to 4.521 kcal/mol; the corresponding NER values are 1.442 to 1.441, 2.536 to 2.535, and 4.703 to 4.703. Appendix H documents these matches and reports reaction-level overlaps across the full MSR, RDB7, GSCDB, and DES15K datasets.

W2 / Q2 We have started a self-consistent evaluation on the full held-out test set (beyond the validation SCF test from submitted text). The run is in progress; once it finishes, we will add the results to the revised manuscript.

[1] J. Liang and M. Head-Gordon, arXiv:2508.13468v2 (GSCDB137), Sec. 3.1–3.2.

[2] Cordella et al., IEEE TPAMI 26, 1367 (2004) https://doi.org/10.1109/TPAMI.2004.75

[3] Kabsch, W. A solution for the best rotation to relate two sets of vectors. Acta Crystallogr. A 32, 922–923 (1976). https://doi.org/10.1107/S0567739476001873

AuthorsOpenReview ↗

Follow-up to W2 / Q2.

W2 / Q2. We have completed a self-consistent Skala/def2-QZVP evaluation of the fine-tuned checkpoint on the full holdout test set. SCF converged for 6083 of 6146 holdout molecular states. It did not converge for 63 states using standard PySCF DIIS SCF algorithm with energy tolerance of 10−6Eh10^{-6} E_{h}, the corresponding reactions are omitted below (3516 of 3646 reactions retained).

Holdout subset metrics:

MAE (kcal/mol)NER
post-SCF (PBE0 densities)1.181.42
SCF (self-consistent Skala)1.451.60

Self-consistency increases error relative to post-SCF, consistent with the slim validation subset (post-SCF MAE 0.84 / NER 1.29 and SCF MAE 1.00 / NER 1.41). We included this holdout SCF comparison in the revised manuscript.

Comments on the paper

1 comment

Dear reviewers, we are truly grateful for your thorough review, careful feedback and helpful discussion. Your constructive comments have significantly improved the manuscript. Also, we are grateful to the Area Chair for great work.

Reviewer EuX5: overlap and self-consistency. We have included an updated Appendix H where explained the overlaps for reactions in different sources and evaluation splits. Also, we have updated "The Results" section and now presented the completed Skala self-consistent evaluation alongside post-SCF results on the same holdout subset.

Reviewer GfaL: quality of references, consistency and experimental details. In a response to the request, we provided more substantial support for the combined use of CAREB reference protocols. We have collected published validations of these protocols against experiment and higher-level theory in Appendix I. Appendix H documents differences in reference energies for the same reactions in the different datasets. We discuss these sources' role and propose transfer learning for property-wise accuracy improvement in Discussion. We also explained shared inputs and different protocol per method, explicitly mentioned fixed-density training and added the SCF holdout evaluation. Multi-seed results are now reported in Figure 3, with means and standard deviations across independent training runs.

Speaking about reproducibility, we clarified the data schema and metrics, expanded the description of the subset-selection algorithm in Appendix C, and linked the training and evaluation code in the Reproducibility Statement.

Reviewer hjng, dataset-benchmark contribution and initial results. We have clarified role of CAREB and the roles of the baseline models in the Discussion and Conclusion of the revised paper. Fine-tuning of Skala improves its accuracy on both main-group and transition-metal reactions. We have also clarified the scope of these experiments and their implications for future model development.

Thank you again for your time and thoughtful feedback, which helped us strengthen the paper. We hope that the revised manuscript and additional results address your concerns and support acceptance of CAREB.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.