- Rating
- 2(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
OSE26 uses MACE and Graph2Mat to predict a symmetric, geometry-dependent AO-space matrix . This matrix enters the exchange energy through , recovering the usual scaled HF exchange when . Training uses reaction and atomization energies with precomputed B3LYP/def2-QZVP density and exchange matrices. The reported test MAEs are 0.85 kcal/mol on MSR-ACC and 1.23 kcal/mol on RDB7. Appendices include scalar, delta-learning, and diagonal-matrix baselines, a 30-reaction self-consistent check, equilibrium bond-length results, and water-cluster evaluations.
Critical strengths
The learned quantity has a useful physical structure. Predicting an exchange-modulation matrix provides a concrete connection between geometric learning and electronic-structure calculations. The scalar limit is explicit, and Appendix A derives the density derivative of the exchange energy for use in SCF. The potential contribution to machine learning is this structured parameterization and its physical readout.
Critical weaknesses
-
The evaluation protocol is insufficiently specified. Table 1 gives dataset counts without explaining subset selection, split construction, or the validation set used for model selection. Section 4.3 alternates between calling the evaluated data test and validation sets. For RDB7, it is important to report how forward/reverse barriers, shared transition states, and related reactions are distributed across splits. For MSR, structure identifiers and duplicate checks are needed. The settings and provenance of the conventional-functional baselines are also unclear. Table 2 reports the spread of test errors, not uncertainty across training runs; no repeated-run or paired uncertainty is given for the 0.15 kcal/mol RDB7 advantage over MACE delta-learning.
-
The ablations do not fully isolate the claimed machine-learning advantage. MACE and Graph2Mat are established components [1]. Both OSE and MACE delta-learning correct B3LYP energies. The OSE correction explicitly contracts with full precomputed density and exchange matrices, whereas the delta head predicts a scalar correction from geometry. Sharing an encoder does not match the information available at the energy readout, and the statement that the heads have similar sizes is not supported by parameter counts. The scalar baseline also varies a conventional hybrid mixing fraction, while OSE keeps the semilocal contribution fixed. A scalar restriction within the same energy expression would isolate the effect of matrix mixing more directly. The diagonal-only ablation needs a precise symmetry definition: elementwise diagonal AO matrices are generally not closed under rotations of p or higher-angular-momentum shells. The discussion should also position the contribution relative to equivariant learned functionals such as EG-XC [2].
-
The main benchmark does not establish the reported accuracy of the self-consistent functional. Training and the principal test results use frozen B3LYP orbitals. In this setting, the learned term is a structured energy correction contracted with fixed electronic quantities. The 30-reaction SCF check is useful, but it primarily reports agreement with post-SCF energies. It does not provide a clear comparison of post-SCF and self-consistent MAEs against the reference, or sufficient convergence statistics across chemical classes. The broader functional claims need to be distinguished from the demonstrated frozen-orbital prediction results.
-
The energy definition contains an omission that must be resolved. Equation (5) contains nuclear repulsion, kinetic, Coulomb, semilocal XC, learned exchange, and dispersion terms, but no explicit electron–nuclear attraction energy . The accompanying definitions identify as nuclear repulsion and as kinetic energy; the precomputation list in Section 4.1 omits the attraction term as well. The authors should give the complete energy decomposition used in the calculations and clarify whether this contribution is included under another name. The interpretation of also needs precision: Equations (2)–(3) use ordinary matrix multiplication to filter the density, which differs from assigning independent elementwise weights to individual exchange contractions.
-
The quantitative computational claims are unsupported. The paper claims an approximately two-order-of-magnitude cost advantage over the double hybrid, but provides no timings under common settings. The reported neural-network latency of less than one second excludes the baseline hybrid calculation and exchange construction. Appendix A explicitly requires two exchange builds per spin channel at each OSE SCF step. A hybrid-like formal cost is plausible; the claimed numerical speedup requires measurements. The RI-K statement also needs the contraction algorithm and its assumptions, including the treatment of occupied orbitals and screening.
-
The implementation and derivative claims cannot be checked from the supplied materials. The manuscript gives training and network settings but identifies no OSE26 code, weights, or split files. Appendix A involves , whose matrix argument is generally nonsymmetric; the PySCF handling of this intermediate is unspecified. Appendix B gives a force expression and geometry results without an energy–force finite-difference consistency check. It should state whether the analytic force applies to self-consistent OSE, post-SCF evaluation, or both, and specify the orbital-response and Pulay treatment. These details are central to reproducing the proposed functional.
[1] Febrer et al. Graph2Mat: universal graph to matrix conversion for electron density prediction. Machine Learning: Science and Technology 6, 025013 (2025).
[2] Gao, Eberhard, and Günnemann. Learning Equivariant Non-Local Electron Density Functionals. ICLR 2025.
Questions for authors
-
Please provide the split identifiers, subset-selection rules, overlap checks, and validation/checkpoint-selection procedure. State the source and computational settings of each conventional-functional baseline, the number of training runs, and paired uncertainty for the main ML comparisons.
-
Please specify parameter counts and the exact scalar-baseline energy expression. Clarify whether diagonal-only OSE means AO-element diagonality, atomic-block diagonality, or a rotation-invariant shell restriction. A focused scalar control using the same fixed semilocal term would be the most useful additional training comparison.
-
For the existing 30-reaction SCF subset, please provide identifiers, post-SCF and self-consistent MAEs against the reference, convergence thresholds, iteration statistics, and the largest energy difference. Add energy–force consistency checks on a few representative molecules.
-
Please resolve the energy decomposition in Equation (5), explain the exchange routine used for the nonsymmetric intermediate, and clarify code and model availability. Report representative end-to-end timings for B3LYP, post-SCF OSE, self-consistent OSE, and the double-hybrid comparator, or remove the unsupported quantitative speedup claim.
4 comments
Official Comment by Authors. Part 2
W3, Q3.
Main MSR-ACC / RDB7 MAEs (0.85 and 1.23 kcal/mol) are post-SCF on frozen B3LYP/def2-QZVP orbitals and matrices. We will label them as post-SCF in Sec. 4.3, Table 2, and the conclusions.
For the existing 30-reaction SCF subset (official RDB7 database indices; FBH: 17, 25, 44, 47, 56, 57, 61, 72, 94; RBH: 14, 15, 19, 22, 38, 41, 42, 47, 56, 64, 84; RE: 12, 22, 30, 35, 47, 48, 62, 68, 71, 98):
SCF timing and iteration statistics versus system size are reported for water clusters in the revised manuscript (Appendix D, Figure 8). Mean SCF iterations per system: B3LYP 7.4, OSE 7.4, ωB97X-V 7.0, DM21 8.9. We used convergence treshold.
Post-SCF MAE vs reference: 1.37 kcal/mol.
SCF MAE vs reference: 1.35 kcal/mol.
Largest : 0.143 kcal/mol (index 64, RBH).
Energy-force consistency is added in Appendix B, Figure 5 (analytical forces overlay finite-difference −dE/dR and potential energy curve).
Q4, W4
We have corrected Eq. (5).
Q4, W5
We remove the unsupported numerical speedup vs double hybrids. Remaining claim: hybrid cost class. NN latency <1 s excludes hybrid SCF and exchange builds. Self-consistent OSE needs two exchange builds per spin channel per iteration. RI-K follows the standard density-fitting exchange path in PySCF. We will not claim a numerical advantage over B97M(2) without matched timings. Regarding availability, we will publicly release the code and trained model upon publication. Their release is not mandatory at the time of submission.
W6, Q4
The nonsymmetric intermediate is handled with PySCF get_k(..., hermi=0). Analytic forces apply to self-consistent OSE (Appendix B); energy-force checks are under W3/Q3.
[2] https://github.com/dftd3/simple-dftd3
[3] https://github.com/Eipgen/BHpredict
Official Comment by Authors
If our responses have resolved the issues, would you consider raising your score?
Thank you for the responses, whcih address some of my concerns. I have raised my ratings.
Official Comment by Authors. Part 1
We thank the reviewer for the detailed critique. We address each point below and will update the manuscript accordingly.
W1, Q1.
Dataset splits.
RDB7: 3,902 train / 6,883 test reaction entries with no shared reaction IDs. Forward and reverse barriers of the same reaction, and their reaction-energy labels, remain in one partition. A geometry-overlap check (0.01 Å) finds no shared transition-state geometries; ~86% of test reactions reuse at least one reactant/product geometry from training. The split is therefore reaction-disjoint, not molecule-disjoint; this is not label leakage. Forward and reverse reactions are assigned to the same split: training or test. For MSR-ACC, we randomly selected 1,200 test structures using a reaction-ID-based split, keeping all entries with the same reaction ID in a single partition to prevent reaction-level overlap between training and test; we used this subset because the original databases provides reference energies but not the density and K matrices required by our model, making calculations for the full dataset substantially more expensive, while the selected subset is sufficient to support the main claims of the paper.
OSE26 and B3LYP-D3(BJ) on the same geometries/basis are computed by us (B3LYP/def2-QZVP + D3(BJ) [2]). MSR values for other methods were taken directly from original MSR database with similar basis set. RDB7 values were computed using PySCF UKS RIJK with energy tolerance of 10−6Eh, there are no double hybrids in official PySCF, we used ORCA 6.1.1 for ωB97M(2) with ωB97M-V orbitals. The standard deviations in Table 2 are the spread of per-example test errors, not uncertainty across training runs; we clarify this in the caption. OSE26 and MACE Δ-learning were each trained once. We have launched additional training runs with 3 different seeds for the ablation study and will include the results in the revised manuscript once they are completed.
W2, Q2
We respectfully disagree that the ablations fail to isolate the claimed advantage. The claim of the paper is not that MACE or Graph2Mat are new. The claim is that, under a shared encoder and matched capacity, parameterizing the correction as an AO-pair exchange modulation (OSE) improves over (i) adaptive global hybrid mixing and (ii) unconstrained scalar Δ-learning, and that off-diagonal AO-pair terms contribute beyond a diagonal restriction. Appendix C is constructed exactly for that comparison and nothing else.
MACE and Graph2Mat are deliberately used as established components. They are the shared implementation substrate, not the scientific claim. Keeping the encoder fixed and changing only the energy readout is the standard way to attribute differences to the readout.
Shared MACE (r_max=6.0, num_bessel=6, polynomial_cutoff=6, max_ell=2, num_interactions=2, hidden_irreps=8×0e+8×1o, MLP_irreps=8×0e, avg_num_neighbors=8.0, correlation=1).
OSE26: MACE + Graph2Mat (edge_hidden_irreps=4×0e+4×1o, basis def2-QZVP) total: 2.168.388.
Adaptive-α, Δ-learning: Shared MACE + MLP: Linear(16x0e, 652), 5×Linear(652,652), Linear(652,1) total: 2,166,933
We agree that elementwise diagonal AO matrices are generally not closed under rotations, we will state this limitation when interpreting the diag vs full comparison. To address the requested concern directly, we additionally trained and evaluated the scalar restriction inside the same OSE energy expression, with the B3LYP semilocal XC held fixed:
E=Eunxc+ExcDFA+a(Z,R)ITr(PK)
Test MAEs (kcal/mol):
The adaptive-α baseline predicts a system-dependent scalar α(Z,R) and evaluates the energy as
Eadaptive=Eunxc+α(Z,R)ExHF+[1−α(Z,R)]ExDFA+EcB3LYP+ED3(BJ).
In adaptive hybrid α baseline, α changes the weights of both exact and semilocal exchange. In OSE26, the semilocal contribution is held fixed. The Δ-learning head predicts a scalar correction to the B3LYP energy from atomic species and coordinates. OSE26 predicts A from the same inputs and contracts it with the frozen B3LYP density and exchange matrices to evaluate the exchange energy. The comparison therefore evaluates the full OSE26 energy readout, including its use of these electronic quantities
Relative to EG-XC [Gao et al., ICLR 2025], OSE learns an AO-pair modulation of hybrid exact exchange with an explicit SCF potential and analytical forces, rather than a learned nonlocal density-to-energy XC map.