- Rating
- 1(−1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 3(+1)
Read review
Paper summary
This paper introduces strategies to improve the downstream utility of predicted Hamiltonian or Density matrices. The paper is formulated as kind of a 'bag of tricks', so I'll comment on them separately. Note that I did not read the appendices unless mentioned.
(1) The problem addressed with the ON-loss has been treated quite extensively in the wavefunction alignment loss paper (I read Appendix F), and the additional contributions made here amount to minor reformulations and pointing out possible inconsistencies in the released code implementation, which are only moderately useful. The core idea behind the wavefunction alignment loss has been used in other work (QHFlow). I would disagree with the statement that supervision in the "ON basis" is poorly understood, at least for Hamiltonian prediction. The NON-basis is used for supervision in models which treat larger systems (eg, DeepH-pack, HELM) due to lower computational cost, with the limitations in mind.
(2) The issue of gauge invariance (here ROCKET) - for molecules, this should not be an issue in theory, because they share a common energy reference (the vacuum level). Maintaining this common reference allows for element-wise referencing procedures to be applied, as was done in MACE-H and HELM. The paper states, "a standard matrix loss would strictly penalize gauge variation between molecules, while Eq. (9) permits arbitrary gauge variation ", however I would expect to see no arbitrary gauge variation in this training data, and am confused why including this degree of freedom results in performance improvements during training. I would like to know the exact details behind how c- and c* are computed over a given dataset, whether element-wise referencing is included in the baselines, and more intuition behind why this helps, beyond ablations. This very interesting point is just skimmed over.
(3) The last point (GROOT) mentions that constrained learned density matrices to the manifold of 'valid' projectors improves learning. This makes a lot of sense, and I've seen similar ideas very recently elsewhere. However, it should be clear that this involves diagonalizing the predicted matrix, which limits scalability.
Critical strengths
I don't see a single, substantial strength in the paper. This might just be because the message is diluted into multiple smaller contributions which are confusingly written, so it's hard to come away with a clear 'takeaway'.
Critical weaknesses
The first significant weakness of this paper is that it's incredibly difficult to read. There is very little signposting, and walls of text which switch between different points. The frequent use of bold font and italic font is distracting. The contributions, ablations, and experiments are not in order or nicely differentiated. It takes a long time of staring at the figures to understand what is being shown, as the way the data is presented is overly 'synthesized'. It feels like this manuscript is not designed with a human reader in mind.
Questions for authors
(Please see the summary section)
I think satisfying the following would improve the overall volume of contribution in the paper to be a strong contribution:
(1) Showing evaluations on open-shell systems and heavier elements than CHNOF. The paper states, "Although a generalization to unrestricted systems is straightforward, this has not been covered by our evaluations. " so this should not be too difficult to evaluate.
(2) Quantifying evaluation runtimes, time complexity, and how they change with system size for each of the optimizations presented. Currently, the paper states that only a single Fock build is used, thus positioning this method in-between MLIPs (zero Fock builds), and multi-build Hamiltonian models. I think the paper should quantify this, to better understand any tradeoffs with scalability. This could be gathered from the existing data collected. It would also be good to understand how this compares with Hamiltonian models which do not require an initial matrix construction.
(3) Data scaling: What would take these contributions to the 'next-level', is if the performance improvements demonstrated are maintained even when larger amounts of training data is used, for example when training (not just evaluation) on fractions of the larger dataset used. This would place these observations into the regime of 'contributions which cannot be effectively replaced with sufficient training data'.
Minor points:
- Number of electrons should be replaced with numbers of atoms. Since heavy elements are already omitted, using # electrons doesn't add anything but makes the sizes difficult to compare with other work.
17 comments
Response to question (2) from summary
The reviewer is right that molecules share the vacuum level as a common energy reference. For absolute Hamiltonians this means that composition determines the constant shift of a molecule almost completely, which is why element-wise referencing as in MACE-H and HELM works there (App. H, Fig. 8, top).
Our Hamiltonian models, in contrast, learn the correction to the SAP guess, . The constant shift is then the offset of this -target, , which is the mean orbital-energy level of the true Hamiltonian relative to that of the guess. Measured from the guess instead of the vacuum, this offset is a property of the whole molecule: it varies between molecules of the same composition and grows with molecule size (Fig. 8, bottom). This is the gauge variation between molecules that the ROCKET loss permits, and ROCKET removes exactly this molecule-specific offset of , on which no observable depends.
In detail, the per-molecule shift is the closed-form minimizer of the loss, computed for every molecule at every training step:
The reference starts at the mean of over the training split and follows the model as an exponential moving average of the batch means:
The second-stage loss combines both:
The gradient of the first term is traceless and supervises everything except the shift; the gradient of the second term is a multiple of and moves the shift alone. The weight is set once from the spread of over the training split (App. H, Eq. 80; Table 17).
The revision includes element-wise referencing among the baselines in two forms (Tables 38–39): HELM-style per-element normalization of the scalar node blocks on absolute targets, and a per-element on-site reference on top of . On QM9 all variants are on par with ROCKET. On QM40 the restrained per-molecule shift of ROCKET is the most accurate, with 0.4 mHa energy MAE compared to 6.6 mHa for the per-element reference on .
The intuition for why this helps is that the offset of is a global quantity that a local model would have to assemble from per-atom outputs, and it changes with molecule size. With the offset removed, the scalar on-site blocks of H, C and O stay close to their training averages across molecule sizes and datasets (Fig. 9). The remaining target is local and transfers to larger molecules. The weak restraint towards keeps the constant in the model output fixed during training (Fig. 10), which makes the per-atom outputs agree on one level also for molecules larger than those seen in training.
App. H makes this connection between ROCKET and element-wise referencing quantitative and explains it in more detail.
| QM9 QM40 held-out test splits (16 ep., all molecules [kept]; Tables 38–39) | QM9 Test (2,000) [mHa] | QM40 (1,000) Rej. | QM40 [mHa] | QM40 [mHa] |
|---|---|---|---|---|
Abs. + per-elem. mean / , NON readout | 0.15 / 0.15 | 84.4% / 84.2% | 5,407 [3.00] / 6,585 [2.78] | 3,534 / 4,178 |
Abs. + per-elem. mean / , ON readout | 1.36 / 0.99 | 93.3% / 86.7% | 45.9 [5.75] / 47.2 [5.57] | 0.59 / 0.69 |
per-element reference (ON) | 0.042 | 12.5% | 6.72 [3.18] | 6.57 [2.47] |
single constant (ON, 3 seeds) | 0.132 | 0.97% | 3.07 [2.91] | 1.09 [0.90] |
| ROCKET stage 1 (unrestrained ) | 0.038 | 8.50% | 6.84 [3.61] | 5.07 [2.16] |
| ROCKET stage 2 (EMA-restrained , 3 seeds) | 0.040 | 0.07% | 1.71 [1.70] | 0.41 [0.40] |
| Values in brackets are averaged over the molecules kept after self-rejection. |
Response to question (3) from summary
It is correct that GROOT requires one eigendecomposition of a matrix to place the prediction on the manifold of valid projectors. This is the same cubic-cost operation that a conventional SCF calculation performs in every cycle. For the system sizes we tested, the cost of a ground-state descriptor model is set by the Fock build that turns the predicted density into observables: with direct four-center integrals, setup and model inference together take about 1–2% of the time of an SCF with forces, up to 182 heavy atoms and 3,378 basis functions (App. N.3, Fig. 20). Training time is the same as for the unconstrained density models (App. N.1, Fig. 17).
The projection also speeds up the Fock build when density fitting is used. Because GROOT outputs a rank- projector, the exchange contraction (RI-K) runs over the occupied orbitals instead of the full density matrix. In our timings up to 101 heavy atoms, this makes the Fock build 1.6–5.4x faster than for the unconstrained density model trained with (Fig. 19). In conlcusion we do not see limited scalability of our method withing the size regimes tested.
Response to question (1) from summary
We thank the reviewer for their response.
What we claim is:
The revised manuscript tests the second and third claim by separating the readout basis from the loss basis (App. R, Fig. 27). On QM9 → QM40, at one Fock build and averaged over all molecules:
NONONONNONONONFor Hamiltonians, predicting in the ON basis gives most of the gain, and LON roughly halves the remaining error. For densities, an ON readout under LNON fails completely, and only the combination of ON readout and LON improves on the NON baseline.
The idea is indeed known for Hamiltonians. What we add is to follow it through, which current practice does not yet do. WALoss adds its alignment term to elementwise terms in the NON basis (their App. K, Eq. 19). QHFlow trains in the NON basis and uses only the orbital-energy (diagonal) part of the alignment term for post-hoc fine-tuning (their Eq. 7).
It is a fair point that the NON Hamiltonian is more localized than its Löwdin counterpart, which matters for linear-scaling and periodic applications. It also explains the deliberate choice by HELM etc. to use the NON basis for their training. For density matrices the NON basis has no such advantage: both representations decay at nearly the same rate with atom-pair distance (App. G, Fig. 6).
The practical question is whether the weaker localization of the ON Hamiltonian hurts at the cutoffs the model uses. We keep the cutoffs of QHFlow2 for all models (
5 Åbackbone,15 Åreadout). At15 Åthe entries of the ON target have already dropped about four orders of magnitude below their typical size (Fig. 6), so the readout cutoff removes almost nothing. In line with this, the ON models keep extrapolating up to 319 heavy atoms at these cutoffs, while the NON baselines break (App. T).On cost, the Fock build dominates one-shot molecular observables: setup and model inference together take about
1–2%of the time of an SCF with forces (App. N.3), and training time is the same for all losses and readouts. The ON-to-NON transform adds one symmetric eigendecomposition of S per molecule, which has the same cubic cost as a single eigen-solve of an SCF cycle.Changes in the manuscript: readout vs. loss ablation (App. R, Fig. 27; Table 37), QHNet validation on QH9 (App. Q), cost breakdown (App. N).