acceptodds
Under review as a conference paper at ICLR 2027

Where the Error Lives: Spectral Representations of Learned Kohn-Sham Hamiltonians

Abstract

Kohn-Sham density functional theory (DFT) is the standard method for computing the electronic structure of molecules and materials, but its self-consistentfield (SCF) iterations make it expensive for large systems and high-throughput screening. Machine-learned Hamiltonian models offer a shortcut: an equivariant network predicts the Kohn-Sham Hamiltonian matrix from the atomic structure in one forward pass, and diagonalizing the prediction yields orbital energies, the HOMO-LUMO gap, molecular orbitals and the electron density, or an initial guess that accelerates DFT. These models are trained to match the reference matrix entry by entry, yet a small entry-wise error does not guarantee accurate eigenvalues and eigenvectors, which are what every downstream use requires. We represent the prediction error in the eigenbasis of the reference Hamiltonian, where it separates into sectors that each property, and each training signal, reads selectively. This representation explains why existing physics-informed losses help only partially: the wavefunction-alignment loss either weights all sectors uniformly or sees orbital energies only to first order, and the electron density is unchanged along about two thirds of the directions in which the prediction can be wrong, including every orbital-energy error. It also points to a representation of the Hamiltonian that sees every sector while staying in the language of the density: the cumulative local density of states (LDOS), the number of electrons at each point in space below each energy. We prove that it contains the density as a special case, responds to every component of the error, determines the Hamiltonian uniquely, and yields gradients that do not vanish for orbital energies far from their targets. Supervising a network through this representation requires no change to its architecture. Fine-tuning three Hamiltonian networks (QHNet, SPHNet and WANet) on QH9 in this way reduces the HOMO-LUMO gap error by 49-79%, 2.0-2.6 times more than the wavefunction-alignment loss, without increasing the element-wise error by more than 0.3%, and the gains carry over to out-of-distribution and molecular-dynamics splits (36-77%), where the wavefunction-alignment loss helps less and, on QHNet, makes the gap worse. These results show that how a predicted Hamiltonian is represented during training, not only the architecture that produces it, decides whether it is useful in practice.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
quantum chemistry, graph neural network, density functional theory

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_EXdnOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper examines why small Hamiltonian errors can produce inaccurate orbital energies and wavefunctions. In the reference orbital basis, the authors separate energy shifts, occupied–virtual mixing, and rotations within occupied or virtual subspaces. They introduce a Gaussian-smoothed cumulative LDOS loss to capture both spectral positions and spatial orbital densities. Fine-tuning QHNet, SPHNet, and WANet on four QH9 splits improves downstream predictions without changing the architectures. On stable-id, HOMO–LUMO gap MAE decreases by 49–79% relative to the pretrained models and 37–65% relative to the tested WALoss baseline, with little change in Hamiltonian MAE.

Critical strengths
  1. The physical motivation is clear. Density matching misses energy shifts when orbital shapes and occupations remain unchanged, which gives a concrete reason to retain energy resolution. The continuous-representation analysis provides a useful theoretical account under its stated assumptions.

  2. The experiments include shared starting checkpoints, matched fine-tuning protocols, and element-wise-only continued-training controls. Density-only, peak-LDOS, and WALoss comparisons help identify the effects of the auxiliary objective.

Critical weaknesses
  1. The relationship to existing objectives needs more attention. NextHAM [1] combines real-space and reciprocal-space supervision with separate weights for low-energy, high-energy, and cross-subspace errors. Cignoni et al. [2] jointly optimize eigenvalues obtained by diagonalizing the predicted Hamiltonian and Löwdin atomic charges. Suman et al. [3] extend this approach to additional electronic observables. These studies are relevant to the proposed combination of spectral and spatial supervision. The paper does not adequately explain what cumulative LDOS adds beyond these approaches.

    A direct predicted-eigenvalue loss combined with the existing density loss would be an informative comparator. Density-only supervision already captures most of the QHNet stable-id gap improvement: 2081 μHa versus 2027 μHa for cumulative LDOS. The additional gains on SPHNet and WANet are larger, approximately 19% and 28%. This variation deserves more discussion.

  2. The supplied evaluation code does not fully match the manuscript. Appendix E defines orbital similarity using the overlap-matrix inner product, ∣c^i⊤Sci⋆∣|\hat c_i^\top S c_i^\star|. In contrast, cumldos/metrics.py, line 35, computes the Euclidean cosine similarity of the coefficient vectors. These are different quantities in a nonorthogonal AO basis. The discrepancy leaves the definition of the reported ψ values unclear.

    The SCF timing protocol is also unclear. In scripts/scf_init.py, the generalized eigensolve and initial-density construction occur before the SCF timer starts. It is unclear whether this cost is included in the externally supplied t_inf. Appendix F states that Table 12 includes both the model forward pass and eigensolve, but the supplied script does not make that accounting explicit.

  3. The theoretical guarantees need to be separated from the properties of the sampled loss. Identifiability is established in continuous real space and over a continuous energy interval; training uses 256 spatial points and 64 energies. Appendix B acknowledges unresolved directions at the actual sampling density. The non-saturating restoring-force result also assumes a single displaced level with fixed orbital shape and is limited by the finite energy window. These conditions belong alongside the main claims.

    Proposition B.7 likewise does not establish that the increase in matrix error must be small. The small trade-off observed in the tables should be presented as an empirical result.

  4. The empirical scope and uncertainty remain limited. All experiments concern small, closed-shell H/C/N/O/F molecules at the B3LYP/def2-SVP level. The shifted splits do not test new elements, periodic systems, or SOC Hamiltonians. Broader chemical spaces have been studied with DeepH-2 [4], HamGNN [5], and NextHAM [1], while QHNetV2 [6] and QHFlow2 [7] provide more recent molecular predictors. Transfer of the proposed loss to these settings remains untested.

    The main comparisons lack repeated-seed uncertainty, which matters for the smaller margins. Training cost is appreciable: Table 13 gives approximately 49% overhead per fine-tuning step. The approximately 5% figure uses pretraining cost as the denominator. Table 12 shows little additional SCF acceleration over the pretrained model, so the practical case rests mainly on improved single-shot properties.

References.

[1] Yin et al. Advancing Universal Deep Learning for Electronic-Structure Hamiltonian Prediction of Materials. ICLR 2026.

[2] Cignoni et al. Electronic Excited States from Physically Constrained Machine Learning. ACS Central Science 10, 637–648 (2024).

[3] Suman et al. Exploring the Design Space of Machine Learning Models for Quantum Chemistry with a Fully Differentiable Framework. JCTC 21, 6505–6516 (2025).

[4] Wang et al. Universal materials model of deep-learning density functional theory Hamiltonian. Science Bulletin 69, 2514–2521 (2024).

[5] Zhong et al. Universal Machine Learning Kohn–Sham Hamiltonian for Materials. Chinese Physics Letters 41, 077103 (2024).

[6] Yu et al. Efficient Prediction of SO(3)-Equivariant Hamiltonian Matrices via SO(2) Local Frames. ICML 2026.

[7] Kim et al. Machine Learning Hamiltonians are Accurate Energy-Force Predictors. ICML 2026.

Questions for authors
  1. Which definition was used for the reported ψ values: the overlap-matrix inner product in Appendix E or the Euclidean coefficient cosine in the supplied code?

  2. Does the total time reported in Table 12 include the generalized eigensolve and initial-density construction? Please clarify what the externally supplied t_inf contains.

  3. Please clarify the relationship to the objectives in [1–3] and state the finite-sampling and energy-window conditions alongside the main theoretical claims.

A matched QHNet stable-id comparison with direct eigenvalue matching combined with the existing density loss would help establish the contribution of the cumulative LDOS representation. Any available repeated-run results would also help assess the smaller differences.

4 comments
AuthorsOpenReview ↗

Response to Reviewer EXdn - Part 1/3

Thank you for the careful review, including the code. We answer your three questions, then the weaknesses.

Q1 / W2: "Which definition was used for the reported ψ\psi values?"

The Euclidean cosine of the coefficient vectors. This is the definition of the QH9 benchmark, computed by the official evaluators of QHNet and SPHNet/WANet that produced every reported value, and the supplied metrics.py reproduces it. The overlap formula in Appendix E is our error and is now corrected there. With the overlap metric the values rise by up to 1.6 points and the ranking of the methods is unchanged (QHNet, stable-id: pretrained 97.85%, WALoss 98.02%, ours 98.61%).

Q2 / W2: "Does the total time reported in Table 12 include the generalized eigensolve and initial-density construction?"

Yes. In the script that produced Table 12 the timer starts before the forward pass and stops after the generalized eigensolve and the construction of the initial density matrix, so t_inf contains all three. The supplied scf_init.py was a simplified re-implementation that left the eigensolve outside its timer. Appendix F now states what the timer covers.

Q3 / W1: "Please clarify the relationship to the objectives in [1-3] [...]" and "The paper does not adequately explain what cumulative LDOS adds beyond these approaches."

  • NextHAM [1] projects the predicted and reference Hamiltonians onto low- and high-energy eigen-subspaces in kk-space and penalizes the blocks with separate weights. In our terms this is a block-weighted form of WALoss: a quadratic form in the error matrix that sees the orbital energies only through its diagonal (Lemma B.1 applies).
  • Cignoni et al. [2] and Suman et al. [3] learn an effective Hamiltonian indirectly, by fitting observables computed from it, rather than the reference Kohn–Sham matrix. [2] fits orbital energies and Löwdin charges, and [3] fits orbital energies together with dipoles, polarizabilities and bond orders. [2] reports that fitting orbital energies alone gives unphysical atomic charges (negative on hydrogen) and that adding the charges repairs them. Our analysis predicts exactly this: orbital energies constrain only sector A, and atomic charges are a coarse-grained density (sector B).
  • What the cumulative LDOS adds. It is the single quantity whose two projections are these targets: integrated over space it counts the orbitals below each energy, and at the Fermi level it is the density. Each projection alone leaves part of the error unsupervised, and the analysis says which part, as [2] found for orbital energies and charges. The cumulative LDOS supervises both projections in one term, with no relative weight between them to choose and without pairing predicted and reference eigenvalues by index.

We added this discussion to Related Work (Sec. 5), and we now state the finite-sampling and energy-window conditions next to the main theorem (W3 below).

AuthorsOpenReview ↗

Response to Reviewer EXdn - Part 2/3

W1: "A direct predicted-eigenvalue loss combined with the existing density loss would be an informative comparator. Density-only supervision already captures most of the QHNet stable-id gap improvement [...]. This variation deserves more discussion."

We ran it on QHNet, SPHNet and WANet (stable-id, same protocol, weights matched by gradient share) and show it next to the rows of Table 1 (Table 15 in the new Appendix G.8 has all eight metrics). Each term is added to the element-wise loss. All entries are mean absolute errors in μ\muHa except ψ\psi (%). Rows marked 3 seeds are means over three seeds, and the others are single runs. Bold marks the best downstream value in each block, and the matrix error H is not bolded.

QH9 stable-idHHOMOLUMOgapεocc\varepsilon_{\text{occ}}ψ\psi
QHNet
pretrained77.56104194459632120696.28
+ WALoss (3 seeds)78.068756124644867196.61
+ density76.827191575208135697.87
+ eigenvalue + density76.356711285181335397.96
+ cumulative LDOS (ours, 3 seeds)74.876471354186833697.98
SPHNet
pretrained36.603451309155438099.06
+ WALoss (3 seeds)37.32344929123528599.19
+ density37.6336368497014399.08
+ eigenvalue + density36.6830257080219899.25
+ cumulative LDOS (ours, 3 seeds)36.6429158280916899.24
WANet
pretrained40.073351454168428898.98
+ WALoss (3 seeds)41.003311015129615699.13
+ density40.21324823110114099.22
+ eigenvalue + density40.2529656077513899.23
+ cumulative LDOS (ours, 3 seeds)40.1628658080013799.23

The combination is on par with ours, as the analysis predicts: its two terms are the two projections of N(r,E)N(\mathbf r,E). It has the lower LUMO and gap errors, with the gap within about two of our seed standard deviations. Ours has the lower HOMO and occupied-orbital errors, with the HOMO lower by more than two of our seed standard deviations on every backbone. Both are far ahead of WALoss, and ours beats WALoss on every metric and backbone by more than the two seed spreads combined. What the cumulative LDOS adds is that it reaches this point with one term, while the combination needs a ratio between its two weights and an index pairing of predicted and reference eigenvalues.

The variation you point out for density-only has one source: molecules whose predicted frontier orbital is the wrong orbital (overlap with the reference HOMO, LUMO or LUMO+1 below 50%). In the pretrained QHNet these are 8.2% of the test molecules, and they carry 79% of its mean gap error, about 100 mHa each. Any loss that reads the spectrum in the energy window removes them, density matching included, so on QHNet density-only lands close to ours in the mean. On the remaining molecules density matching barely changes the gap error (−9% / −4% / +5% on QHNet / SPHNet / WANet), whereas ours lowers it by about a fifth on all three (−19% / −22% / −23%). SPHNet and WANet have far fewer such molecules (0.6–0.8%), so there the difference shows in the mean. The new Appendix G.7 gives this explanation. (The 2027 μ\muHa in the submitted paper is the run with w = 30. With w = 100, the weight used on the other QHNet splits, three seeds give 1868 ± 50. Table 1 now reports the w = 100 runs, and Table 9 reports both weights.)

AuthorsOpenReview ↗

Response to Reviewer EXdn - Part 3/3

W3: "The theoretical guarantees need to be separated from the properties of the sampled loss. [...] Proposition B.7 likewise does not establish that the increase in matrix error must be small."

Agreed on both. Next to Theorem 3.5 we now state its conditions. Identifiability and the absence of blind sectors hold for the continuous representation. At 256 points and 64 energies some directions are not resolved, mainly couplings between occupied orbitals and virtual orbitals above the window (Appendix B.3 measures them). The restoring-force bound assumes a single displaced level and a finite window. The unresolved directions are what the element-wise term supervises: removing it lowers the frontier errors further but multiplies the matrix error by 19 on QHNet and by 2.4–3.1 on SPHNet and WANet (see our response to WCpw). Proposition B.7 shows that a small increase of the matrix error is possible, not that it must be small. The observed change, at most +0.22% over the 12 settings (3 seeds), is now presented as an empirical result (Appendix B.4).

W4: "The empirical scope and uncertainty remain limited."

  • Seeds. All main comparisons now have three seeds (72 runs), reported as mean ± standard deviation in Tables 1 and 6. Ours has the lower mean gap error than WALoss in all 12 backbone × split settings, by more than one seed standard deviation in each. The smallest margin is WANet stable-ood (528 ± 18 vs. 575 ± 2 μ\muHa).
  • A more recent predictor. We added QHFlow (Kim et al., 2025), a flow-matching model whose matrix error on stable-id (23 μ\muHa) is a third of QHNet's, on all four splits (one seed, fine-tuned from the official checkpoints on the stable splits and from a pretraining with the official recipe on the dynamic splits). Our loss lowers its gap error on every split, by 12–42%, and is better than WALoss on three of the four (WALoss is better on dynamic-geo). On stable-id it leaves only 1 molecule with a wrong LUMO, against 18 for the official checkpoint and 9 after WALoss. QHFlow is in Table 1 and Figure 2, and its settings are in Table 18.
  • Larger molecules and new elements. ∇2\nabla^2DFT (up to 62 atoms, with S, Cl and Br, at ω\omegaB97X-D), detailed in our response to kmdt (Q3): the released QHNet has the wrong frontier orbitals for every test conformer, which element-wise fine-tuning and WALoss do not repair and our loss repairs for most of them (new Appendix G.9).
  • Periodic and SOC Hamiltonians are outside our scope: in metals there is no gap, so the energy window and the Fermi-level slice need a finite-temperature form. We state this as a limitation (Appendix I).
  • Cost. Per fine-tuning step the term adds 49% (0.72 s on QHNet). Over the 30k fine-tuning steps this is about 5% of the cost of the 300k-step pretraining, and inference is unchanged. We now give both numbers (Sec. 4.1 and Appendix G.6).
  • SCF. Agreed: the loss does not improve the SCF initial guess. This follows from the analysis: an initial guess uses only the occupied subspace (sector B), which the pretrained models already predict well, while our gains are in the orbital energies and the frontier orbitals. We present SCF acceleration as unchanged and the practical case as resting on the single-shot properties (Appendix G.5).

Thank you for the responses, whcih address most of my concerns. Overall, I keep my ratings.

Reviewer_WCpwOpenReview ↗
Rating
4
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The authors learn the hamiltonian which enables finding the electron density through diagonalization. They note that the element-wise matching of the hamiltonian does not emphasize accuracy in the most aspect of this problem: the eigenvalues and eigenvectors of the hamiltonian. They propose a representation of the hamiltonian, the cumulative local density of states, which can be tuned to handle relevant errors better and in a more targeted way.

Critical strengths
  • The paper clearly identifies limitations of element-wise, density only, and WALosses.
  • The paper proposes an alternative with numerically improved performance and a clear theoretical basis for why their alternative addresses the concerns laid out.
  • The paper is well organized and clearly written in a didactic style.
Critical weaknesses
  • The paper only fine-tunes models and does not try from-scratch training with their loss alone. I believe this is likely due to the high computational cost of training these models; nevertheless, it limits our understanding of the objective from the paper alone. I do not see it as grounds for rejection but it might limit the impact.
  • The paper needs to explain why the element-wise loss is still necessary, if it is. It would also be a good thing to ablate if there was time for more experiments, but I would still recommend acceptance if this was not done.
Questions for authors
  • The organization of the physics informed losses section seems a bit random. If the main point is to just go over the different options, maybe a table or bullets would be helpful? After a very clear introduction this paragraph seemed a bit disjointed to me. It also shows an important result, your density matching loss, but without much context. Could you reorganize this?
  • Can you clarify visually how ij, with i,j∈{O,V}i,j\in\{\mathcal{O}, \mathcal{V}\} , relates to the sectors A, B, and C?
  • You mention div(r) in eqn 3, but don't explain it. Can you make sure it is explained and referenced somewhere, even in the appendix. Why do you even use the element wise loss? Is it still necessary? Why? This must be
  • Table 1, can you plese make sure the words "error" appear. You mention the units, but it's not said that these are errors.
  • Table 1, you have +, but are they cumulative? I guess they are because you are using pretrained models, but you do not continue to use the WALoss during your finetuning? you should make sure that this is clear. I suppose density only loss is an ablation of your since your LDOS loss contains density only and other terms, right?
  • Figure 2, you could add an arrow to indicate which direction is better given that you are measuring here error reduction (with errors written directly above in the table).
  • Why not try HELM from Manasa Kaniselvan, et. al. 2026 & follow ups? Isn't it state of the art by far? It's also not mentioned in related work.
1 comment
AuthorsOpenReview ↗

Response to Reviewer WCpw

Thank you for the supportive and detailed review. We start with W2, for which we ran the ablation you suggested.

W2 / Q3: "The paper needs to explain why the element-wise loss is still necessary, if it is. It would also be a good thing to ablate [...]."

The cumulative LDOS, as sampled during training, constrains only part of the matrix: the orbitals inside the energy window, seen at 256 points per molecule. The element-wise term holds everything else: the deep occupied orbitals, the virtual orbitals above the window, and the directions the sampling does not resolve (Appendix B.3). The ablation (stable-id, our loss with and without the element-wise term):

QHNetSPHNetWANet
matrix error (μ\muHa): with / without74.9 / 143336.6 / 87.240.2 / 122.9
gap error (μ\muHa): with / without1868 / 950809 / 470800 / 474

Without the element-wise term the frontier orbitals improve further, but the matrix error rises 19-fold on QHNet and 2.4–3.1-fold on the others, and on QHNet the mean error of the occupied orbital energies grows from 336 to 53,956 μ\muHa, almost all of it in the core orbitals below the window. The two terms are complementary: the element-wise loss keeps the whole matrix close to the reference, and the spectral term corrects the part the properties read.

W1: "The paper only fine-tunes models and does not try from-scratch training with their loss alone."

We position the loss as a fine-tuning objective: it costs about 5% of pretraining and nothing at inference. The ablation above suggests that from scratch it should still be paired with the element-wise loss, since on its own it leaves the matrix outside the energy window unconstrained. We did not run from-scratch training and list it as future work (Appendix I).

Q1: "The organization of the physics informed losses section seems a bit random. [...] Could you reorganize this?"

We restructured the paragraph (Sec. 2) as a list of the existing spectral losses, WALoss (in its full and diagonal forms) and density matching, and it now points to Table 3 in Appendix B, which lists the sectors each loss sees. Density matching is introduced as the Fermi-level slice of the cumulative LDOS.

Q2: "Can you clarify visually how ijij, with i,j∈O,Vi,j \in \mathcal{O},\mathcal{V}, relates to the sectors A, B, and C?"

Figure 1d shows them as blocks of the error in the reference orbital basis: A is the diagonal Δii\Delta_{ii}, B the occupied–virtual blocks Δai\Delta_{ai} with i∈Oi \in \mathcal{O}, a∈Va \in \mathcal{V}, and C the off-diagonal entries Δij\Delta_{ij}, i≠ji \neq j, with i,ji, j both in O\mathcal{O} or both in V\mathcal{V}.

Q3: "You mention div(r) in eqn 3, but don't explain it."

It is the geometry-only core-suppression factor of LAGNet (Ushenin et al., 2025). It does not depend on the prediction: it is close to 1 except near the nuclei of the heavy atoms, where it grows large so that the density cusps do not dominate the residual. We now define it at Eq. 3 and give the formula in Appendix J.

Q4: "Table 1, can you please make sure the words 'error' appear."

Added to the column headers and the caption, in Table 6 as well.

Q5: "Table 1, you have +, but are they cumulative? [...] I suppose density only loss is an ablation of yours [...], right?"

Not cumulative: every row starts from the same pretrained checkpoint and adds one term to the element-wise loss, and our rows do not use WALoss. Yes, density-only is an ablation of our loss: the slice of N(r,E)N(\mathbf r,E) at E=μE=\mu. The caption now says both.

Q6: "Figure 2, you could add an arrow to indicate which direction is better."

Added. Figure 2 now also shows the seed spread and a QHFlow panel.

Q7: "Why not try HELM [...]? Isn't it state of the art by far? It's also not mentioned in related work."

HELM is a backbone, and our loss is a training signal that applies to any backbone that outputs a Hamiltonian, HELM included. HELM reports the lowest matrix-element errors on ∇2\nabla^2DFT, about 60 μ\muHa and 3–5 times below the previous best, and state-of-the-art errors among single-pass models on MD17. It has not been trained on QH9, and its trained weights have not been released, so there was no checkpoint for us to fine-tune. Training it on QH9 from scratch did not fit in the discussion period: its authors trained their ∇2\nabla^2DFT models for 1,536 GPU-hours and stopped before convergence. They also report only a weak correlation between matrix-element errors and eigenvalue errors on ∇2\nabla^2DFT, which is the problem our loss addresses. On ∇2\nabla^2DFT we find that the released QHNet has the wrong HOMO and LUMO for every test conformer, and our loss repairs most of them (see our response to kmdt, Q3). We now discuss HELM in Related Work (the submitted version cited it only in Appendix G.2).

Reviewer_kmdtOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper analyzes Hamiltonian errors in the reference orbital basis and separates their effects on energies, occupied–virtual mixing and rotations within subspaces. It then proposes cumulative-LDOS supervision, which retains spatial and energy information. The loss is added when fine-tuning existing Hamiltonian networks, without changing inference.

Critical strengths

I find the paper's idea to separate the error components and link to some physical quantities quite helpful and interesting. It also explains why some existing approach would fail for some Hamiltonian or orbital energy predictions.

Critical weaknesses

Although the authors tried for establish some mathematical formalism to make this paper look more solid and with more theoretical innovation, I still find this paper's contribution is incremental. See more information below

Questions for authors
  1. I really like the idea to attribute the errors to different physical quantities, but instead of just adding some terms in the loss function, would it be possible to encode these physical intuition into the model architecture development? This is probably a high-level question, but I think it worths thinking about.
  2. the most useful comparision to add is probably directly add some addition terms in the loss function. For instance, if the authors claimed that the orbital eneriges' errors are significant for Hamiltonian prediction, can one simply add them as addition loss terms instead of doing the LDOS loss proposed in this paper? Do you think the LDOS approach would outperform it?
  3. QH9 are quite small molecules, probably much easy for some proposed methods, do you expect to see similar results for larger molecules?
3 comments
AuthorsOpenReview ↗

Response to Reviewer kmdt - Part 1/2

Thank you for the review. Your central question is whether the formalism does real work or dresses up an extra loss term. We answer it with the observation the paper started from, predictions of the analysis that new experiments confirm, and the comparison you asked for.

W: "Although the authors tried to establish some mathematical formalism to make this paper look more solid [...], I still find this paper's contribution is incremental."

The method is deliberately simple: one extra term and no change to the architecture. The analysis is what selects this term. It also predicts, for each kind of training signal, which errors it fixes and which it leaves, and the new experiments below confirm these predictions.

  • Observation. Training QHNet further on the element-wise loss alone lowers its matrix error by 4% and raises its HOMO–LUMO gap error by 53%. On larger molecules (∇2\nabla^2DFT, Q3 below) the released QHNet has the wrong HOMO and LUMO for every test conformer, and element-wise fine-tuning lowers its matrix error by 25% without repairing any of them.
  • Where the error lives. On QH9, the pretrained models get the LUMO wrong for 5.9% / 0.42% / 0.51% of the stable-id test molecules (QHNet / SPHNet / WANet), and one part of the error causes it: the part that mixes occupied orbitals among themselves and virtual orbitals among themselves (sector C in the paper). For 81–90% of these molecules, adding only this part of the predicted error to the reference Hamiltonian already makes the LUMO wrong. Adding only the orbital-energy errors does so for 3–8%. Our loss removes almost all of these failures: the share falls to 0.14% / 0.010% / 0.015%, 6–26 times fewer than with WALoss (3.5% / 0.061% / 0.11%, 3 seeds each; new Appendix G.7).
  • Which training signal fixes what. The analysis predicts that matching the density cannot correct orbital energies and that energy-resolved supervision can. We test this on the stable-id test molecules whose frontier orbitals the pretrained model already gets right. Each entry is the change of their mean gap error relative to the pretrained model. Negative is better, and the best entry in each column is in bold (Table 14 in the revised paper).
training signal added in fine-tuningQHNetSPHNetWANet
none (element-wise loss only)+74%0%+3%
WALoss (3 seeds)+50%+15%+15%
density matching−9%−4%+5%
cumulative LDOS (ours, 3 seeds)−19%−22%−23%

Density matching repairs the molecules with a wrong frontier orbital, as any loss that reads the spectrum does, but it barely changes the energies of the other molecules. Our loss improves them by about a fifth on all three backbones, while element-wise training and WALoss leave them unchanged or make them worse. The analysis predicts each of these outcomes, and none of them was used to build it.

Q1: "would it be possible to encode these physical intuition into the model architecture development?"

It is a good direction, for example an output head that predicts the Hamiltonian in an eigen-decomposed form, and we now discuss it as future work (Appendix I). We chose the training signal because that is where the evidence points: the same architecture trained longer on the element-wise loss gets a worse spectrum (above), and the same loss works unchanged on four architectures of different design, from QHNet to the flow-matching QHFlow.

AuthorsOpenReview ↗

Response to Reviewer kmdt - Part 2/2

Q2: "can one simply add them as addition loss terms instead of doing the LDOS loss proposed in this paper? Do you think the LDOS approach would outperform it?"

We ran it: an eigenvalue loss on the predicted orbital energies within ±10 eV of the gap, added to the element-wise loss with a weight that gives it a gradient share comparable to ours, on 4 backbones × 4 splits of QH9. The new Appendix G.8 (Setting and Findings) reports this comparison.

On the orbital energies the two are on par, as the analysis predicts: the gap errors differ by −14% to +24%, and neither loss is consistently lower (sign test over the 16 settings, p = 0.2). Integrating N(r,E)N(\mathbf r,E) over space counts the orbitals below each energy, which is exactly what an eigenvalue loss compares, so the eigenvalue loss is the spatial projection of the cumulative LDOS.

That projection discards where the orbitals are in space, and the eigenvalue loss gives no signal on it. On the orbitals ours is consistently better: the orbital similarity ψ\psi is higher in 13 of the 16 settings and lower in 2 (sign test p < 0.01), and the density error is lower on all four backbones (stable-id, paired over the 13,084 test molecules, significant on three).

So, to your question: the cumulative LDOS matches a direct eigenvalue loss on the orbital energies and outperforms it on the orbitals. The eigenvalue loss supervises only the energies, while the cumulative LDOS supervises the energies and the orbitals in one term. Adding a density term to the eigenvalue loss brings it level with ours (EXdn's comparator, see our response there), at the cost of a second term and a ratio of weights to choose.

Q3: "do you expect to see similar results for larger molecules?"

Yes, and the case for supervising the spectrum grows with size.

  • QH9-stable-ood trains on molecules with at most 20 atoms and tests on molecules with at least 23. Our loss lowers the gap error by 67% / 48% / 50% (QHNet / SPHNet / WANet, 3 seeds), while WALoss raises it by 69% on QHNet (Table 6, Figure 2).
  • ∇2\nabla^2DFT contains drug-like molecules of up to 62 atoms with H, C, N, O, F, S, Cl and Br, computed at ω\omegaB97X-D/def2-SVP. The publicly released QHNet (trained on 10k of them) has a matrix error of 540 μ\muHa, yet its orbital similarity is 13%, its density error 11.9%, and its predicted HOMO and LUMO are the wrong orbitals for all 9,532 test conformers. Fine-tuning it with the element-wise loss or WALoss lowers the matrix error by 24–25% and leaves the spectrum as broken as before (similarity 13–16%). Fine-tuning with our loss raises the similarity to 72%, lowers the density error to 0.75%, and lowers the share of conformers with a wrong HOMO to 11%, at a matrix error 15% below the starting checkpoint. The weight is larger than on QH9 (w = 500) because the starting spectrum is far from the reference, and the results improve monotonically from w = 5 to w = 500 (one seed; new Appendix G.9, Table 16).
AuthorsOpenReview ↗

Invitation to reviewer kmdt to engage in our rebuttal discussion

Dear reviewer kmdt,

Thank you for your review, which greatly helps in improving our work. As the discussion period is coming to an end, we would like to invite you to join our rebuttal discussion, and we would love to know if our current rebuttal responses have addressed your questions about our paper.

Best

The authors

Comments on the paper

1 comment
AuthorsOpenReview ↗

General response to all reviewers

We thank the reviewers. Three questions recur: whether a simpler spectral term would do the work of the cumulative LDOS (kmdt, EXdn), whether the gains are robust (EXdn), and whether they hold beyond small QH9 molecules (kmdt, EXdn, WCpw). We ran the experiments below, and each response gives the details.

  1. Seeds. Every main comparison now has three seeds (3 backbones × 4 splits × {WALoss, ours}, 72 runs). Our loss has a lower gap error than WALoss in all 12 settings, by more than one seed standard deviation in each. Tables 1 and 6 now report the mean ± standard deviation, and Figure 2 shows the seed spread.
  2. Simpler spectral losses. On QH9 we ran a direct orbital-energy (eigenvalue) loss on 16 backbone × split settings, and EXdn's eigenvalue + density comparator on three backbones. Both targets are projections of the cumulative LDOS N(r,E)N(\mathbf r,E): integrated over space, NN counts the orbitals below EE, which is what an eigenvalue loss compares, and its slice at the Fermi level is the density. As this predicts, the eigenvalue loss is on par with ours on the orbital energies, where neither is consistently lower across the 16 settings. On the orbitals, which it does not supervise, ours is consistently better: higher orbital similarity ψ\psi in 13 of the 16 settings and lower density error on all four backbones. Eigenvalue + density is on par with ours. It needs two terms with a ratio of weights to choose and an index pairing of predicted and reference eigenvalues, while the cumulative LDOS carries both targets in one term and needs neither (new Appendix G.8).
  3. Where the error lives, measured. Frontier orbitals go wrong because of one part of the error: the part that mixes occupied orbitals among themselves and virtual orbitals among themselves (sector C). Of the stable-id test molecules whose LUMO a pretrained model gets wrong, 81–90% get it wrong again when only this part of the error is added to the reference Hamiltonian. Molecules with a wrong frontier orbital dominate the mean gap error of a pretrained model (QHNet: 8.2% of the test molecules carry 79% of it), and any loss that reads the spectrum in the energy window removes them. On all other molecules, our loss lowers the gap error by about a fifth on all three backbones, density matching barely changes it (−9% to +5%), and element-wise training and WALoss leave it unchanged or make it worse (new Appendix G.7).
  4. Larger molecules and a stronger backbone. On ∇2\nabla^2DFT (drug-like molecules of up to 62 atoms, including S, Cl and Br), the publicly released QHNet has the wrong HOMO and LUMO for every one of the 9,532 test conformers. Fine-tuning it with the element-wise loss or WALoss lowers its matrix error by 24–25% and repairs none of them. Fine-tuning with our loss raises the orbital similarity from 13% to 72% and lowers the density error from 11.9% to 0.75% (new Appendix G.9). On QHFlow, a recent flow-matching predictor with a third of QHNet's matrix error, our loss lowers the stable-id gap error by 42% and the number of molecules with a wrong LUMO from 18 to 1 (Table 1).
  5. Corrections. The reported ψ\psi follows the QH9 benchmark (Euclidean cosine), and the overlap formula in Appendix E, which was wrong, is corrected. The inference time in Table 12 includes the eigensolve, and Appendix F now says so.

The revised paper includes these results and corrections, with the changes highlighted in yellow. The new appendix sections are G.7 to G.9, and all other appendix letters are unchanged.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.