- Rating
- 3(+1)
- Confidence
- 4(+1)
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2(+1)
Read review
Paper summary
The paper extends surrogate functionals, a framework developed for machine-learned orbital-free DFT, from the electronic energy to arbitrary scalar molecular properties. The motivation is the Hohenberg–Kohn existence argument: since the ground-state density determines all ground-state properties, each property is in principle a functional of the density. For each target property, the authors train a dedicated network using the same pipeline as the energy surrogate, with the ground-state energy label replaced by the property label. On QM9, the surrogate models outperform geometry-only and density-input versions of the same architecture on all seven extensive properties, beating the prior state of the art on six. They also extrapolate substantially better to larger QM40 molecules.
Critical strengths
A clean controlled comparison. The three arms (geometry-only, density-input, surrogate) share architecture, training settings, splits, and at inference the same predicted density. The only difference is the training procedure. So the central finding is clear: surrogate training improves the model without giving it any extra information at inference.
Strong in-distribution results. On QM9, U0 error drops from 4.9 meV (density input) to 2.8 meV, below GotenNet-L (3.37 meV), with similar gains on U, H, G, ZPVE and Cv. Meaningful improvements on a heavily optimized benchmark, though from single runs.
Practical inference cost. Replacing ~100 minimization steps with two forward passes makes inference comparable to a standard model. Training cost is likely higher but not reported.
Size extrapolation. Per-atom U0 error stays roughly flat with molecule size for the surrogate, while it grows for all other models (Figure 4). This extends the earlier result of Wagner et al. for energies to other extensive properties.
The paper is well written and is easy to read and understand.
Critical weaknesses
The physical motivation doesn’t transfer. Only the energy obeys a variational principle. For other properties, Hohenberg–Kohn guarantees that a functional of ρ* exists, not that it is minimized at ρ*. A minimum can be imposed on any property (S[ρ] = Q* + ‖ρ − ρ*‖² is enough), but then the value at the minimum is just a learned function of the molecule, and information about the property still comes only from ground-state labels. Since the optimization loop is removed at inference, the model is effectively a density-input model trained with an extra regularizer (a hypothesis). The paper would be stronger framed this way.
Calibration is borrowed from the energy. The energy-bound loss was motivated by real Kohn–Sham energies rising quadratically around ρ*. No such data exists for other properties; the energy-calibrated bounds and GDI parameters are simply transferred by rescaling labels. For intensive properties there is also a conceptual mismatch: ‖p − p*‖² grows with molecule size, and the fixed GDI step needs per-coefficient gradients of order one, while the mean-pooled readout (Appendix A.4.1) makes each atom’s gradient scale as 1/N. No clear cost shows up in the results (on QM40 the surrogate is best for HOMO and LUMO), but the mismatch is not discussed.
The benefit is property-specific, and regressions are not discussed. For properties that are essentially linear in the density, the surrogate is clearly worse than the density-input model on QM9: µ 12 vs 7.9 mD, ⟨R²⟩ 480 vs 140 a₀². Table 2 reports this, but the text does not address it. A possible explanation is that gains follow energy-likeness rather than extensivity: all improved properties are energies or second energy derivatives (ZPVE, Cv, α). In QM9 the two groups almost coincide, so the claim “beneficial for all extensive properties” cannot be separated from this alternative. QM40 is mixed here (⟨R²⟩ is best with the surrogate).
No mechanistic analysis. The training constraints improve a model with identical inputs at inference, but nothing explains why. Possible reasons: the cache acting as constrained data augmentation, the stationarity and curvature constraints forcing a more structured use of the density, or the extensive bound suppressing size-dependent drift. Also, density input alone helps only a little (5.3 → 4.9 meV on U0 with beSEN) and not at all with eSEN (Table 3). The density seems to become useful mainly with the surrogate constraints, which deserves explanation.
Ablations are hard to interpret. In Table 6, results range from 2.8 meV (full setup) to 5.7 meV for one partial configuration, worse than geometry-only. The benefit seems to depend on specific combinations of components, with little discussion of which interactions matter.
Some QM40 baselines look anomalous. GotenNet’s HOMO errors (6300 and 4700 meV) are about ten times its LUMO errors (540, 520) and beSEN’s HOMO error (300 meV). Geometry-only and density-input beSEN have identical ZPVE errors (150 meV) with the same per-atom drift (~3 meV/atom, Figure 5), for a property that is roughly a sum of local contributions. If this is a systematic readout bias that the surrogate constraints suppress, part of the large QM40 margins (ZPVE 4.9 vs 150, Cv 170 vs 3100) comes from fixing a specific failure, not from better generalization in general. The QM9 comparison is the cleaner evidence.
Limited reproducibility. There is no reproducibility statement and no link to code or data. This matters because the density labels are new: they were computed by the authors at B3LYP/6-31G(2df,p) and fitted to an even-tempered basis (Appendix A.3), and small differences there may affect results. The QM40 test set cannot be reconstructed: binning and filters are given, but not how molecules were selected within each bin, and no identifiers are listed. Results are single runs without seeds or error bars, so it is unclear which differences in Table 2 are meaningful.
Unsupported claim. Section 3.6 says variational minimization and the direct density shortcut give “comparable performance,” but no results are shown.
The weaknesses that matter most for my decision come down to three points. First, a method that is well motivated for the energy is transferred to other properties without a physical reason why it should work there, and without another explanation of why it works in practice. Second, several issues with the results (the anomalous QM40 baselines, the missing seed variance, and the regressions on µ and ⟨R²⟩) leave me not fully convinced by the reported gains. Third, no code or data is provided for reproduction, which is especially relevant here because the density labels were computed by the authors and the QM40 test set cannot be reconstructed from the description. I would be happy to update my recommendation if the authors provide an explanation of why the approach works, and with it a clearer picture of its limitations, along with answers to the questions above.
Questions for authors
-
Did you train a density-input model on perturbed densities (similar to the cache) but without the surrogate losses? If it reaches similar accuracy, the gain comes from seeing more densities, not from the variational structure.
-
How were the GotenNet checkpoints evaluated (units, atomic references)? HOMO errors of 6300 and 4700 meV look like a mismatch. And why do geometry-only and density-input beSEN have identical ZPVE errors on QM40?
-
How large is the run-to-run variance for the results in Tables 2 and 3? Several differences (e.g., surrogate vs. GotenNet-L on U0, 2.8 vs 3.37 meV) may be within noise.
-
Why does the surrogate degrade µ and ⟨R²⟩ on QM9 (12 vs 7.9 mD, 480 vs 140 a₀²)? Do you have results on any extensive property that is not energy-derived, to test whether the benefit follows extensivity or energy-likeness?
10 comments
Answer to Reviewer xwgv (continued)
Q3. Run-to-run variance
"Results are single runs without seeds or error bars, so it is unclear which differences in Table 2 are meaningful."
"How large is the run-to-run variance for the results in Tables 2 and 3? Several differences (e.g., surrogate vs. GotenNet-L on U0, 2.8 vs 3.37 meV) may be within noise."
We retrained all models with three seeds and now report mean and standard deviation in almost all tables (Tables 2–6, 10, 11 and 13; the block ablation of the beSEN components in Appendix A.8 uses one seed per combination). The seed standard deviations are small compared to the differences we discuss. Differences that are within the seed variance are now marked as ties.
Table. MAE on the QM9 test split, mean and standard deviation over three seeds. All models that use a density take the predicted ground-state density as input. GotenNet-L is reproduced as published. The best value per row is bold, ties are bold together.
| Property | Geometry only | Density input | Surrogate | Auxiliary surrogate | GotenNet-L |
|---|---|---|---|---|---|
| U0 (meV) | 4.23 ± 0.09 | 4.14 ± 0.06 | 2.18 ± 0.09 | 2.51 ± 0.02 | 3.37 |
| U (meV) | 4.20 ± 0.06 | 4.07 ± 0.02 | 2.12 ± 0.02 | 2.4 ± 0.1 | 3.41 |
| H (meV) | 4.17 ± 0.04 | 4.07 ± 0.02 | 2.12 ± 0.06 | 2.4 ± 0.1 | 3.30 |
| G (meV) | 5.52 ± 0.06 | 5.47 ± 0.03 | 3.6 ± 0.1 | 4.00 ± 0.06 | 4.98 |
| ZPVE (meV) | 1.022 ± 0.004 | 1.0152 ± 0.0007 | 0.903 ± 0.007 | 0.901 ± 0.002 | 1.08 |
| α (m a0^3) | 33.9 ± 0.3 | 33.3 ± 0.6 | 23.1 ± 0.3 | 23.2 ± 0.2 | 28 |
| Cv (mcal/mol/K) | 19.94 ± 0.10 | 19.5 ± 0.1 | 16.1 ± 0.2 | 16.0 ± 0.3 | 19 |
| HOMO (meV) | 13.7 ± 0.2 | 13.7 ± 0.1 | 14.0 ± 0.3 | 11.1 ± 0.6 | 13.4 |
| LUMO (meV) | 13.5 ± 0.7 | 13.2 ± 0.5 | 13.7 ± 0.8 | 12.2 ± 0.6 | 12.2 |
| gap (LUMO − HOMO) (meV) | 20.5 ± 0.5 | 20.1 ± 0.2 | 20.7 ± 0.7 | 17.6 ± 0.8 | 19.8 |
| μ (mD) | 6.8 ± 0.5 | 6.52 ± 0.08 | 8.3 ± 0.5 | 5.2 ± 0.4 | 6.7 |
| ⟨R²⟩ (m a0^2) | 15 ± 1 | 13.1 ± 0.5 | 11.91 ± 0.10 | 11.0 ± 0.5 | 24 |
Q4. µ and ⟨R²⟩ regressions; extensivity vs energy-likeness
"The benefit is property-specific, and regressions are not discussed. For properties that are essentially linear in the density, the surrogate is clearly worse than the density-input model on QM9: µ 12 vs 7.9 mD, ⟨R²⟩ 480 vs 140 a₀². Table 2 reports this, but the text does not address it. A possible explanation is that gains follow energy-likeness rather than extensivity: all improved properties are energies or second energy derivatives (ZPVE, Cv, α). In QM9 the two groups almost coincide, so the claim “beneficial for all extensive properties” cannot be separated from this alternative. QM40 is mixed here (⟨R²⟩ is best with the surrogate)."
"Why does the surrogate degrade µ and ⟨R²⟩ on QM9 (12 vs 7.9 mD, 480 vs 140 a₀²)? Do you have results on any extensive property that is not energy-derived, to test whether the benefit follows extensivity or energy-likeness?"
With three seeds and the updated training setup, the surrogate model no longer degrades ⟨R²⟩ (11.91 ± 0.10 compared to 13.1 ± 0.5 m a₀² for the density-input model), but it still degrades µ (8.3 ± 0.5 compared to 6.52 ± 0.08 mD). We attribute this degradation to surrogate losses contradicting the actual behavior of the property.
We therefore added the auxiliary surrogate model, where the surrogate is trained on a separate head of the same network, while the property head is trained only on the labels. The auxiliary surrogate resolves the regressions and gives the best results for µ (5.2 ± 0.4 mD), ⟨R²⟩ (11.0 ± 0.5 m a₀²), HOMO, LUMO and the gap, while it is on par with the surrogate for ZPVE, Cv and α (Table 2 and Section 4.1 of the revised manuscript).
Regarding extensivity vs. energy-likeness: QM9 contains no size-extensive property that is not derived from the energy, so the two hypotheses cannot be fully separated on this dataset. However, the auxiliary surrogate improves on all properties compared to the density-input model, also the non-energy properties. The benefit of the surrogate training therefore does not require the property to be energy-like, what fails for non-energy properties is placing the surrogate loss on the property output itself.
Answer to Reviewer xwgv (continued)
Q5. Physical motivation does not transfer; reframe as a regulariser
"The physical motivation doesn’t transfer. Only the energy obeys a variational principle. For other properties, Hohenberg–Kohn guarantees that a functional of ρ* exists, not that it is minimized at ρ*. A minimum can be imposed on any property (S[ρ] = Q* + ‖ρ − ρ*‖² is enough), but then the value at the minimum is just a learned function of the molecule, and information about the property still comes only from ground-state labels. Since the optimization loop is removed at inference, the model is effectively a density-input model trained with an extra regularizer (a hypothesis). The paper would be stronger framed this way."
We agree with the reviewer and adopted this framing in the revised manuscript (introduction, Section 3.3 and discussion in Section 5). For properties other than the energy, the surrogate does not follow from a variational principle, and we now describe it as a regularizer of the shared network. This view is supported by our new results: the auxiliary surrogate, which does not touch the property output at all, gives similar or larger improvements for most properties, and perturbed densities without the surrogate losses do not help (see Q1). The energy remains the case where the surrogate is physically motivated.
Q6. Calibration borrowed from the energy
"Calibration is borrowed from the energy. The energy-bound loss was motivated by real Kohn–Sham energies rising quadratically around ρ*. No such data exists for other properties; the energy-calibrated bounds and GDI parameters are simply transferred by rescaling labels. For intensive properties there is also a conceptual mismatch: ‖p − p*‖² grows with molecule size, and the fixed GDI step needs per-coefficient gradients of order one, while the mean-pooled readout (Appendix A.4.1) makes each atom’s gradient scale as 1/N. No clear cost shows up in the results (on QM40 the surrogate is best for HOMO and LUMO), but the mismatch is not discussed."
We agree that the calibration of the surrogate losses (the energy bound, the GDI step and the target scale) is taken from the energy and transferred to the other properties by rescaling the labels. Following the reframing in Q5, we view these settings as hyperparameters of the regularizer rather than as a physical model of the property away from the ground state.
Regarding intensive properties, we agree with the reviewer: with the mean-pooled readout the per-atom gradients decrease with molecule size, so the fixed GDI step and the bound act differently on small and large molecules. The auxiliary surrogate avoids this mismatch by construction: it uses no bound, applies GDI to a separate head, and trains the property head only on the labels, so no calibration taken from the energy is imposed on the intensive property. As shown in Q4, it gives the best results for the intensive properties on QM9.
Answer to Reviewer xwgv
We thank the reviewer for the detailed and constructive review. In the revision, all results are now reported as mean and standard deviation over three seeds, we added control experiments for the data-augmentation explanation, a new auxiliary surrogate variant that resolves the regressions on the non-extensive properties, an analysis of why the approach works, and results on a second level of theory (PBE). A summary of all new results can be found in the general response.
Q1. Density-input model trained on perturbed densities, without the surrogate losses
We trained the density-input model on perturbed densities with the ground-state label and without any surrogate loss, with three seeds each. As perturbed densities we use the MINAO initial guess, which replaces the ground-state density as input for half of each batch.
This control does not improve over the density-input model: U0 on QM9 is 4.15 ± 0.05 meV, compared to 4.14 ± 0.06 meV for the density-input model and 2.18 ± 0.09 meV for the surrogate model. On QM40 it is even worse than the density-input model (190 ± 10 meV compared to 150 ± 10 meV), and HOMO does not improve either. We also tested weight decay as a conventional regularizer, which does not reach the surrogate either. The gain of the surrogate model therefore does not just come from seeing more densities. The results are shown in the table below and in Appendix A.6.
Table. Mean absolute error of U0 and HOMO on the QM9 test split and on QM40, mean and standard deviation over three seeds. All models use the predicted ground-state density as input. The controls are density-input models trained on perturbed densities or with weight decay, without any surrogate loss. Best value per column in bold.
Q2. QM40 baselines: GotenNet evaluation, identical ZPVE errors
We evaluated the released GotenNet checkpoints with the code provided by the authors. To check our evaluation (units and atomic references), we evaluated the checkpoints on GotenNet's own QM9 test split and reproduce the published numbers. The large HOMO errors on QM40 are real and follow from GotenNet's readout.
GotenNet predicts both HOMO and LUMO as a sum of per-atom contributions. For an intensive property, the network therefore distributes the mean value of the property over the atoms, which no longer adds up for the larger molecules of QM40: the HOMO errors grow linearly with the number of heavy atoms, whereas for LUMO, whose mean value is close to zero, this drift is much smaller.
Regarding the identical ZPVE errors: on QM40 the geometry-only and the density-input model share a size-dependent drift in their ZPVE predictions, which the surrogate models largely reduce. We agree that part of the QM40 margin for ZPVE and Cv comes from removing this drift. However, the surrogate also improves properties without such a drift, e.g. the HOMO energy on QM40, and it shows clear gains on QM9.